My Blog

One thing I learned from my work is that even though having proper table definition is good for the data standardization, sometimes we just want to have the data be ingested to Snowflake without worrying about if all the column types are 100% correct or not. The following is the code I use for this case which can help me easily ingest data into snowflake with only 4 main required parameters, SOURCE_NAME, TARGET_NAME, FILE_FORMAT, and ALL_COLUMN_TYPE_AS_STRING.

Wed, Oct 11, 2023 is National Coming out Day. I did not realize that until I received an email from my HR to ask if I would love to share my personal coming out story in the panel session on that day. Even though, I won’t be able to join the panel in person because of my planned trip, I decided to tell my story through the recording and hope I still can bring some love to the audience in my video.

Few days ago, I asked myself a question about how can you better manage all the database changes defined in SQL scripts with DDL (Data Definition Language)? Maybe try to use version control on the SQL code and then use DevOps pipelines to achieve continuous deployment. Then, I came up with a poc solution, choux130/DevOps_In_DE/jenkins_mysql_flyway. This is the architecture overview, Here are some undo works on this poc and I wish I can improve it in the future.

In 2017, I started my first job as an Engineer Intern in the Predictive Analytics team, and after a year, I was promoted to Data Engineer. But to be honest, I was never sure if the titles I got can really describe my role in ChannelAdvisor since I was the first Data Engineer in our company. There were no clearly defined responsibilities in this role at the beginning and the responsibilities changes based on the tasks I did.

The problem I faced I am a R user who love using R Shiny to do any interactive visual related projects. My colleague who is more familiar with Python love to use Dash to do his dashboard tasks. One day, our manager asked us if we two could work together to create an internal dashboard. We looked at each other and started thinking which path is easier, have me learn Dash from the beginning or have him learn R Shiny from the beginning?

Version control or source control is a tool that helps users keep track of all the changes on files in a project. It is like a time capsule, it allows users to go back to the versions saved before. That is, no matter what horrible mistakes you make on your files, you can always go back to the last clean state and pretend nothing happened. It is not only good for personal use but also for a group of people collaborating together on the same project.