Few days ago, I asked myself a question about how can you better manage all the database changes defined in SQL scripts with DDL (Data Definition Language)? Maybe try to use version control on the SQL code and then use DevOps pipelines to achieve continuous deployment. Then, I came up with a poc solution, choux130/DevOps_In_DE/jenkins_mysql_flyway. This is the architecture overview, Here are some undo works on this poc and I wish I can improve it in the future.
Read MoreOh! I think I am a Recovering Data Scientist!
In 2017, I started my first job as an Engineer Intern in the Predictive Analytics team, and after a year, I was promoted to Data Engineer. But to be honest, I was never sure if the titles I got can really describe my role in ChannelAdvisor since I was the first Data Engineer in our company. There were no clearly defined responsibilities in this role at the beginning and the responsibilities changes based on the tasks I did.
Read MoreCombine R Shiny and Dash using Docker Compose
The problem I faced I am a R user who love using R Shiny to do any interactive visual related projects. My colleague who is more familiar with Python love to use Dash to do his dashboard tasks. One day, our manager asked us if we two could work together to create an internal dashboard. We looked at each other and started thinking which path is easier, have me learn Dash from the beginning or have him learn R Shiny from the beginning?
Read More