All Stories

Don't tell people what to do, ask them how to do it

When I first joined LinkedIn, there was very little automation in production deployments. Believe it or not, each deployment day had a wiki page with a run book of all...

Kafka change data capture breaks database encapsulation

Realtime change data capture (CDC) is becoming a popular architecture for data integration and data pipelines. The rise of Kafka connect, in particular, has triggered a lot of interest in...

Models and microservices should be running on the same continuous delivery stack

I’ve been interested in data science platforms for a long time. My fascination began when I was at LinkedIn, and helped build out the first model building and deployment system...

Your First Technical Presentation

Congratulations! You’ve been picked you to give a technical presentation. People are interested in what you have to say, and you’re excited and nervous. What now?

Trust, but automate

If you want to enforce a technical guideline or rule, you must automate the enforcement mechanism. Relying on human beings to do the enforcement is too error prone, yet this...

Using YARN with Cgroups

I’m still a novice with Cgroups, but I thought it would be worth documenting how to set YARN up with them, since there seems to be a surprising lack of...