All Stories

Trust, but automate

If you want to enforce a technical guideline or rule, you must automate the enforcement mechanism. Relying on human beings to do the enforcement is too error prone, yet this...

How to paint a bike shed: Kafka topic naming conventions

Today, I’ll be tackling the controversial subject of Kafka topic names. Not only will I review various schemes, but I’ve decided to take a stand on a few issues and...

Choosing where to work

Making the right decision about where to work can have a profound effect on one’s life. A company needs to want you to join, but you also need to want...

One big cluster, or many small ones?

I bumped into something recently that seems to recur at every company I work for. Should we run one big cluster, or many smaller ones? The discussion is usually triggered...

So, you want to build a Kafka Connector? Source edition.

I’ve been talking to some of the folks at Data Mountaineer about their new Cassandra CDC connector for Kafka connect, and I wanted to record some of the nuances that...

Using YARN with Cgroups

I’m still a novice with Cgroups, but I thought it would be worth documenting how to set YARN up with them, since there seems to be a surprising lack of...