Software engineering stuff
Using YARN with Cgroups
I’m still a novice with Cgroups, but I thought it would be worth documenting how to set YARN up with them, since there seems to be a surprising lack of...
14 Jun 2013
What follows is a brain dump of some things that I’ve learned about the way that software companies are sold on the public and private market.
22 Oct 2012
Kafka Consumer Memory Tuning
Yesterday, I had a process that was consuming a single Kafka topic. I was running it in our “staging” environment, and everything worked great. My heap space for the process...
05 Oct 2012
Schedules & Scores API for Streaming Live Sports Stats
Recently, I’ve been looking for an API that exposes schedules and scores for “the big four” American leagues. Here’s what I was looking for:
29 Sep 2012
Killing Subprocesses in Linux/Bash
Lately, I’ve been working with YARN at LinkedIn. This framework allows you to execute Bash scripts on one or more machines. It’s used primarily for Hadoop. When using YARN, you...
25 Sep 2012
Sorting Reducer Input Values in Hadoop
I HIGHLY recommend that you read the email thread by Owen O’Malley that describes this technique in brief. I should also note that this example is using the 0.18 Hadoop...
13 Nov 2009
Never miss a
from me, subscribe to my newsletter