How an audacious Hack Week project saved Dropbox millions

Not only did Pied Piper, led by Daniel Horn, win prizes and top-tier press mentions, it saved Dropbox loads of money

Most fans of the HBO series Silicon Valley don’t realize how much its creators modeled their fictional startup, Pied Piper, on Dropbox. Even the streaming services and garage-hosted infrastructure built on their lossless compression algorithm, middle-out, may have been based on a conversation with James Cowling. Our story starts with engineer Daniel Horn on an airplane watching an episode of Silicon Valley that introduces this far-fetched algorithm. “That’s ridiculous,” he thinks. “This plot is nonsense.”

But something stuck in his head that, combined with the epic of cost savings that was Magic Pocket, got Daniel thinking about compression. He started tinkering around with images and then realized that lossless video compression, the fictional Pied Piper’s market differentiator, might actually be possible. The lightbulb flipped on. He thought, “This would be a really funny project to do over Hack Week!”

It turned out to be one of the most impactful projects in Hack Week history. But like all big undertakings, things got worse before they got better. Videos are already highly compressed, so the first step was taking a video apart and undoing all of the compression — which made the file ten times bigger. The challenge was to coordinate a team of 11, who are each trying to make their part of the video file smaller.

“So everyone got just a piece of the H.264 standard and had to figure out how to put it back together, and maybe use what other team members had done to help,“ says Daniel. “This set of puzzle pieces slowly started getting better. It was a race from 10x inefficiency back to 1x. Every day different team members chipped off a percent here or a fraction there. And then on Wednesday night… the team broke through 1.0, where every fraction of a percent of savings was pure cheddar.”

But the story doesn’t end there. After “Most Valuable Player” and Powerthru Hack Week prizes, a write up in Fast Company, and a mention in WIRED, Daniel applied the video techniques back to images in a streaming compression format called Lepton. Dropbox not only open-sourced the technology, but also used it to losslessly compress billions of images by 22% — saving millions of dollars in its first year. Not bad for an inside joke.

Happy hacking!