Book Reviews

Dreaming in Code by Scott Rosenberg

ISBN-13: 978-1400082476
Publisher: Three Rivers Press
Pages: 416

In 1981 Tracy Kidder gave the world The soul of a new machine . The book described how a new computer came alive. In the process, Kidder wrote some of the best portraits of the developers behind the machine and the motivational factors that drive them. A quarter century later, Scott Rosenberg tries to do the same for software as Kidder did for hardware.

Software is hard. It's buggy, over budget, and most of the time not even close to the original idea. Yet our civilization relies on software in an ever increasing pace. With the goal of describing the nature of software development and why it is so hard, Scott Rosenberg followed Mitch Kapor's Chandler project. With Rosenberg's goals, Chandler was the ideal project. And to anyone who's spent some years in the software industry, the story is way too familiar. Chandler has its roots in an idea Kapor had nurtured since the 1980s: a free-form approach to personal information management (or, to put it straight: a calendar application). Chandler is an open source project, that also intended to embrace the philosophy behind open source in its very core. Thus, the team set out to create an extensible architecture that would encourage sharing and collaboration. From a marketing point of view, Chandler aimed to challenge Microsoft Outlook. Interestingly enough, Chandler was finally released just as I finished reading this book in August 2008. At that time, Chandler was 5 years over time, way over the original budget, and Rosenberg had left the project a long time ago. In other worlds, a typical software project. The difference here is that this very project has resulted in a book. And I hold this book as much more important than the Chandler software itself.

Rosenberg gives us an outside perspective that makes it easy to retrospectively see all the mistakes. They're all classics and, unfortunately, by no way unique to the Chandler project. First of all, no one really knew what they were actually building. Sure, Mitch Kapor had a grand idea, but it was still little more than a vision. As the technology goes, the team kept changing fundamental architectural decisions. Worse, they did so on pure speculations. Another classic mistake is to build the solution on today's hot technologies without anyone actually mastering them. In Chandler's case, it was the Python programming language and accompanying frameworks. This is a key point; not matter how skilled the programmers are (and the Chandler project probably had more talent than most other projects), this is a recipe for mediocrity. A new language is so much more than the syntax itself. The hard part is learning the idioms of the language. Until you're there, you'll never write great code.

The Chandler story is interesting, but also extremely painful; the team keeps lining up mistake after mistake. As Rosenberg lets a fictive programmer scream in pain Stop the madness! They're making every mistake in the book! , I couldn't agree more. But in doing so I, in company with Rosenberg's fictive programmer, fall in a psychological trap known as hindsight bias. As humans, our source memory is rather poor; we're simply unable to ignore all our current knowledge in order to put ourselves back in a previous state. The consequence is that once we know what happened, we think it was bound to happen. The reason why no one simply stopped the madness is probably also the reason why software is hard.

But Rosenberg doesn't stop there. He spends the last third of the book hunting for answers and solutions to the problems faced by the Chandler team and software projects in general. If the previous part of the book was good, this is were it becomes truly great. Rosenberg manages to put together a remarkably clear and accessible view of the software development field, drawing on classic writings and interviews with software legends. While there are some interesting ideas here (for example Alan Kay's vision to lend ideas from cellular biology), it's also painfully clear that there's no silver bullet for software development in sight. And until we discover it, I believe that the best thing we can do is to learn from what we actually get right and from our mistakes. For the latter, Dreaming in Code provides more than its fair share of lessons.

Reviewed January 2009