Inside Lotus

I've been asked to moonlight at the Inside Lotus blog. Ostensibly, my presence in that corporate blog is to give a human face to the corporation and also because it's more fun to mix development perspectives among the marketing that tends to go on in "corporate" items. I still have to get my bio posted on the site and to find an appropriate photo but I thought I'd get started. The format is meant to be short and snappy but as you might know, I tend to pontificate, so I'll likely write expanded entries here at the Toli. Anyway here's the first post.

Explaining Software Designs

The ever-incisive writer, Malcolm Gladwell, has been on a whirlwind promotional tour in support of his new novel, Blink which is about snap judgments and unconscious decisions. In a recent interview, he spread his focus to sports

JM: Talk a little about tennis coach Vic Braden, the subject of one of your anecdotes. He says, "We haven't found a single (tennis) player who is consistent in knowing and explaining exactly what he does."

MG: Braden's experience is really interesting. He would ask, say, a world-class tennis player to describe precisely how they would hit a topspin forehand, and they would invariably say that they rolled their wrist at the moment of impact with the ball. And then he'd do a digital analysis of videotape of them actually hitting a topspin forehand and find out that at the moment of impact with the ball their wrist was rock solid. They didn't roll it at all. The expertise of a world-class tennis player, in other words, is instinctive, which means that the knowledge behind their actions is buried in the corners of their brain. They hit a ball unconsciously.

JM: Is that why, quite often, great players don't make such great coaches?

MG: Yes, that's precisely why top athletes so often make bad coaches or general managers. They often don't really know why they were as good as they were. They can't describe it, which means that they can't teach it and they quickly become frustrated at their inability to lift others up to their own level. Mediocre players -- or non-athletes -- tend to make better coaches because their knowledge isn't unconscious. It's the same thing with writing. I know very little about science. But I think I write about science more clearly than many scientists, because I have to go over every step, carefully and deliberately.

Gladwell's words were echoing in my ears when viewing a talk given by Bram Cohen at Stanford. Cohen is the programming wizard behind BitTorrent which is responsible for something like 35% of the traffic on the internet these days and which has gained a life of its own. Listening to him explain the intricacies of his software, I was struck by the number of times he freely admitted that he wasn't sure why things worked the way they did or how he came about to make certain design decisions. He was only able in retrospect to give a hint as to the why and how and oftentimes it was a case of hand-waving or of "magic numbers".

A big part of my job is trying to clearly articulate sofware design and architecture to fellow developers, UI designers, the documentation writers or even to the users of the software. Engineering is all about tradeoffs and pragmatism in the face of complexity. When you have spent time struggling with some coding or design problem and come to some sort of solution, it's often the case that you find it a little difficult to describe the core of your design. I envy those who are able to consistently present great and clear technical rationales for their work and to get at the heart of the matter. Maybe there is a kernel of truth to Arthur C. Clarke's Third Law: "Any sufficiently advanced technology is indistinguishable from magic."

File under: software, sports, decisions, language, technology, comprehension, Bittorrent, Inside Lotus, toli