Version Hell Revisited

A surprisingly large proportion of the issues a working software engineer deals with on a daily basis turn out to be cultural. My theory is that this is because software ultimately amounts to people problems, and that it's all about coordination costs and human factors, as I've written about previously. This is not to say that we don't deal with the hard problems, but perhaps the "soft" in software hints at the lingering craft aspect even as we try to escape from Deadwood and leap into our industrial revolution.

This is one reason that I've found that, similarly, a large proportion of the best software engineers I've encountered in my now 25 years experience have not been traditional computer scientists. The linguists, historians and even that rare anthropologist, some of whom picked up the profession on the side, or out of expedience as the web has allowed, have been the real dark matter of technology. The web with its radical simplicity, has allowed mass amateurization to prevail. Now mind you, you do need your stereotypical straight ahead just-the-facts engineers with blinkered and relentless nerdy application, and, increasingly as software becomes more professionalized, the guilds and dark factory mills have started appearing. But hold that thought for now, I have another story to tell, indeed you may call it a technology folktale.

I came across this memo written a long, long time ago, in a far faraway land, (to be truthful, it was a Friday afternoon and I was fed up). Certain names have been changed to protect the guilty, and don't assume the anachronistic references mean that it is of recent vintage. Consider it perhaps a glimpse at the way the software sausage is made and we all know the scandal of slaughterhouses although we now call them meatpacking plants in polite society.

The context is some wrangling where at least 6 teams were pointing fingers at each other about who was responsible for uploading some tools that The Company was using. The flow of data through the system had at length been established, and it was now a coordination problem more than a design and architecture problem. The problem was that different teams were responsible for different components. The email chain had been playing out for months it seemed. I was stuck in the middle making arbitrary decisions mainly by virtue of having written a core piece of the system. I kept asking and everyone kept dodging my increasingly pointed questions. And so I wrote the following:

Previously in the same vein: Version Hell

---

The Memo

From: Koranteng@Toli
To: A long list of interested parties
Date: Friday afternoon, a long, long time ago
Subject: Toolchain versioning Re: provide version number Re: Signing Re: [snip]

I want to tease out the various strands because I'm finding it hard to serve the competing masters here.

Let me pitch it as a folktale, it's a pandemic and we all need comforting narratives.

Master Johnson, my director, would be paraphrased as follows:

Our current process is the moral equivalent of taking a USB stick dropped by our Vendor in our parking lot, and placing that software in our august data centers to sign our Crown Jewels. Now this may be what we're reduced to, but don't blindly use any old tools we are given, let's at least make sure that we track and bless known versions of this toolchain.

There were stronger words said at the outset back when we had our deep dive on our last integration project late last year.

Master Security Team pitched in, again paraphrasing...

Well, in this sorry business, we've normally done integrations of these tools for a given year. We're already overtaxed and busy and we burn weekends to get the integration working. But once we've got it working, we've got clean hands. We don't touch it anymore unless there's an error using them. We're not happy with this but ye olde signing tools ultimately come from the Ministry of Information team. We're middlemen here and the Signing and Software Release teams are the ones that integrate these tools into The Company's IT solutions.

Master Ministry of Information Team is also in a pickle. He receives vendor builds and has to produce release builds with the help of Butler Jenkins. These blasted tools are just one piece of the overall puzzle (you should see what else is on his plate). His main requirement is to be able to reliably sign and update this software for the lifetime of The Product, a decades long quest.

His junior sister, Miss Android Team actually works closely with Messers Qualcomm, Broadcom and Intel, and gets periodic updates from Vendor Google who changes signing requirements at a pace of its own choosing, those Mountain View people move at the speed of the web.

I'll omit the other players who are also similarly overtaxed, although I should make special mention of Mistress Software Release Team who actually owns the official release process and is always playing catch up integrating with Information and with Signing and other things I'm not even aware of.

Meanwhile I'm sitting like Ananse the Spider tending to my farm in the land of Signing and perhaps you can see the roots of my dilemma:

Each build could have a new version of the dismal signing tools.

Per Master Johnson, I can't blindly just sign with any old version of the Signing tools I am presented with. That man signs off on my annual review, so I cross him at my peril.

Per Mistress Software Release Team, I have a frozen API in place which doesn't specify the toolchain version. In any case, there is a long chain between Android, Information, Software Release to Signing, and the lead time to change things is measured in the cost of "Projects". We are booked for months, nay years ahead and agility is a problem here at The Company - The Company's initiatives on agility and the cloud native adoption notwithstanding.

Anyway my solution, as it is, is that I will support "blessed" versions of the toolchain. All that is left is for someone to upload these versions.

Now I'm stuck in the middle of all this, and so far it has been yours truly that is arbitrarily uploading versions of the toolchain based on what I've seen coming through in our test environment. I actually don't know what is a significant version and I don't really want to be in this loop. Further I am making decisions above my pay grade without any domain knowledge of these dismal signing tools.

The details matter and someone needs to own this and solve my riddle.

Again

• Who is it that uploads the "blessed" versions of the toolchain to Signing?
• More fundamentally, who decides what is a blessed version of the toolchain?

I'll end by invoking my friend Sam Ruby's Postulate

The accuracy of metadata is inversely proportional to the square of the distance between the data and the metadata.

I am clearly furthest from the data, so somebody closer needs to make the call. Perhaps I'm reading from the wrong playbook, but I'm merely trying to add value here.

Koranteng
--
Chief Toli Monger
The Company

[snip distressing long chain of emails going back 5 months of people passing the buck]

Postscript

I'd like to think that my late Friday email did have the clarifying effect I intended with its visions of Ruby's Postulate and The Tower of Babel. There were certainly calls for action, and immediate action at that. I learned that I had underestimated the extent of the dysfunction at The Company. There were actually at least 8 other teams that I might have added to my folktale, and they each had their own sorry origin story. The list of people inside The Company who I learned had read my missive was impressive and growing.

There were ruffled feathers however. People who should have dealt with the underlying organizational and communication issue were upset that I had revealed it so starkly. Lots of meetings were called. Action items were issued and so forth. I was advised to let others handle things at this stage. I was also reminded of John Kenneth Galbraith writing in The Great Crash of 1929.

... the rite of the meeting which is called not to do business but to do no business...

One of the oldest, most important - and unhappily, one of the least understood - rites in American life.

Men meet together for many reasons in the course of business. They need to instruct or persuade each other. They must agree on a course of action. They find thinking in public more productive or less painful than thinking in private. But there are at least as many reasons for meetings to transact no business. Meetings are held because men seek companionship or, at a minimum, wish to escape the tedium of solitary duties. They yearn for prestige which accrues to the man who presides over meetings, and this leads them to convoke assemblages over which they can preside.

Finally there is the meeting which is called not because there is business to be done, but because it is necessary to create the impression that business is being done. Such meetings are more than a substitute for action. They are widely regarded as action.

Everyone remembers the story of The Emperor's New Clothes, I should note however that Hans Christian Andersen didn't write the sequel about what happened to that little boy and his family six months later when The Authorities could finally deal with them.

Slightly related, the Gambian proverb goes "Words are like bullets, once you release them you can't call them back.

It is said that a prophet is never recognized in his own country and the verdict was still out on if I was to become The Company's John the Baptist. I won't tempt fate and leave it to the historians to make that determination. In closing, I'll again turn to Belloc

It is always a relief to believe what is pleasant, but it is more important to believe what is true.

— Hilaire Belloc

Office Tales

I'd previously pointed to this juxtaposition, a survival guide to life in any workplace

• Office life by Keith Waterhouse
• Office Politics by Wilfrid Sheed
• The Organization Man by William H Whyte, Jr
• Managing Humans by Rands in Repose
• The Bad Child's Book of Beasts by Hilaire Belloc
  Belloc's cautionary tales and bestiary of observed pathologies are essential for office life.

Soundtrack for this note

File under: software, technology, complexity, data, metadata, design, architecture, best practices, coordination, organization, social, work, office, observation, decisions, process, programming, versioning, humour, toli,

Minutiae

A surprisingly large part of a software engineer's life is spent dealing with the little things. Much as I like to write about grand designs and architectural issues or people, processes and communities, all too often, the devil is in the details and I get lost chasing technological quirks. Herewith a sample of just the past week's minutiae, perhaps fodder for historians of science or anthropologists.

• Boolean Identity Crisis
• Structured Data Footnotes
• Highlights, Lowlights
• Which Side Are You On?
• The Null Hypothesis
• Camel Humps
• The East Australian Singularity

Boolean Identity Crisis

I was going over some code in my pet XForms processor implementation and wondering what was going wrong - incidentally IBM has written (at least) 6 forms processors in the past 4 years along with contributing to the Mozilla XForms effort but we won't get into that - I'll save that for a business school case study or something.

After an hour or so of head-scratching this was what I found. In XML Schema Datatypes, the lexical space of boolean values includes not only "true" and "false" but also "0" and "1". I assume the inclusion of 0 and 1 in the specification comes from the legacy of the C language and presumably that makes sense. But I ask, was that a wise decision? The designers of the specification chose a binary representation in a textual format.

The forms processor is written in Java. It turns out that in the Java language, the lexical space of boolean values is case-insensitive "true" and "false". So if you have a mapping layer that goes from schema datatype to Java you have to add special case code to deal with 0 and 1. Presumably also in the other direction, you have to make sure that you don't output "True" instead of "true". I had been linking to a library that hadn't bothered to implement this more robust logic and ran into this boundary condition. Oh well, I thought, I'll have to add an adapter around this library or rip it out and roll my own - an assignment for the weekend.

I vaguely remember Sam Ruby mentionning that this was one cause of interoperability issues between SOAP implementations and it stands to reason: we've codified an identity crisis.

Structured Data Footnotes

Jon Udell is a careful man whose

blog works differently from the rest of the InfoWorld blogs. The content is well-formed XML, and it follows certain self-imposed rules.

This discipline enables him to work miracles and regularly come up with lots of cool applications. But in the wider world, getting people to author structured data is an often intractable problem. XML is often structurally invalid; as an example the Google Reader team have reported that 15% of feeds on the web are not well formed and that is before considering the feeds' semantic validity.

Thus I was tickled by this footnote he wrote when playing with microformats.

The difference? I ran the page through HTML Tidy to get well-formed XML ²...

²   It wasn't entirely automatic, unfortunately, I had to wrestle with character encoding issues too.

Ladies and Gentlemen, I give you The Gruesome Twosome of Computer Science: Structured Data and Character Encoding. We call them footnotes.

Highlights, Lowlights

It started with an offhand comment, a report that one couldn't set the background color on text in the rich text editor in Mozilla Firefox. Every other command on the editing palette worked. You thought, "15 minutes tops, I'll take it".

So you look up the list of command identifiers in Internet Explorer and notice that the execCommand method has an attribute named BackColor which

Sets or retrieves the background color of the current selection.

You head back to the Midas spec for Mozilla and see that for the backcolor entry

This command will set the background color of the document.

Hmm... Your first thought had been that it was a case-sensitivity problem, "backcolor" rather than "BackColor", but the documentation notes that case doesn't matter. So the problem is rather what you highlighted, in Mozilla, backcolor applies to the document and not to the current selection. This means that you have to use a different identifier to achieve the same effect in Mozilla. You promptly notice one called hilitecolor

This command will set the hilite color of the selection or at the insertion point. It only works with useCSS enabled.

Okay, someone decided to not use the same command as Internet Explorer, fair enough, you've seen worse. As you change the code, you say, it's Mozilla and CSS is enabled by default so all I have to do is switch to using hilitecolor.

Of course that doesn't work. You then tell yourself that you'll just enable CSS with the useCSS attribute (you had looked more closely at the code and noticed that CSS styling had been deliberately turned off for some obscure reason).

Hmm, that didn't work...

So you go back to the spec and you notice a styleWithCSS attribute. Interesting... You try that instead of useCSS. Of course, you happen to be testing using a version of Firefox (1.04) that doesn't support that attribute so there's an exception. Presumably this attribute was introduced in Firefox 1.5 or something. Still it's a good thing that you are using an older version for testing otherwise this would have been another bug (feature doesn't work in Firefox 1.0 etc).

You bang around for a while and go back to the original Midas demo which seems to work correctly if you have the "use CSS" checkbox checked.

You view the source of the demo to figure out what could possibly be making it work and you notice that they are setting the useCSS property to false when the checkbox is checked. Huh? They set useCSS to false in order to enable CSS!

So you go back and look more closely at the documentation. First you notice that the useCSS property is deprecated. Hmm... the plot thickens. Then you read this

useCSS - value: true/false

Note: This command has been replaced with styleWithCSS. It takes the same values as styleWithCSS, but the meaning of true and false are inversed.

Up is down in other words. It's like Alice in Wonderland or something. Do note that there is no word on what version the replacement occurred. Nor indeed is there any footnote on why true and false are "inversed".

Anyway you've finally figured it out, you make the change to enable CSS styling only when applying the hilitecolor attribute, add some future-proofing to use the styleWithCSS attribute in case the deprecated useCSS attribute is removed in later browsers, submit the patches and 3 hours of your life have passed. It was frustrating but there's an object lesson somewhere. If you are building a robust cross-browser rich text editing application, you have some code that has to go through this rigmarole.

I have a confession. I recounted the above tale because I feel guilty: I could have prevented all this four years ago.

Internet Explorer was the first browser to introduce rich text editing. It was not pretty since they didn't want it to be as good as Microsoft Word, but it worked reasonably. IBM later contributed resources to Mozilla to beef up its rich text editing. Indeed four years ago, I was asked to review the resulting Midas spec by my colleagues who had been working on that functionality. The spec looked serviceable, pointing mostly to the Microsoft documentation since they wisely chose to follow the de-facto standard. I reported a few bugs with the intial implementation and we built a rich text editor around it. It's widely used on the web these days.

I obviously was a bad reviewer because I certainly didn't notice that there was a change in the semantics of the backcolor attribute from "current selection" to "document" nor indeed that a new attribute, hilitecolor, had been introduced in Mozilla. There is absolutely no reason for these discrepancies. Nor do I want to go down the forensic trail that explains why you can't have a background color on text if you don't style with CSS. That's a rathole of its own.

So what do we have here? A de-facto standard was implemented but someone took the liberty to change one aspect of it for whatever reason - semantic purity or something. I suspect I won't be the last developer to waste an afternoon on this or indeed the last user to be cursing about why I can't add a background colour to a piece of text. I hope that Opera and Safari, if and when they have their rich text editing implementation in place, will do a wholesale copying of Internet Explorer's behaviour. I don't want to be chasing these lowlights again. I often see criticism of Microsoft's inconsistent approach to standards but everyone can be as guilty as them on occasion.

Which Side Are You On?

Stefan Tilkov recently asked what's wrong with Javascript? The answer of course is nothing really, it's a fine language as evidenced by looking around current thinking on the language. Indeed Brendan Eich's biggest admitted gotcha about the language is automatic semi-colon insertion. Thus there is no reason it can't be used in environments outside the browers in which it is most widely deployed as glue - I've played with Rhino on the server-side without any problem.

From what I understand, Jotspot, which incidentally I consider a cunning plan to showcase the Dojo toolkit, uses Javascript as the scripting language on both the server and client. All power to them. Of course with Javascript from the same document executing in both environments, you can get very confused. The answer to one of Jotspot's most frequently asked questions, "Why isn't my Javascript function being called?" is that "your code is not running where you think it is" ergo, you're expecting that the current code you're looking at is server-side rather than client-side, or vice-versa. This is especially true since the object model is likely to be different, the browser DOM is a deliberately constrained environment whereas on the server side you'd want to allow your plugins to do more.

Thus there is a little impedance with using Javascript everywhere. Perhaps it's less confusing to use a different language and syntax for server side code - a tag library in jsp, or embedded java, php, asp or whatever. Or maybe clever syntax coloring in your editor or IDE would do the trick to remind you of the context. Needless to say, you have to decide what side you're on...

I've been working on a project in which the others on my team are seasoned PHP gurus and are occasionally petrified of Javascript - the reason of course being the continued brittleness of the browser platform. When I started work, this bias showed and their initial recommendation was to do as much as possible in PHP on the server-side. They recognize however that that we are living in an age of interactivity so we need that shine that comes with moving intelligence to the client. Still when you start doing data-binding and automatic JSON serialization of PHP objects, you get constructs on the browser client that you wouldn't normally use if you were a client-side person. As someone who's very comfortable with both client and server side code (perhaps more comfortable with Java than PHP), I keep running into such peculiarities all the time. Of late I find myself prototyping code in client side Javascript even if it will eventually morph into server side code. Perhaps I need to get more into Python or that Ruby bandwagon. Still you tend to develop a split personality when you develop for the web.

The Null Hypothesis

I got the note: "you can't append a column if you click on a cell in the last column of a table in Internet Explorer".

Huh? I attempted to reproduce the bug and, sure enough, that was the case. Vaguely at the back of my mind I recalled from painful experience in K-station that there were special APIs in the HTML DOM for dealing with tables. Thus I searched the codebase for insertRow and insertCell. Hmmm those functions were nowhere to be found. How were they doing the column insertion, I wondered? My guess was that this was either some innerHTML tricks or simple standard DOM manipulation. Thus I had to dig through the code and eventually encountered the Node.insertBefore conundrum.

Now insertBefore is a method on the Node interface that is part of DOM level 1 specification. Every browser claims at least DOM level 1 support.

It turns out that in certain versions of Internet Explorer the second parameter to the insertBefore call can't be null, you get an Invalid Argument exception. Mozilla handles this condition as one would expect in their Javascript binding; Internet Explorer chokes. [Obscenity]. There was too much code to change to use the HTML-specific methods so I just hacked special case code that ensures that I don't pass a null to Internet Explorer - 3 hours of my life perhaps.

Now this doesn't amaze me really, thinking back on it, this is probably the reason that somewhere in the bowels of every Javascript library that deals with dynamically adding a new option to a select control, you'll find code like the following:

function appendOptionElement(select, newoption){ if(is_ie) // test for internet explorer somehow   select.add(newoption); else   select.add(newoption, null); }

The browser is a fragile place and hopefully the frameworks that are being developed will shield you somewhat from such issues, but it is telling that you can't rely on core DOM functionality. Even if these quirks are fixed in Internet Explorer 7, this patched-up code will have to hang around for another 5 years before users will no longer use older versions of the browser. We're in a world of pain in the browser world.

Nulls however are problematic throughout computer science. The arguments around them may sound like angel and pinhead discussions but they are fair questions. What indeed is null? What is zero for that matter? For centuries and throughout the Dark Ages of the West, there was no concept of zero, it took Arabic Algebra to spread that notion. Why should one expect that programmers would have internalized the null concept? Reasonable people can and do differ on how to treat null.

Just now, reading through the latest Dr Dobbs journal, I noticed the following in an article about Consuming .NET Web Services in Oracle JDeveloper

The ATL Server SOAP handler generates an xsi:nil="1" attribute when the element's value is null, or when the array is null or zero-size. Unfortunately, Apache SOAP fails to deserialize UDTs whose fields contain the xsi:nil="1" attribute and expects zero-size arrays to be represented as XSD arrays with the dimension parameter set to zero.

The author then proceeds to outline a variety of workarounds. Now I happen to not be a fan of the SOAP style of programming (in the past I've called it Crusty Old Architecture pronounced SOA with a silent P) but as a developer, I feel the pain. The article is a catalog of kludges and likely mapping errors that have to be worked around - the word "unfortunately" is used entirely too often.

I started my career doing graphical programming thus I perked up when Raymond Chen recently outlined the consequences of invalidating the null window and his anecdote is worth quoting at length.

If however you end up passing NULL as the window handle to the InvalidateRect function, this is treated as a special case for compatibility with early versions of Windows: It invalidates all the windows on the desktop and repaints them.

Even more strangely, passing NULL as the first parameter to ValidateRect has the same behavior of invalidating all the windows. (Yes, it's the "Validate" function, yet it invalidates.) This wacko behavior exists for the same compatibility reason. Yet another example of how programs rely on bugs or undocumented behavior, in this case, the peculiar way a NULL parameter was treated by very early versions of Windows due to lax parameter validation. Changing nearly anything in the window manager raises a strong probability that there will be many programs that were relying on the old behavior, perhaps entirely by accident, and breaking those programs means an angry phone call from a major corporation because their factory control software stopped working.

The null hypothesis strikes again.

Camel Humps

Even when you get past these details, you get into matters of syntax - a longstanding pet topic of mine. Consider the separators that people use for readability. Some people like to use Hungarian notation, others prefer hyphens... well I've already written a hyphenated parable so I'll skip that aspect. Anyway, assume for some insane reason that it makes sense for your spec to have an attribute named "windowTop". There'll undoubtedly be people who will write it "window-top", "window.top" or with some other variant of case, "WindowTop"? If you're dealing with XML where case matters, things will fail. You say potato, I say pubDate anyone?

I've been playing with a product that is essentially a wiki and it turns out that by convention, camel case (or should I write it as CamelCase) is significant in the wiki world as denoting a "WikiWord". I even had to spend half an hour writing glossary entries for these concepts since in our system, user names and page names had to be in that format.

You can imagine however the kinds of issues that arise when you have an html editor that allows you to enter HTML and Javascript along with php code and a specialized wiki syntax - since for some reason, it was decided not to use angle brackets in this product. The most complicated piece of code is going to be the parser. As currently implemented, the parser is a mass (or should I say, a morass) of regular expressions and all kinds of things I'll never understand. There's even special case code in there to handle nested comments in Javascript. What was never forseen however was that you'd have to deal with user-entered script code. What would happen if camel case is used for variables inside of said script? Well it wasn't pretty when the wiki engine jumped in and treated script as wiki words, let's just say that someone had to come up with a solution.

There's this concept known as McCabe Cyclomatic Complexity which is basically an application of graph theory to software and is often used to pinpoint potential problems. The basic insight is that if there are too many decisions or paths through your code, it will be buggier, harder to maintain and test. A good guideline is that anything with complexity greater than 15 in this scheme is ripe for refactoring. Luckily I have access to some tools that can generate reports and provide some numbers to validate that nagging sense that a piece of code is getting harder to understand. What worries me is that I keep running into code that laughs at such guidelines - after the latest change the parser code just hit 41 (to give some context, values between 21 and 50 indicate "a complex, high risk program" and with the 50 barrier looming, we are verging towards that notable status of the "untestable program - very high risk"). I'm sure it didn't start out this way and that the steady accretions have been solutions to real problems. Still, technical arteriosclerosis continues its inexorable spread...

Ruby on Rails and the Zend framework for PHP are founded on favouring convention. We had this code that automatically created the schema for what amounted to a database table. We never actually told users that this was what was going on. Of course once a wider audience started to play with the product, we had to fix the bugs that arose to enforce the convention so that databases would be named as the code expected. A classic case of leaky abstractions I suppose. You can guess the complexity the additional error checking added.

Moving up a level, you get politics - the most aberrant of which have been the feed wars of the past few years perhaps best summarized hilariously by Shelley Powers. Truly, Jesus wept.

Bill de Hóra in an aside wisely noted that

Programmers would rather squabble about minutiae - it seems this transcends community or language choice.

But even when you move beyond whiplash and abrasive personalities, who would have thought that the Atom working group would have spent so much time on the concept of dates. When I saw dissertation-length essays on the various types of dates that might be used in publishing systems, I was content to continue lurking in that community. The thing however is that these details do matter and they are best dealt with up front. The aggregate waste of programmer effort in pursuit of minutiae might keep the profession in business but it surely isn't sustainable.

The East Australian Singularity

I'll conclude by noting that acts of God (or in this case, his flawed proxies: politicians) can come into the picture. Thus I read the following this week: Eastern Australia: Java applications impacted by the change to daylight savings time dates

The running of the Commonwealth Games in Australia in March and April 2006 has resulted in an extension to daylight savings time in the eastern states of Australia, which includes New South Wales, Victoria, South Australia, Australian Capital Territory and Tasmania.

Rather than the clocks being put back an hour at 03:00 on Sunday 26 March 2006, they will now be adjusted at 03:00 on Sunday 2 April 2006. The IBM Software Development Kit (SDK) or Java Runtime Environment (JRE) is not currently aware of the one-time change to the daylight savings time extension. An interim fix is required for your application to have the same time as your operating system.

This change applies to all of the Java environments, irrespective of their setup, whether they are set up to use the operating system time zone information, or the user.timezone custom property that can be set for the SDK or JRE.

I assume television schedules were behind this change to the time/space continuum since the Commonwealth games have just begun. Or perhaps the politicians were concerned about athletes missing their events or something. Still imagine if you're the Java programmer who has suffered through the badly defined date apis in Java 1.0 a decade ago, adapted to the improvements in succeeding versions and finally, finally gotten a robust application deployed. Then your manager walks in and tells you about the East Australian Singularity. You're going to have to rework everything since bank transactions might be messed up and there are likely to be blinking clocks etc. We live in a global village so you have to worry about what happens if some mission-critical application that you rely on is being run in Eastern Australia. You can't even have the special case restricted to that country, it's only a sub-region that is affected. At such times I'd prefer being the poor Eastern Australian sheep or lamb, their lot is much easier.

What a way to make a living, if it isn't one thing it's the other. And then there's version hell. Lord help me.

File under: technology, software, abstraction, details, minutiae, glitches, programming, standards, language, absurd, toli

Version Hell

In the beginning we called it "CoolCodename". But that was then... That was before fate stepped in. A one-act play...

[Demosthenes is lying on his chaise longue, eating grapes, he appears to have been in this state of lassitude for a long time. Ctesiphon enters stage right in a hurry, he appears dishevelled.]

Demosthenes: [between munches] So what's bothering you, Ctesiphon? Software got you down?

Ctesiphon: The software's no problem, it's the naming thing. Those Gods are capricious beasts.

Demosthenes: [puzzled look]

Ctesiphon: I mean its the version thing. It's the fact that there is no longer an XYZ 2.5, it's now ABC 2.5.

[he continues, gesticulating wildly]

Did you know that the ABC 2.5.1 stream into which I've checked in 8 bugs was renamed on Wednesday to be the ABC 2.6 stream?

Demosthenes: Slow down a little, let me get it straight. There's a 2.5.1?

Ctesiphon: Slow down? Demo, 2.5 was last month's story, it's all 2.5.1 these days. And 2.6 of course. Yes. Well...

[slowly now, pausing to collect himself]

The ABC 2.5.1 project was renamed to ABC 2.6 after people like myself had been checking things into it for weeks. Ergo, post-facto crapulum.

Demosthenes: Hmmm. Ipso facto absurdum.

Ctesiphon: Greek not Latin, where's your classics, you anachronistic orator. This means that I have to recreate all my activities and redo the check-ins into whatever project is now supposed to be ABC 2.5.1.

Demosthenes: But I heard from the Oracle that the latest thing was XYZ 2.5, I mean JKL 2.5.

Ctesiphon: Rebranding they call it, that's the thing with Delphic Oracles... [pauses] Then they pretend that we were always calling it ABC 2.5 and not XYZ 2.5 and that we were always calling it JKL 2.5 and not CoolCodename. But that's a seperate issue. Don't get me started.

[he starts pacing up and down the stage, suggest crackling thunder in the background as he enumerates each point]

What gets me steamed is that

1. They only told us now (Friday night) that the ABC 2.5.1 stream is open for submissions, and
2. All submissions are due by Sunday, and
3. The ABC 2.5.1 stream is now the ABC 2.6 stream, and
4. The place to check in the ABC 2.5.1 fix is the old XYZ 2.5 stream which was not renamed to ABC 2.5.1
5. And... oh forget it, who knows what the ABC stream is for....

Thus when I check in for ABC 2.5.1, I have to remember not to use the streams I had created, named ABC 2.5.1, but rather the old stream called XYZ 2.5. Reductio ad absurdum.

Demosthenes: [After winking at the audience] But I just read this week's scrolls, it was all about 2.6. What's going on, Ctsey?

[Ctesiphon becomes manic as he replies, perhaps his toga starts coming loose]

Ctesiphon: Which 2.6, I ask? Which 2.6, Demo? Let's not mention that we are going to have a new release of XYZ. They only tell you later. As you know XYZ 2.6 is based on CoolCodename, I mean JKL 2.5, but they'll be wanting some of the fixes from ABC 2.5.1.

[louder]

So this Saturday, after I finish merging my work into ABC 2.5.1 in the old XYZ 2.5 streams, I'll have to find the new streams for JKL 2.6, whatever they are named, because they won't do automatic merges. Then I have to diff the 2.5.1 code against the JKL 2.6 base (I mean the XYZ 2.6 base) and take only those changes that are in ABC 2.5.1 and hand-merge them.

[Even louder and faster]

But of course, they made some security fixes for that Trojan issue in JKL and so the security model has changed because the SMB market you aren't guaranteed to be using DBO instead it's XBO or SBO. And of course, there is no machine to test any of this stuff whether ABC, XYZ or JKL. Come to think of it, I've only ever tested my patch on a old build of ABC 2.5 that was using DBO, I never tested with...

Demosthenes: All right, all right... [winking again at audience then patting Ctesiphon on the back]

You better get some sleep, sounds like you've got a busy weekend ahead of you.

Ctesiphon: That Sisyphus had it good, the boulder always rolled down the same hill.

[Demosthenes walks him out stage right. Comes back alone, pours himself a goblet of wine, sips it and sighs... looks straight at the audience.]

Demosthenes: That's the thing about managing engineers, you've got to tolerate all this background noise. [Picks up a grape and sits down]

[Curtain falls.]

Note: the foregoing is a work of fiction, any resemblance to actual persons or products is entirely coincidental. Your mileage may vary. Always fasten your seatbelts when on a chariot. Objects in the mirror may be closer than they appear. Do not drink and drive.

Further reading

• "Also, as widely expected, the version formerly code-named 'Office 12' will be known as Office 2007 when it ships in the second half of this year."
• Calling Things By Their Names
• Rosencrantz & Guildenstern Are Dead
• Chimera by John Barth

Soundtrack for this dialogue

• UB40 - Version Girl
• Mica Paris - I'd Hate To Love You

Playwright's Note

A number of people have been reading themselves or their products into my little tale of version whiplash and confusion and I've even been asked to decode ABCs and JKLs. I rather thought the point was that the three letter acronyms didn't matter and indeed will change without notice until something sticks in the market place (ESB, SOA etc)

I will admit that almost all the lines of dialog sounded vaguely familiar to some, "verbatim" was what a friend said. But really, it's just fiction...

Still one might well construct a parallel universe in which CoolCodename might be "Portal" or "Workplace". Lotus Workplace (LWP) might become IBM Workplace and then might morph into Workplace Collaboration Services (WCS) for a while. Product managers might get the idea to attack the SMB market and turn a "lightweight" WCS into a CoolCodename-d PortalX which might in turn become Workplace Services Express (WSE) and then you might need to get version so-and-so out for... Well you get the picture.

It's all Greek to me.

See also: Version Hell Revisited

File under: naming, technology, software, absurd, versioning, branding, process, programming, life, humour, toli

Flickr's Godfather

The 3,000th photo I uploaded to Flickr is a howler. Well at least I think it is, let's have a look (full size image).

A little context is in order...

I, and countless others, had complained about Flickr's excessive use of Flash. First I had inveighed about rendering and accessibility concerns in Cultural Sensitivity in Technology. Then I used Flickr's Flash buttons as a prominent example in The Unloved HTML Button and other Folktales.

In any case, 2 days after the folktales were told, Flickr finally switched from Flash. They no longer use a wrapper for image display - allowing native browser image rendering, and they changed their button toolbar from Flash to DHTML. I'd like to think my snarky comments were the tipping point but I won't flatter myself. They now only use Flash where it's appropriate, for drag and drop organization of photo albums, the kind of job that Flash or applets are particularly well suited for. Now of course they didn't use unloved html buttons in their toolbar but we'll take what we get. DHTML has greater reach than than Flash and accessibility concerns are more easily addressed on that front. Also there are an evolving set of design patterns for dealing with unobtrusive DOM scripting and forms.

So there I was, pleasantly surprised by their responsiveness, and going about uploading a little comic image to punctuate some later toli essay. You'll notice from the image that there were a bunch of glitches on the first morning of the big switch. The icons for some of buttons of the toolbar weren't showing up. Oh well, we'll ignore that but simply note that if they were standard HTML buttons, there would be no images to download. Moving right along...

Then I noticed a couple of typos, I had tagged the photo as sopronos instead of Sopranos and the image's title mentioned Godfarther which tickled me somewhat.

Flickr's Godfather or Flickr Goes Further?

Well anyway, Flickr implements a Click-to-Edit feature, a little unobtrusive DOM scripting that allows you to edit in place, so I corrected the title and hit the save button that appeared. That's when this error message came up and I took the screen capture

Taking a step back for a moment, let me just say that I love glitches. They expose the interesting aspects of complex systems and, much as we aim for simplicity, software tends inexorably towards complexity. As users of software we see lots of glitches daily. As an engineer, I am always interested in the first few days of a new deployment. You can test all you want but all bets are off when you get contact with real users and the real world. As an example, Technorati's recent makeover exposed lots of unforeseen glitches and they have had to work hard to address most of them in the past month. I was chatting recently with Dale Schultz, globalization architect at IBM and noted that I have a standard set of user names when I test new pieces of software, I make sure to have hyphens (hence I use my surname), ampersands (Sun & Sun is my canonical company), and, of late, accents (Rokia Traoré) because I've been bitten by various curses in the past in the software I've written. My former team has a José López test user for the same reason. Sam Ruby uses the word Iñtërnâtiônàlizætiøn as his proving ground in the same vein. We got to discussing the tyranny of patents at IBM and I pointed him to the Prior-Art-O-Matic for a laugh. Dale is obviously many steps ahead of me and of course he tried entering a euro symbol and immediately noted that that CGI application was broken and couldn't handle euros. Glitches often tell you a lot about application internals and the things that the developers tried to foresee or, as the case may be, ignored.

But back to Flickr's Godfather, what can we say about the glitch?

• Flickr is passing xml back and forth in their API calls.
• They are likely using XMLHttpRequest to do the voodoo of incremental loading without refreshing the page.
• There's an API key, probably tied to the user's identitiy that is likely passed around in every call. Sensible enough.
• They have to implement a Javascript layer to catch API errors and display something to the user.

Now I could have determined a lot of this and more by poking around and doing the View Source investigation. At the time, I wondered if I would have done any different and concluded that my implementation would have been much the same.

I have to say that like many others I'm highly impressed with Flickr, they had defensive programming and had appropriate error messages. Most people wouldn't have bothered dealing with these boundary cases. I haven't seen similar glitches since that first day thus the teething pains were temporary and they continue to add nice features to their service.

In any case, the juxtaposition of Silvio growling and in full bloom, the Godfather typos and the error message that popped up under Silvio's hands certainly made for a little amusement then and even today and now has occasioned a short blog entry. It reminded me of an advertisement for Fosters beer I believe that goes "It touches the parts other beers fail to reach". I guess the analogue in this case is "Flickr Goes Further".

As to why I had uploaded that particular Sopranos image, well let's just say that there's a famous quote from that scene and that's for some later toli.

[Update] I tried to cross-post this to my internal IBM blog only to find that the post was chopped off at Ruby's Iñtërnâtiônàlizætiøn magic word. Thus ironically as I was pointing out glitches, I just got bitten by one. I believe BlogCentral is based on Roller software and I suppose that I'll have to figure out whether the problem is in IBM's additions or in the core framework. The interesting thing about bugs with special characters is that sometimes you can't write the issue up because the software can't handle the characters in question. Perhaps BlogCentral needs a Godfather.

File under: flickr, technology, web, dom, scripting, glitches, funny, whimsy, programming, development, javascript, DHTML, software, error handling, design, photo, error, message, collage, toli

Deadwood and the Web Application Leap

I came across the following script in my travels. Thought I'd share. The note scribbled on it was:

A spaghetti western in the technology world of the town of Deadwood, Wild West, USA.

• Elevator Pitch
• Scene I: The Web Application Leap
• Scene II: Every Man For Himself
• Scene III: Deadwood
• Scene IV: Frontier Justice
• Scene V: Calamity Jane
• Scene VI: Winner Takes All
• Coda in the Commons
• Wild Wild Web Soundtrack
• Credits (A Chat Transcript)

Elevator Pitch

My previous Ananse folktales seemed to go down well with a certain audience as a kind of biblical, Grimm's tales of "African magic realism" as The Guvnor put it. At the risk of carrying that Autumn of the Patriarch conceit too far, I was reminded of westerns, of the spaghetti variety, when I came across the following insightful tidbit from Rands in Repose who you can imagine as Sergio Leone surveying the state of the current technology world and deciding to go local. It's not quite the masterpiece that was The Good The Bad & The Ugly which saw a great print released last year or the earlier and crueler For A Few Dollars More, it's rather that first shot across the bow of B-movie legend, A Fistful of Dollars. So without further ado as your novice director, let me sit back in my chair.

Lights. Camera. Action.

Scene I: The Web Application Leap

[Voiceover by "Rands" as a wagon pulls into town]

The size of the fervent rush towards the Ajax reminds me that folks are dying to discover the Next Big Thing. It's been awhile since we've had a Netscape-magnitude holy shit and while there are have been many false positives... has any Internet technology in the past two years seriously knocked your socks off? Well, Ajax has not knocked my socks off, but I believe it's a indication of a revolution that will involve all of us... but understanding that revolution will take one leap of faith on your part... It's illuminating that the first stumbling block folks who are playing with Ajax are finding is that damned back button. At my prior gig, we had the same problem. We gave our users a complex web application that provided all sorts of cool filtering and sorting controls.... that could be instantly totally nuked by the back button... a shift reload.... you name it... ka-blooey, your state is gone. Ajax's doesn't care about the stateless nature of a web browser. Ajax just wants to quietly update your current page... and that's the second part of the first leap:

Stop thinking of a web application as a collection of pages.

The back button is not a bug in Ajax, it's a flaw in the browser metaphor.

I hearby serve notice to the following browser controls: forward, back, home, reload, and that URL field. You need to die... unless I need you. These controls are (rightfully) designed around the idea of the web as a collection of pages, but an application is collection of objects where you, the user, are guided by a well designed interface to get your job done. We don't actually need to kill these controls because they do serve a purpose, but the web application developer should be able to choose when they're available because the developer is designing the application and they are incented to do what's right for their users.

No. The perfect web application will never be a direct replacement for your favorite native application. The medium, the technology that is the Internet, will always change the content that it delivers, but web applications are still in the dark ages. Take a leap with me and think about web applications that do not compromise in user interface, that do not settle for interaction models designed around a clumsy metaphor. All the richness of the best desktop application belongs in web applications.... all you gotta do is want it.

Scene II: Every Man For Himself

With this increased interactivity comes a few pitfalls and lets hope people don't go overboard. For the vast majority of applications it should be a case of adding unobtrusive javascript to enhance the experience (mostly to reduce latency). Things like in-place editing without page reloads are fair enough. We should think clearly about some of the additional things that are indeed possible in the user interface. As people start experimenting with the drag and drop and the enhanced user experience that Lotus K-station and various other applications were implementing 7 years ago, lets be judicious about what makes sense.

Let's not forget the basic principles of interaction design and mundane things like bookmarking, the back button and the other familiar landmarks our users have come to expect. That's not to say that the browser is the be-and-end-all of user interface design. As we "innovate" in this Brave New World, lets think about best practices from the Old Country and codify the design patterns both in terms of the coding that we actually come up with but crucially also in terms of the user interfaces we develop.

Scene III: Deadwood

We're seeking a Beverly Hills of web applications not a Deadwood Wild West of Windows applications each with their idiosyncratic notion of how things are done.

I love cursing, outlaw justice, bodice ripping pimps, prostitutes, con men, adventurers, opportunists, charlatans and all sorts of entrepreneurial adventurers as much as the next guy. Truth be told, it has pretty much been a Wild Web for the past decade. But lets not forget the downside:

• the grim sloppiness and greyness of it all
• the stench of human and other waste
• the corrosive effect of smoke
• the pervasive mud soiling everything in sight

Let's not forget the hard and brutish lives of those indentured to 19th century brothels, the brackish drinking water, the syphillis and smallpox, the rampant infant mortality and young mothers dying while giving birth to children. And then let's not mention the life expectancy in this environment and the Chinaman's pigs eating up the detritus of dead bodies as a convenient disposal method for the canny, vicious strongman who really runs the town. The Chinese saying goes "May you live in interesting times" and indeed in these times of retrenchment of IT budgets, deficits, crusades and Global Wars on something or other, these are interesting times...

Scene IV: Frontier Justice

While I loved the Wild Wide Web, I've mostly hated the monopoly-induced stagnation of the past 5 years. It was jarring and a blow to the great momentum and innovation that had been built up in the early days of the web. Now that last comment might seem like a cheap shot but remember that in the movie, Clint Eastwood's Man with No Name is treacherously playing off rival gangs of criminals and smugglers against each other like in Yojimbo so bear with me.

In any case, as Dare Obasanjo keeps reminding everyone these days the XMLHttp voodoo that is now being exalted was developed by Microsoft. And he has the right idea, Internet Explorer 5.0 was indeed the first browser to have a reasonably complete Dynamic HTML implementation where one could treat the browser as a dynamic surface that could be manipulated programmatically. Indeed Netscape and Opera didn't have monopolies on innovation and adopted lots of the great ideas from Microsofties who sat alongside them in the same W3C and IETF mailing lists where the constitutionnal council of the web was hashed out. And yes Dare, we really need to celebrate all the marshalls who were part of Buffalo Bill's posse.

Scene V: Calamity Jane

When I was developing K-station, I might have complained about the long nights I had to spend pondering the sometimes willfully idiosyncratic syntax in MSIE, the numerous undocumented quirks, the constant battles with focus, the inexplicable modal dialogs, the buggy scripting engine (what is it about scope and JScript by the way? what was wrong with calling it JavaScript? and why did everything need a setTimeout call to work robustly in IE 5.0) and don't get me started about the quirks in the Trident engine's implementation of CSS box model which has caused much head scratching and even is the focus of countless QuirksMode sites and Usenet groups. But with pickaxes, perspiration and youthful zeal to survive, having crossed oceans to escape Potato Famines, religious persecution or plain "invitations with shackles" for those unlucky to hail from Dark Continents, those of us who arrived in Deadwood were able to make do of things as we found them and actually build useful things in browsers.

Madman that I am, I tried to wrangle that beastly rodeo horse called Netscape 4.7 and almost lost my mind. I certainly broke a few bones in the process: it was a thoroughbred that couldn't be tamed. Thus the competition wasn't pretty and was even lacking in parts although they had advantages in areas other than programmability: faster rendering, multiple platforms beyond broken windows, better standards support etc. But lets be clear, once the Netscape/Sun/Novell/IBM threat was dealt with circa 1999/2000, development of Internet Explorer was plainly mothballed. It was only once the rewrite-from-scratch coal miners in the caves of Mozilla-land struck gold that we now hear about Internet Explorer 7 and that that new features will be coming in the browser which apparently can evolve seperately from the operating system.

Since I've always wanted the code that I write to work in my browser of choice, I continued to try to make our portal framework run in Mozilla-based browsers and was a frequent visitor to the Bugzilla bar-rooms in the hope that I would see progress. The Mozilla 0.7 milestone was the point at which I could say that we had a new sheriff in town; Calamity Jane no longer had to stand alone mourning Wild Bill Hickok. Certainly it was the first release in which one could see part of the K-station portal user interface show up. At that stage there were still things like an incomplete scroll model, lack of xml serialization and easy xml loading. But that lack was understandable, as Creationist Pilgrims that had read from the gospel of the W3C, the Mozilla folks would only take features that were inscribed therein. It took some convincing for them to implement items that were useful even if not New Testament material: Apocryphal books from the Latter-Day Church of Redmond.

Once the Mozilla M8 milestone arrived, the air had cleared and the new sheriff was beginning to assert himself. I was able to unleash the Mozilla support in the portal on scared product managers everywhere. People began to take notice and a little ripple of excitement run through everyone in the bar.

Scene VI: Winner Takes All

I recall a memo from someone about getting hardcore about the web in the hazy past. Was that a flashback or repressed memory? But I haven't seen a single CSS box model issue addressed in 5 years (I hope to be proved wrong in this summer's release). I've seen security updates and of late a pop-up blocker but did an asteroid hit all the browser developers in Redmond when Y2K came about? Did Lee Van Cleef ruthlessly deal with them in his ugly but efficent way? Or was it rather that a bunch of Apaches, led by Burt Lancaster, raided The Alamo?

Now all of the above was written with tongue firmly in cheek of course. I've read the "Don't Pick Fights" injunction in IBM's blogging guidelines. More to the point, I'm sure that if Netscape/AOL, Lotus/IBM, Apple, Sun, Adobe, Novell, Oracle or any of these other players had their way in the game of high stakes poker that is the technology world, their behaviour would be similar to what we have seen and often, when they have a winning streak going, that's what we do see. There are no good guys in A Fistful of Dollars, we're all empire-building cads fiddling with loaded dice.

Coda in the Commons

Unlike many, what interests me is not who wins the game, it's rather the commons in the village.

Via Kingsley Idehen, I was recently reading Brad Cox's prescient 1990 article:

Planning the Software Industrial Revolution

The possibility of a software industrial revolution, in which programmers stop coding everything from scratch and begin assembling applications from well-stocked catalogs of reusable software components, is an enduring dream that continues to elude our grasp. Although object-oriented programming has brought the software industrial revolution a step closer, common-sense organizational principles like reusability and interchangeability are still the exception rather than the rule.

According to the historian, Thomas Kuhn, science does not progress continuously, by gradually extending an established paradigm. It proceeds as a series of revolutionary upheavals[KUHN]. The discovery of unreconcilable shortcomings in an established paradigm produces a crisis that may lead to a revolution in which the established paradigm is overthrown and replaced.

The software crisis is such a crisis, and the software industrial revolution is such a revolution. The familiar process-centric paradigm of software engineering, where progress is measured by advancement of the software development process, entered the crisis stage 23 years ago when the term, software crisis, was first coined. The paradigm that may launch the Information Age is similar to the one that launched the Manufacturing Age 200 years ago. It is a product-centric paradigm in which progress is measured by the accretion of standard, interchangeable, reusable components, and only secondarily by advancing the processes used to build them.

Whereas mature engineering domains define standard products and allow diverse processes to be used in making them, in software we do the reverse, defining standard languages and methodologies from which standard components are to magically ensue.

That's the concise articulation of what those who designed the web were thinking about. This sentiment goes under many names (loose coupling, object oriented programming, service oriented architecture amongst others) my favourite embodiment is a certain architectural style but I believe that no one has a monopoly on insight into these things and indeed the marketplace will sort these things out.

The question is how do we get from the Deadwood of 1877 through the plain industrialization of Henry Ford's assembly line and the infrastructure build-out of the past century in the West to say modern day San Francisco or to the marvelous gleaming skyscrapers of Manhattan and the state of the art in Europe and the Eastern Tigers. I am hoping we don't need to go through Great Depressions and World Wars in order to step into the metaphorical 21st century of software.

The internet of course doesn't stand still and the web development world has routed around the various tree trunks that blocked the trail: just look at the new canvases that are being drawn and the GreaseMonkeys popping up everywhere. But maybe that is being a little naive, everyone ultimately has to pay the bills. Maybe we are all inward-looking criminals and smugglers despite our protesting our good intentions and adherance to standards. Who knows?

What I'm really looking for, and I think the ultimate users, those citizens of San Miguel, are looking for, is an escape from the Wild West. We're not Searching for John Wayne or like Morgan Freeman seeking Forgiveness for our partners, we don't want to Dance with Wolves like Kevin Costner (his last decent movie, remember what followed: the execrable Waterworld and that thing called The Postman?). At the risk of sounding like an anachronistic "Why can't we all get along?" Rodney King, a character that our man Clint would surely clown with a characteristically raised eyebrow, how about this:

Let's try to build a Wonderful World-Wide Web. Lets try to build it together.

Wild Wide Web Soundtrack

Essential tunes to this movie

• Robert DeCormier et al. - Kumbaya
• Ennio Morricone - The Good, The Bad And The Ugly
• Cutty Ranks - 6 Million Ways To Die - "Choose one"
• Kool Moe Dee - Wild Wild West I can't believe I forgot about this song. The lyrics stand in for everything that I've written; the aesthetic also is right up my alley: respectful but trying to see a way out of Deadwood. And some kind professor has recognized the significance of the way it deals with the problem of violence and placed the song online.

I used to live downtown 129th street
Convent everything's upbeat
Parties ball in the park
Nothing but girls after dark
We chill nobody gets ill
In the place we call the hill
But if you try 'em
That's when they will
Get wild but they don't fight they kill
At the...

Wild Wild West (Repeat 4x)

Credits (a chat transcript)

James M. Snell Mixing metaphors can be dangerous to your health ;-)
Koranteng ;-)
K    6 million ways to die: choose one
JMS    Death by Back Button?
K    the Bad Guys Always Wear Back
JMS    how Refreshing :-)
JMS    guess we can only move forward from here
K    absolutely... the quote for which I am most famous during my time on Freelance Graphics was

"Lets not give users enough rope for them to shoot themselves in the foot"

Mixing metaphors is my kind of thing
JMS    heh. it's not the users I'm worried about. it's all the darn developers
K    yep... and it comes down to frameworks and wikis and faqs and View Source ... and ultimately to Googlejuice: when Jane Programmer does a search they had better get good results otherwise they'll be hung high by their copy-pasted stuff
JMS    indeed

See also: The Unloved HTML Button and other Folktales

File under: whimsy, web, technology, adoption, standards, browser, software, satire, history, best practices, wit, parable, deadwood, story, storytelling, programming, design, patterns, movies, spaghetti, westerns, rest, microsoft, netscape, mozilla, lotus, ibm, strategy, Clint Eastwood, toli

The Unloved HTML Button and Other Folktales

A long, long time ago in a far, far-away land, there lived an HTML Button.

It lived happily with its parents, Mr HTML and Mrs Form, and its siblings: the older brother Text Entry Box and younger sisters, Drop-down List and Radio Button. As a family, they didn't look like much, but they got on with things. Indeed some would say that they had been second thoughts since grand-parents, Netscape, Microsoft, Apache and Berners-Lee, preferred to fawn and monopolistically squabble over cousins, Blink and Marquee Tag, the mobile duo, Java Applet and ActiveX, and the gruesome but quietly efficient twins, CSS and JavaScript. Let's not forget the ugly outlier, the Behaviour, and that perennial favourite of old Sir Tim, Semantic Purity. Still those pragmatic matchmakers, Fielding and Andreesson got together, sacrificed a few goats on the altar of sloppy expediency and dotcom hubris respectively and got the marriage together. The good Reverend HTTP was the officiating priest. Thus they called for a great many festivities and plenty palm wine and Schnapps was consumed by the merry populace. Yahoo! and Google, they ululated as they devoured the fatted Browser calf.

The Lord looked unto them and saw that it was good. He was pleased with his creation, thus he thundered: "Let my children prosper. Go forth and multiply forthwith". The VC Angel of the Lord blessed them with a strong essence and wide child-bearing hips. Verily I say unto thee, many offspring were spawned throughout the land. This allowed Uncle Amazon and Auntie eBay to garner many an oxen, selling little e-commerce gnomes on the child labour, and great work-ethic, of these precocious, and outsourced, children. The UPS town criers beat FedEx talking drums loudly and strew countless packages in villages from Kokomlemle in the south to Bolgatanga in the north, all the way to the far corners of this Flat Earth we live on. Big Blue-suited Armonkians day-dreamt of middleware and on-demand eBusiness, Oracles everywhere hummed expensive ballads of Databases, and itch-scratching Scandinavian Viking ants from the village of Linux (they were of the GNU genus) began to colonize many a continent.

Seeing all the fun they were having, Godfather Bray and Grand Wizard Paoli spawned another cousin to add to this fertile mix: the virtuous XML. Of course you have to understand that back in the old days, before all this boom-boom stuff, Real Men could hug each other without fear of overzealous prosecution.

XML was a boy genius. As an industrious type, he was singularly prolific and many offspring were issued from his legendary nightly encounters including the incomprehensible RDF and the Siamese twins XSL and XSLT (but we all have skeletons in our closets), and those sloppy but similarly prodigious critters, RSS and Atom, who founded a land called Blogosphere and struck fear into all and sundry so unruly was their behaviour. The Ananse's web that was woven was wondrous and resplendent in its glory. Hosannas were sung and there was much myrrh and frankincense which caused much mirth and frankly increased the joy felt by all who alliteratively assembled at the proceedings.

Periodically however, people got jealous and countless trials and tribulations were launched in the direction of this happy family. Where others would turn to ashes and sackcloths, throughout they kept the faith, and never complained about the insane contortions that those two wicked aunts, Gertrude UI Designer, and the quadruple-barreled Felicia Unsatisfied-Application-Framework-Developer, made them go through. Still, with the help of those Glue Layer People, Shadrach, Meshach and Abegnego who came armed with duct tape and spanners, they persevered in the righteous way. They were kindly sorts, steadfast, humble and true to the Lord. As I start my tale, the future looked ever so bright for them.

Lest you think I have too much time on my hands or that I don't have a day job, I should preface my folktale by saying that it was actually 10 months ago in Cambridge that I started writing a few fragments for what I codenamed the Toli Technology Series. And so, dear reader, the topic of this long-mulled note is that fair and foulweather friend of ours: the unloved html button and its form family.

Now 9 or more months from conception to birth is typical for me and for most mammals even if it is a long time in the technology world, and it leaves much opportunity for others to make much the same point and vividly so. Thus I can make no claims to originality or insight. I happen however to believe in Gabriel Garcia Marquez's maxim that

"Any idea that couldn't stand a few decades of neglect is not worth anything".

My first novel is certainly pushing in that direction but that is water under the proverbial bridge.

Before I get into our tangled tale of prodigal buttons, let me begin by recounting what prompted this current jaundiced delivery...

• A case of mis-directed advocacy
• The Unloved HTML Button
• MBTA Buttons
• Flickr Buttons
• Missing Blogger Buttons
• Why Specs Matter and the Law of Leaky Abstractions
• A Hyphenated Parable
• Acela Blues
• Forms in Browsers
• Pushing Buttons

A case of mis-directed advocacy

Over the weekend, in one of my episodic eruptions, I found myself trying to knock heads together to make sure that the right lessons had been learned in the reaction to last week's release of the beta of Google's Web Accelerator (GWA).

To give some context, the GWA is simply a client-side proxy server application that can speed things up for the user by making use of Google's wonderful infrastructure. Of course Google have other reasons (read metrics) for introducing this product but there is some real innovation here (additional compression of documents, the insightful idea that only updates to documents are transferred, the so-called differential exchange, and, most of all, leverage of the scalability of Google's unparalleled and massively parallel server farm). The browsing experience for millions of people could be markedly improved especially for those on dialup if this product is widely adopted. This is plainly an opportunity for the immediacy of communication, conversation and plain social intercourse to further flourish. I won't even dwell on the likelihood of this product further lubricating promiscuous commerce throughout the land.

Thus it would seem a caching intermediary like Google's should be fair game. Caches are used everywhere in the internet and there were a lot of tradeoffs that were made in the architecture to accommodate them. HTTP bends over backwards to make possible the magic of such intermediaries. Indeed almost all the differences between the HTTP 1.0 and HTTP 1.1 specifications relate to this issue of giving visibility to intermediaries. Tim Berners-Lee's prototype was turned into the "product" we know by repeated contact with the real world.

The trouble however is that many people, almost since the beginning of our tale, have been abusing HTTP and misusing its semantics especially as regards HTML forms and their offspring. People regularly use hyperlinks, the kind that you would type in in a browser location bar, to cause side effects. This goes against the injunction in the good book that goes under the name of idempotency.

Google's web accelerator in barely a few hours exposed how widespread this abuse has been. Like an assiduous cache who had read those ancient tablets, GWA simply assumed that there would be no side effects to performing an HTTP GET on a hyperlink on a web page, it did so and tried to prefetch things intelligently. Of course since these hyperlinks were doing unsafe things, sometimes things got deleted. Now this train wreck was predicted in the wise counsel of the Reverend HTTP in his opening address and the toast that matchmaker Fielding gave at the wedding where the whole world was invited to witness. Apparently however, many of the guests were drinking too much; their calabashes runneth over with Schnapps. The Yahoo! and Google choruses were rather symptoms of their inebriated bubbly state.

That being said, I was actually surprised that some of the brightest and most inventive developers around were metaphorically caught with their pants down, dead drunk, and, even worse, that they were then protesting their good intentions and rectitude. I was therefore fulminant when I weighed in:

There is an architecture to the World Wide Web.

Truly.

Really.

I kid you not, there is an architecture to this messy thing that we live on. This architecture was worked out in blood, sweat and tears (and a lot of fun during the dot com boom) over the past 15 years in countless mailing lists, with countless dear lessons learned.

Now when you start a post in that vein, what is to follow might get a little heated. As ever, I should have been calmer and like Joe Gregorio framed that episode as a parenting lesson:

I'm sorry, I can't kiss it and make it better.

I firmly believe there are just some things that a child will only learn through experience (read pain). Many times children can just be stubborn, or choose not to believe you, and ignore your warnings. I understand this. I did this myself. No matter how many times my parents told me that the iron was hot, it wasn't until I actually touched it and burned my finger did I believe them. That feeds into my parenting strategy, if they misbehave and end up in pain, I believe that's the right time to reinforce the message, make sure the lesson is tightly coupled with the pain, that way they're less likely to forget.

Instead my discourse devolved into a case of I told you so, a kind of Old Testament view of things instead of the softer New Age stylings that are in vogue these days. Sure there was a little concern for the users that had been hurt by lost data, but there was almost no empathy for the developers who had to lose their weekends furiously reworking their applications to do the right thing especially because it appeared that they would rather persist in trying to do the wrong thing.

The sentiment behind that mini tempest-in-a-teapot however was a recognition of the fact that those who have been quietly evangelizing the web style were talking about the wrong thing and to the wrong people.

Advocacy is great on xml-dev and the various forums where the REST/SOAP debate is played out but it is missing entirely the audience that counts. We might as well ignore those RPC and "complicated framework people" even if their claims are sometimes so unbelievable. Thus I believed that my own REST intervention and the countless other inquisitions were a touch mis-directed. According to de.lici.ous, 30 people appreciated my piece; my efforts would have been best directed at the 3 million or more people out there who are simply trying to build web sites and web applications and proliferate those gnomes.

As a community, we should have been like the Web Standards Project who stuck to their guns and instead compiled a litany of examples of best current practices that everyone could have seen, understood and copied. There is no reason that developers should only now be sensitized that HTTP GET SHOULD be safe and not have side-effects.

Your first encounter with a web application or web site should have this lesson hardwired in it. The recipes that you did view source on to learn about the web (like the thousands of sites that have lifted the stylesheets from the CSS Zen garden) should have been had such practices encoded in them so that it would be second nature.

Zef Hemel (his tagline: "your daily tech snack"), who in my folksonomy is tagged with very little tongue-in-cheek as "The next Linus Torvalds" later admitted to this blindspot

But to be honest, I wasn’t so aware of the badness of using GETs for this purpose until yesterday.

All this to say that the things one thinks are obvious are often not.

The View Source imperative was crucial in how most people learned about HTML and the Web and to its widespread adoption.

As I wrote those words, I looked over to my bookshelf and saw my dogeared copy of the 1995 edition of Laura Lemay's Teach Yourself Web Publishing With HTML In A Week which should be a collector's item for all historians of science. I smote myself upside the head as I remembered how irascible young me had gotten onto this web of ours. I recalled that I first fully read the HTTP 1.1 spec in the year 2000 out of a sense of curiosity and when I saw that great dissertations were being written about the design philosophy behind it. I had heard rumours that there was much that was good in the web and like everyone had seen the daily evidence everywhere. Instinctively I had realized that some of the decisions that had been made at its formation were simply inspired and had subconsciously followed those design patterns and best practices that had been implicitly articulated. On the other hand, I had lived and written web software for 5 years with only the occasional peek at the HTTP spec to learn about things like content negotiation.

And that's the nub of the issue, who other than curmudgeons, has the time to read the HTTP spec where the importance of idempotency is prominently inscribed?

The vast majority of the world won't read long treatises. People will however copy and paste your code. Thus, in addition to good well-designed specs, we also need good samples and documentation.

Joe Gregorio's Show Me The Code tactic is the right way to go about education as is Mark Birbeck's quiet and repeated advocacy (first RSS Readers in XForms, then Google Suggest using XForms and now Google Maps using XForms). I bow down to these folks and others who are actually taking on this heavy task.

The prescient Ryan Tomayko of course was in the mix anticipating these sorts of things just weeks earlier: On HTTP Abuse

There's been a lot of good discussion around Udell's End HTTP abuse article. We need to get this figured out because it's almost embarrassing to be an advocate of standard approaches to building web applications when something as fundamental as the correct use of HTTP GET is butchered so often. Unfortunately, misuse of HTTP GET is just the tip of the iceberg. There's a whole slew of HTTP abuse going on out there (often in the form of neglect) that can be laid at the footstep of two parties: frameworks and evangelists. The frameworks suck and the evangelists aren't trying hard enough.
The small community that is coalescing around REST/HTTP should prioritize the following objectives above anything else at this point:

1. Raise awareness of what HTTP is capable of.
2. Fix the tools.

He then points the way to a more effective advocacy strategy namely

Can we take a page from their book? In my opinion, we're just as entitled to the phrase "Designing with Web Standards" as they are. We have a three legged stool too: HTTP, URIs, and XML. Except our stool is more like a three legged dog - you can get around but it is not quite optimal.

There is a lot more in that vein and it appears that there is a lot of soul searching among my fellow travelers. I would only add that there is also the problem that those lowly HTML Forms and Buttons are addressing. We can't afford to forget them in our efforts, indeed they might be more crucial than the rest.

The Unloved HTML Button

Thus I find myself turning back to that good friend of ours: the unloved html button.

It really doesn't get much love.

No one likes the standard HTML buttons and the other form elements. More precisely no one except users who are just trying to get on with things, and even they just see them as a means to ends that can range from awful bureaucratic process paperwork to buying that seductive, new-and-improved, ecommerce gizmo.

The first thing that Geoffrey UI Designer wants to do when he meets them is to change the appearance of html buttons. Designers just don't like them, they want to style them with cooler looking images or they want things to be hyperlinks instead. The stereotypical designers have come from the background of rich desktop applications where all sorts of interface excess has been possible. Also inevitably everyone wants to create "lickable" interfaces that make you drool. Not to be pejorative, but deep down every designer fancies themselves an aesthete ala Kai Krause or Steve Jobs and very few have looked to Web design patterns. Those Human Interface Guidelines of mundane standardization that made things like the Macintosh user interface much beloved by users everywhere are beside the point. Users might have liked them since after using just a few applications one would know how to use that darn computer thing without requiring expensive "training courses". But there was a sameness in the land and even Apple, prompted by that returned prodigal and effete Jobs, now breaks its own guidelines with increasing frequency.

"We want chrome! We want skins!", chanted the crowd as the unloved HTML button was dragged out for display.

As an application developer I like to get as close to the metal as I can and don't like having too many layers between me and the technology that I work with. Thus I often cuddle up in my midnight hours next to plain old HTML forms and buttons. In the land of the Long Tail of Application Authoring however, most people will use a framework to "program" their web site.

I would note that every web application framework has its own ideas about how forms and buttons should behave. Just in the past 6 months, the assortment of Three Letter Acronym frameworks that have been foisted on me (WCL, JSF and ODC) have each had their differing conceptions of how forms and buttons work. Needless to say, I've suffered mightily and all of these frameworks have caused much head-scratching. They leak mightily and that is because they are engaged in a war with our html forms and buttons.

In my weekend missive I inveighed, "read Joel Spolsky on Leaky Abstractions (more on that below). Let's not fight against the underlying architecture; let's understand what we're building on. The tradeoffs that were made in the complex system that is the web were not made without care."

When I got into the brave new world of XForms, I assumed that all of these lessons had been learned. Thus it was to my surprise that the first deviation from the XForms spec in IBM's first pass at an XForms Processor (on AlphaWorks) was a non-standard implementation of an image button trigger and a slider control. Now that processor was written as a guinea pig to provide input to the XForms 1.0 specification and indeed was an attempt to fit in with existing browsers. But some cunning UI designer or Framework Developer had gotten to them and they were trying, and failing I might add in these cases, to make more snappy UI widgetry within the confines of the native browser widgetry and infrastructure. Hmmm.

Now let's get concrete here ("You're so long-winded Koranteng. Boo. Hiss. Cut to the chase")

MBTA buttons

Since January 2002 and indeed just until yesterday, I haven't been able to purchase my monthly T bus Pass at the MBTA website. The web application framework that was being used in that site was only tested with Microsoft Internet Explorer. Having been true to Grandpa Netscape and Uncle Mozilla during the years in the wilderness, I would fire up the browser hoping to see a "Buy Now" button that simply never showed up in Mozilla. Thus 40 times in a row I was disappointed and tempted instead into my one monthly use of the great Internet Explorer (other of course than the frequent Windows Updates and the required testing of the products I develop). The story of Windows is of course another Book of Proverbs but that's for others to tell. I would hazard that eschatologists and numerologists everywhere would see some significance in the number 40. Presumably the Greek chorus of masses of unwashed Firefox users had more sway than my lowly monthly email missives to the MBTA tech support team about getting support for the aging Mozilla suite - again there's someone who doesn't get much love and who is being cast aside in this brave new firefoxy world.

Flickr buttons

In Flickr, I can't add a photo to a set or add a note because "To take full advantage of Flickr, you should install the latest version of the Macromedia Flash Player". Now I love Flickr, I have the greater part of the photographic record of my life on that service, but the very serious message they seem to be telling me is that they are above placing an ugly HTML button above my images. It is an affront to their sensibilities. I can understand that drag and drop organization of hundreds of photos might require Flash. But is it too demeaning to be able to simply click Add Note? It appears that they'd rather have the enhanced experience of Flash effects which can cause other issues as I've pointed out - see Cultural Sensitivity in Technology.

I can't blame them however, Gertrude and Felicia do have their seductive appeal, it does look prettier right?

Now that Buy Now button is one of those flashy mouseover-y kind of things. We're downloading 2 additional images instead of the five characters that came down with the page, slowing things up for those unfortunate to be on dial-up. There is no semantic value to the button. There's no tooltip to explain what it's for. The image doesn't have any alt text. It's less accessible wouldn't you agree? Users with different browsers had to beg for 40 months in order to be granted the courtesy to even click on it. If I had problems with my eyesight and needed to use a high contrast scheme, I couldn't leverage the operating system's built-in support for html buttons. If I had to use a screen reader, it would have been lost to me. As a developer, I would have had to write code to build keyboard navigation to the button to add tab indexes and the like. Ad nauseum.

I suppose I shouldn't complain, it is possible to style various html form elements many input or a submit buttons have lovely images. What gets me is when people don't use what facilities do exist and even worse go out of their way to avoid them.

What is wrong with a simple html button I sometimes ask?

But I lost that argument a long time ago. Indeed whenever I bring up the additional work required in the alternatives to HTML forms and buttons, since for example back in the old WebSphere Portal, I've had to clean up the accessibility and keyboard navigation messes of yore, I am treated as a wild-haired party-pooper claiming that the Emperor has no clothes. Recently I spoke of John the Baptist and Salome so perhaps this time I should say that I rather fear that a Delilah will crop my metaphorical RESTafarian dreadlocks, hence I tread carefully these days before I engage in Jeremiads...

Missing Blogger buttons

As a practical case in point of the perils of HTTP abuse, just fifteen minutes before I wrote this post, I had discovered a couple of broken links in my Best Left Unread post. Completist that I am, I was going through and adding technorati tags to my posts - the links were about the Completist Syndrome.

As a sidenote: you get a better idea of how to tag things with the benefit of hindsight. The categories or tags you place on things right after you've written something are not very good. Why is that?

Now it turns out that in Blogger's table listing of blog posts there are 2 links next to each other. Actually let's have a look at this:

2/24/2005

Edit

Best Left Unread

Koranteng

View

Delete

Let's leave aside the usability issue of why View and Delete are right next to each other. Deleting posts should typically be a rare occurence unless your former employer's lawyers object to your ruminations or more likely if the RIAA or MPAA come after you. But that's another story... Moving right along...

At 11pm, tired after a long day of work and with faulty muscle memory, I right-clicked on what I thought was the View link and opened it in a new tab. When a few minutes later, I switched to the tab I was a little perplexed by what I found. Thankfully, Blogger actually knew a little about idempotency and had a big "Are you really sure that you want to delete this post?" question with the post listed in its minor glory. But how was it possible for me to so easily get into this state? Why was it a link in the first place? I suppose that it's fair to put a trash icon next to a tag in Flickr, after all metadata is not that valuable. But a blog post?

Coincidentally, I sent the following enhancement request just yesterday to Monsieur Feinberg on a completely different issue:

Usability. At very first order right-align the "delete this post" link. In any case put it as far away from buttons that are typically used (Save and Cancel). Deleting shouldn't be a frequent activity. My own preference is to make the user have to look a little before deleting. There should be a little cognitive impedance before they delete. The system loses value when content gets deleted, let them work for it... Make them check a box and search for a button to click before deleting and make sure there's a great reason for them to do this...

Now as the Google Web Accelerator has made clear, this is not just an issue with parts of Blogger, Flickr or the MBTA. There is a fundamental problem with how people want to use forms, buttons and the user interfaces they build on the web.

HTML forms were indeed an afterthought. All the browser vendors wanted to lock you in their schemes of empire building and dreams of gold and glory. This meant that the forms that we did get were not as good as they could have been (they certainly aren't as pretty as they could be). I've gone through many midnight gymnastic sessions to try to get them to do things that Gertrude and Felicia asked. But they work, they are well known and their semantics widely understood. I would say to the my peers toiling away to fix their problem, let's tread very carefully and make sure the right lessons were learned.

Why Specs Matter and the Law of Leaky Abstractions

Your pedantic narrator actually reads and writes specifications. As you can tell, he doesn't just code or refer to himself in the third person, he can also write a lot. Indeed it is a pet peeve of his that not enough time is spent getting good specs together. Thus at this note's conception, he would write things like the following

It is sad fact of life that we don't get enough time to work on design and architecture; most of a developer's time is spent dealing with crazy schedules, task lists, status updates and the general drudgery of the "Software Process"... I was thinking about this during the rare design discussions during our meeting yesterday. So I thought I'd point out a couple of articles you might find interesting:

• Mark Pilgrim does some of the best technology writing and also implements of a lot of cool software on the web. Here's his latest piece, a short one on Why Specs Matter. His language can be a little strong at times, but he gets the point across. You can defintely do without specs and we are often tempted to do as much; but a good spec really does make a difference. Think of XML 1.0, HTTP 1.0 and 1.1 or TCP/IP which have bred entire industries.
• Similarly, think of 'leaky abstractions' the next time that someone tries to push their framework on you, whether it's EJB, WCL, EMF, SDO, JSF, Portlets or whatever the flavour-du-jour is. If you've ever had to spend an afternoon figuring out why the heck your tables or buttons don't do obvious things (the answer being that ultimately our output is html and no abstraction in an API can overcome the limitations of that underlying infrastructure), you'll know what I mean. Now understanding this is an argument for reducing the layers that you have in your software architecture. So for example in a case close to my heart XForms, if your underlying abstraction is HTTP, hyperlinks, XML Events and Schema and your output is HTML or some rich client UI, you should think very hard before introducing any additional layers with their own modeling and semantics since ultimately you're going to have to deal with that infrastructure. Joel Spolsky's Law of Leaky Abstractions is the final authority on that front. His canonical example is the ASP.Net framework's long crusade against my lamented HTML button.

My great friend Rob [redacted] (why don't you start a blog Rob instead of being a best-kept secret in [redacted subdivision of blue company]?) had this to say:

90% of everything is crap. That's Sturgeon's Law. Software is not excluded from this principle. We live in a mass-market, low-bid, first-to-market world. Our goal ultimately is to be less sucky than our competitors. Many of the quality things you point to, like XML Schema had amazingly long schedules, compared to our 8 releases a year, schedules so long that they could only be done by non-profits, or in a deep research group.

A written spec is key to giving an organization the flexibility to grow. Without it, adding new people to a project requires that the existing engineers take time out to bring the new members up to speed. This is the mechanism behind Brook's Law ("adding people to a late project only makes it later"). But a good set of project documentation can temper the effects of Brook's Law and provide some scalability. Imagine how much of an easier time we would have bringing new people on board, if we had a complete design doc for [redacted hotshot product].

The person in [redacted services organization] who wants to make a custom solution based on your component, the person in research who wants to use it as the base of a prototype, the co-op who wants to base a summer project on it, the lead of a new project in [redacted] who wants to reuse your code - all these people will seek information from you. You can either have a spec and a design document ready for them and be seen in a very positive light. Or you can come off looking like a fool. The difference is whether you are designing throwaway code, or code that has wings. Very little code goes far which does not have a spec or design document. You don't win an [redacted] Award for throwaway code.

Eventually we'll all move on to new challenges, and eventually active development of [redacted] will be canceled. Your code will be maintained by someone in [obvious redacted locations] who will have nothing but your spec and code to work from. And in 10 years, someone will try to revive part of your code, and without a spec, they'll be lost. Trust me, it happened to me, trying to revive parts of [redacted bemoaned product killed by monopolist] to write a [redacted useful feature] for [redacted new-and-improved hyped product]. So, if you don't write a spec, your name will be cursed in languages unknown to you in far away countries, and also by kids in middle school today when they try to figure out your code 10 years from now.

I can't add much more than that as far as development of software goes but that is sadly an unpopular perspective when it's often a case of aggressive, Real-Men-ship-code-and-batter-the-enemy warlike rhetoric (rest assured that Steve Ballmer types don't only exist at [redacted monopolist]). Sometimes a little estrogen is needed in the technology world. Further, as my own experience has shown, I know that specification-reading will likely remain an afterthought in the realm of application authoring.

View Source is the thing.

A Hyphenated Parable

I have a hyphen in my name. 2 generations back in the colonial Gold Coast, it was quite a popular thing to assert a family's identity and that is part of my story. Thus in modern Ghana you'd find many an Ofori-Adjei, Owusu-Ansah or Casely-Hayford. This assertiveness continues to this day. Naming is culturally important in my society; the outdooring ceremony of week old babies is their introduction to society and the ancestors are duly consulted so that the correct name is chosen. We pour Schnapps on the ground as the name is announced for the first time dressed as we our in ceremonial white and blue. A similar inventiveness in naming exists in those whose ancestors were 'invited' to cross the Atlantic Ocean in the past few centuries and, through various grisly legacies and traditions, I am now able to pursue 15 year searches for a certain Shaniqua who's got me woked. But I digress... In the French and Francophone traditions also, hyphens are big items: think Jean-François types. More generally in almost every culture, it is increasingly popular when people marry for a hyphen to unite the two families.

I like hyphens, they are promiscuous, sociable and attach themselves willy-nilly to all they encounter. Call me a hyphenphile if you will. It's no accident that my future wife has a hyphenated name (with the same initials no less); Freudians would say that this was a subconscious part of our mutual attraction. If we weren't patriarchal in our ethnic traditions, I might even be thoroughly modern and go for a quadruple-barreled billing for our children like Auntie Felicia.

The problem with hyphens is that well... Scratch that, let me tell you a bit about hyphens.

In computer science there is a general category of problems that have to do with delimiter characters (underscores, hyphens, colons, semi-colons, angle brackets and other special characters). Software is very concerned about structure and the parsers of our Tower of Babel file formats need to know where records begin and end. A similar issue is currently putting the tagging community into a tizzy with some favouring spaces (flickr, del.icio.us), commas (allconsuming) or semi-colons (furl) as delimiters. It's very confusing for someone who samples all these services and many an item has been mistagged by mistake.

I got my first credit-cards just as online ecommerce was picking up in the early 1990s so I have lived through the evolution of that industry's handling of hyphens.

I stopped using a certain Visa card, which was actually my introduction to Generation Debt, when it turned out that I couldn't use it for online purchases because certain backend systems couldn't handle the hyphen in my name. Circa 1999 however, this got sorted out. But that company lost the wool of the sheep I gifted to Uncle Amazon for half a decade. By and large these days, the credit card industry have fixed their issues, I am no longer forced to lose the hyphen when I order something online.

This is not the case with travel reservation systems however. The hyphen simply doesn't exist as far as they are concerned.

My very chatty travel agent was explaining all of this to me this past weekend, "The system doesn't take the hyphen. It's persona non-grata!", excited as he was to talk with someone who knew a little about the history of the Sabre system. I suspect that IBM built some of that core technology 50-60 years ago.

Most airline reservation systems handle hyphens gracefully but some don't, and, over the past few years, I've had instances where plane tickets that I thought had been purchased days earlier actually hadn't causing panicky travel agents to call me to figure out the mess so they could get their commission. I have literally lost hundreds of dollars when I've had to rebook the flights in question at higher rates. The curse of the hyphen struck repeatedly.

Thus I don't have my Delta frequent flyer number saved on Travelocity, Expedia or Orbitz. When I travel on that airline I have to remember to give my frequent flyer number at the check-in counter to get credit for mileage. As it is, I pretty much stopped flying Delta and indeed the last time I flew on Delta and gave my number, I was told that they had cancelled my frequent flyer account for non-activity... Such is life.

I booked a trip this past weekend and when I tried to check my itinerary on checkmytrip.com, I kept getting errors. I was worried and kept trying without success looking at my keyboard with bewilderment Caps Lock? No. Num Lock? No. Typo? No. etc. Then I remembered my conversation, checked my reservation slip, and realized that I had to log in with the hyphen removed from my name. Ofosu-Amaah became OfosuAmaah.

That's not the end of it however. I am regularly scrutinized by immigration or customs officials and of late, overzealous TSA officials on the lookout respectively for a certain class of economic migrant, drug smugglers and Al Qaeda splinter cells. I have always thought that my problems stemmed from the radioactive effects of carrying a Ghanaian passport, that reliable sign of economic desperation.

Another way to look at things however is that these officious officials are simply wondering if I am the hypenless person printed out on my plane ticket.

Thus they tilt their head askew as they scrutinize the face that I naively assumed soothingly said "trust me".

Hillary, when her husband's campaign made her become Rodham Clinton, chose not to use a hyphen and I think she's on to something. She may even accept campaign contributions from Delta Airlines as they struggle against bankruptcy. With some Schadefreude, I wonder if my double-barrelled hyphen boycott is having an effect on Delta.

Acela Blues

Similar fairy tales have been said about how the standards for rail gauges were developed in the US. Bob Frankston's sharp ears recently perked up when he heard about the current problems with the Amtrak's Acela trains and he speculated as follows: Acela -- A Casualty of Risk Aversion?

From the Boston Globe article on the Acela brake problems (22-Apr-2005)

Bombardier of Canada, the company that helped to manufacture the Acela, had to satisfy the Federal Railway Administration's "buy American" rules. This meant that parts for the train had to be designed for the United States. As a result, the Acela Express trains are twice as heavy as comparable European high-speed trains.

As I see it the issue is the constraints that lead to the trains being much heavier than the European trains. This is the price for risk aversion – it isn’t free. In the case of Acela trying to remove the risks by overbuilding creates its own problems and can increase the risk. Given their track record (sorry, I couldn’t resist) it seems that the Europeans made reasonable tradeoffs.

He backtracked a little with a possible correction a few days later

30-Apr-2005: Correction? A letter in today's Boston Globe noted that the heavier carriages were needed because unlike France and Japan the US trains work on existing rails. I can understand the design tradeoff between building new infrastructure and adapting to what is available. A reminder to be wary of stories that confirm ones expectations. Was it the right decision? Perhaps yes given a political climate that wouldn't accept the capital costs of a new rail line through the crowded Northeast Corridor.

And that was that you would say? But I don't think it ends there. It belies the question of why indeed the standard US railroad gauge is so different from those successfully used in continental Europe and Japan where the rail systems are models of efficiency. And this of course is fodder for historians of science and technology, train buffs and urban legend hoaxers (and debunkers alike) everywhere, nicely discussed here

The US standard railroad gauge (distance between the rails) is 4 ft 8 1/2 in (1.44 m). That's an exceedingly odd number.

Why is that gauge used? Because that's the way they built them in England, and the US railroads were built by English ex patriots.

Why did the English build 'em like that? Because the first rail lines were built by the same people who built the pre-railroad tramways, and that's the gauge they used.

Why did *they* use that gauge then? Because the people who built the tramways used the same jigs and tools as they used for building wagons, which used that wheel spacing.

[snip worryingly long 2,000+ year chain of plausible links]

So who built these old rutted roads? The first long distance roads in Europe were built by Imperial Rome for the benefit of their legions. The roads have been used ever since.

And the ruts? The initial ruts, which everyone else had to match for fear of breaking their wagons, were first made by Roman war chariots. Since the chariots were made by or for Imperial Rome they were all alike in the matter of wheel spacing (ruts again).

Thus we have the answer to the original question. The United States standard railroad gauge of 4 ft 8 1/2 in derives from the original military specification (MilSpec) for an Imperial Roman army war chariot. MisSpecs (and bureaucracies) live forever!

Now although this particular alternate history is something that many have tried to debunk, its essential appeal is that it serves as a fabulous object lesson about standards and how technology gets adopted when pragmatism and leverage is the key rule of thumb. Some things have longer lives than one would expect.

Update: Bob Franskton adds

You'd appreciate a show on the travel channel. If you take a train from China to Mongolia it goes through a stop where they change the wheel carriages for a difference in gauge and then the train moves on.

Perhaps I should ask my travel agent to book me a trip to Mongolia, I hope the rail reservations systems can handle hyphens.

Forms in Browsers

It seems to me that we have 2 ways forward to improve the experience of html forms on the web: the XForms and Web Forms 2.0 specifications.

If I were to return to my conceit, I would call them two tribes squabbling in the desert with Grandpa Mozilla feeding both sides at different times. On the one hand, you'd have the XForms buffs and there are many of them, who first saw the discord in the land: Gertrude and Felicia did have something of a point in that the Form family could be improved. Thus toiled mightily and tried to distill the kind of patterns that experience had shown were being solved in the wild.

Their advocacy problem is that they were tainted by their early association with that disreputable couple XSL and XML Schema from whom they adopted little XPath. Now XPath was a fair kid but would always be tarred by the brush of that loose XSL mother of hers. She had a few learning disabilities (or rather eccentricities) and insisted that all who entered her house take off their slippers and wear kimonos made of parentheses. Even visiting royalty, King True and Queen False from the land of Boolean were given her functional treatment.

Then there was the WHAT tribe, who were old browser calf types, now lean and mean, the stern and Operatic Håkon (screaming Howcome! Footprint! and Though shalt have no other solution except CSS and Javascript) along with latterday convert, the CSS prophet Hixie. The more pragmatic Hyatt who takes frequent Safaris may be receptive to the one but is a touch wary about how it might bite the Apple. The good Baron Brendan Eich is enigmatically philosophical about the whole thing as he nurtures the wily Firefox. Hopefully those grand viziers at the house of W3C will knock some heads together and all will learn to get along without suspicion. The jury is out for the next 18 months.

I have my hat working on the one but truth be told, I couldn't care which wins. I've pointed out some of the comprehension problems that might arise with XForms's choice of XPath see On XForms, XPath, CSS, Brevity, Syntax And More but even though I've been a heavy Javascripter I'd like to make sure that Jill Department Head doesn't have to learn script just to pull a survey together. Clearly this area remains a work in progress.

We are still in the baby years of the web, the idea is to do things right, to learn the lessons and fix problems quickly so that 50 or 5,000 years later people aren't hobbled in literal ruts about the descendants of HTML Forms.

Returning to the abuse of the buttons of my discursive tale, Phil Ringnalda devastatingly make a similar point

I wonder why buttons look like that:

(designers are looking for) an easy way of using JavaScript to tunnel their looks-like-a-GET link through POST.

Apparently in all the rush to read Section 9.1 of RFC 2616 and to misinterpret the meaning of SHOULD NOT, nobody is actually reading the part of Section 9.1 that says

This allows user agents to represent other methods, such as POST, PUT and DELETE, in a special way, so that the user is made aware of the fact that a possibly unsafe action is being requested.

Making the fire alarm look just like a light switch isn't an example of daring and innovative design, it's just dangerous. Pushing the table out of the way won't keep you from falling off the chair.

This is exactly Tim Bray's sentiment

"I can remember like yesterday a Content Management conference about 1997, a woman from a big computer company talking about how great it was when they switched their CM system over from custom clients to the browser: "It’s so great! The browser is so limited, so they had to throw away three-quarters of the buttons and sliders and pulldowns and options, and just do it with hyperlinks and simple forms... it was so much easier to use!"

The ugly duckling user interfaces of the web have worked to a surprising extent, if we do give them the extreme makeover treatment, I hope one will still recognise their fundamental utility. I can still see traces of little Michael Jackson in the face of the current train wreck.

Placing a cache closer to the user which is what Google have attempted is simply doing what Akamai have done at a different layer. And it makes sense, caches should be as close to the user as possible, the benefits in terms of the interaction experience are obvious.

I know that down at the ground level, one might be a designing a web site and the client might want prettified links instead of those unloved html buttons. The core of web user interfaces are not sexy yet they work, and users understand them. The visual cues about what may or may not be safe are well known.

As application developers we should ask for better forms, we should be demanding of browser makers things like XForms or Web Forms 2.0 to make sure that we can go beyond the kind of stilted usability that we currently have. Our users would appreciate our efforts in that vein but for now, they know what to expect. Until then application developers should push back when we are told to "do the wrong thing".

Pushing Buttons

Micah Dubinko who was one of the chief authors of the XForms spec has it right when he calls his blog push-button paradise. That is the nirvana we're aiming for: a Garden of Eden where Forms and HTML Buttons will finally get some love. Thus as I conclude my folktale, let me give you one glimpse of our teenager button. Go ahead click on him, he won't bite, there's nothing unsafe here, a few hormones perhaps. And I know a bit about idempotency so presumably I won't lose anything.

There. That felt good, didn't it?

See also Technical Arteriosclerosis and again On The Importance of Biting Satire

Update: James Snell drops further Proverbial Zingers in Not tonight honey, I'm having idempotency issues...

Update: Holy Serendipity Batman!. Those defending Hillary in the midnight hours, no doubt through an (idem)potent keyword-searching conversational engine brew or other, happened onto my hyphenated parable. My tags are accordingly updated. I do believe this land called Blogosphere is no Kansas. I love tomorrow.

(Enough already, with the puns. Like those buttons, get on with it.)

File under: whimsy, technology, adoption, standards, html, forms, software, button, xforms, web, satire, history, best practices, wit, parable, story, storytelling, programming, design, patterns, hyphens, delimiters, railroads, trains, REST, Google, Lotus, IBM, strategy, zingers, Hillary, serendipity, toli