Version Hell Revisited

A surprisingly large proportion of the issues a working software engineer deals with on a daily basis turn out to be cultural. My theory is that this is because software ultimately amounts to people problems, and that it's all about coordination costs and human factors, as I've written about previously. This is not to say that we don't deal with the hard problems, but perhaps the "soft" in software hints at the lingering craft aspect even as we try to escape from Deadwood and leap into our industrial revolution.

This is one reason that I've found that, similarly, a large proportion of the best software engineers I've encountered in my now 25 years experience have not been traditional computer scientists. The linguists, historians and even that rare anthropologist, some of whom picked up the profession on the side, or out of expedience as the web has allowed, have been the real dark matter of technology. The web with its radical simplicity, has allowed mass amateurization to prevail. Now mind you, you do need your stereotypical straight ahead just-the-facts engineers with blinkered and relentless nerdy application, and, increasingly as software becomes more professionalized, the guilds and dark factory mills have started appearing. But hold that thought for now, I have another story to tell, indeed you may call it a technology folktale.

I came across this memo written a long, long time ago, in a far faraway land, (to be truthful, it was a Friday afternoon and I was fed up). Certain names have been changed to protect the guilty, and don't assume the anachronistic references mean that it is of recent vintage. Consider it perhaps a glimpse at the way the software sausage is made and we all know the scandal of slaughterhouses although we now call them meatpacking plants in polite society.

The context is some wrangling where at least 6 teams were pointing fingers at each other about who was responsible for uploading some tools that The Company was using. The flow of data through the system had at length been established, and it was now a coordination problem more than a design and architecture problem. The problem was that different teams were responsible for different components. The email chain had been playing out for months it seemed. I was stuck in the middle making arbitrary decisions mainly by virtue of having written a core piece of the system. I kept asking and everyone kept dodging my increasingly pointed questions. And so I wrote the following:

Previously in the same vein: Version Hell

---

The Memo

From: Koranteng@Toli
To: A long list of interested parties
Date: Friday afternoon, a long, long time ago
Subject: Toolchain versioning Re: provide version number Re: Signing Re: [snip]

I want to tease out the various strands because I'm finding it hard to serve the competing masters here.

Let me pitch it as a folktale, it's a pandemic and we all need comforting narratives.

Master Johnson, my director, would be paraphrased as follows:

Our current process is the moral equivalent of taking a USB stick dropped by our Vendor in our parking lot, and placing that software in our august data centers to sign our Crown Jewels. Now this may be what we're reduced to, but don't blindly use any old tools we are given, let's at least make sure that we track and bless known versions of this toolchain.

There were stronger words said at the outset back when we had our deep dive on our last integration project late last year.

Master Security Team pitched in, again paraphrasing...

Well, in this sorry business, we've normally done integrations of these tools for a given year. We're already overtaxed and busy and we burn weekends to get the integration working. But once we've got it working, we've got clean hands. We don't touch it anymore unless there's an error using them. We're not happy with this but ye olde signing tools ultimately come from the Ministry of Information team. We're middlemen here and the Signing and Software Release teams are the ones that integrate these tools into The Company's IT solutions.

Master Ministry of Information Team is also in a pickle. He receives vendor builds and has to produce release builds with the help of Butler Jenkins. These blasted tools are just one piece of the overall puzzle (you should see what else is on his plate). His main requirement is to be able to reliably sign and update this software for the lifetime of The Product, a decades long quest.

His junior sister, Miss Android Team actually works closely with Messers Qualcomm, Broadcom and Intel, and gets periodic updates from Vendor Google who changes signing requirements at a pace of its own choosing, those Mountain View people move at the speed of the web.

I'll omit the other players who are also similarly overtaxed, although I should make special mention of Mistress Software Release Team who actually owns the official release process and is always playing catch up integrating with Information and with Signing and other things I'm not even aware of.

Meanwhile I'm sitting like Ananse the Spider tending to my farm in the land of Signing and perhaps you can see the roots of my dilemma:

Each build could have a new version of the dismal signing tools.

Per Master Johnson, I can't blindly just sign with any old version of the Signing tools I am presented with. That man signs off on my annual review, so I cross him at my peril.

Per Mistress Software Release Team, I have a frozen API in place which doesn't specify the toolchain version. In any case, there is a long chain between Android, Information, Software Release to Signing, and the lead time to change things is measured in the cost of "Projects". We are booked for months, nay years ahead and agility is a problem here at The Company - The Company's initiatives on agility and the cloud native adoption notwithstanding.

Anyway my solution, as it is, is that I will support "blessed" versions of the toolchain. All that is left is for someone to upload these versions.

Now I'm stuck in the middle of all this, and so far it has been yours truly that is arbitrarily uploading versions of the toolchain based on what I've seen coming through in our test environment. I actually don't know what is a significant version and I don't really want to be in this loop. Further I am making decisions above my pay grade without any domain knowledge of these dismal signing tools.

The details matter and someone needs to own this and solve my riddle.

Again

• Who is it that uploads the "blessed" versions of the toolchain to Signing?
• More fundamentally, who decides what is a blessed version of the toolchain?

I'll end by invoking my friend Sam Ruby's Postulate

The accuracy of metadata is inversely proportional to the square of the distance between the data and the metadata.

I am clearly furthest from the data, so somebody closer needs to make the call. Perhaps I'm reading from the wrong playbook, but I'm merely trying to add value here.

Koranteng
--
Chief Toli Monger
The Company

[snip distressing long chain of emails going back 5 months of people passing the buck]

Postscript

I'd like to think that my late Friday email did have the clarifying effect I intended with its visions of Ruby's Postulate and The Tower of Babel. There were certainly calls for action, and immediate action at that. I learned that I had underestimated the extent of the dysfunction at The Company. There were actually at least 8 other teams that I might have added to my folktale, and they each had their own sorry origin story. The list of people inside The Company who I learned had read my missive was impressive and growing.

There were ruffled feathers however. People who should have dealt with the underlying organizational and communication issue were upset that I had revealed it so starkly. Lots of meetings were called. Action items were issued and so forth. I was advised to let others handle things at this stage. I was also reminded of John Kenneth Galbraith writing in The Great Crash of 1929.

... the rite of the meeting which is called not to do business but to do no business...

One of the oldest, most important - and unhappily, one of the least understood - rites in American life.

Men meet together for many reasons in the course of business. They need to instruct or persuade each other. They must agree on a course of action. They find thinking in public more productive or less painful than thinking in private. But there are at least as many reasons for meetings to transact no business. Meetings are held because men seek companionship or, at a minimum, wish to escape the tedium of solitary duties. They yearn for prestige which accrues to the man who presides over meetings, and this leads them to convoke assemblages over which they can preside.

Finally there is the meeting which is called not because there is business to be done, but because it is necessary to create the impression that business is being done. Such meetings are more than a substitute for action. They are widely regarded as action.

Everyone remembers the story of The Emperor's New Clothes, I should note however that Hans Christian Andersen didn't write the sequel about what happened to that little boy and his family six months later when The Authorities could finally deal with them.

Slightly related, the Gambian proverb goes "Words are like bullets, once you release them you can't call them back.

It is said that a prophet is never recognized in his own country and the verdict was still out on if I was to become The Company's John the Baptist. I won't tempt fate and leave it to the historians to make that determination. In closing, I'll again turn to Belloc

It is always a relief to believe what is pleasant, but it is more important to believe what is true.

— Hilaire Belloc

Office Tales

I'd previously pointed to this juxtaposition, a survival guide to life in any workplace

• Office life by Keith Waterhouse
• Office Politics by Wilfrid Sheed
• The Organization Man by William H Whyte, Jr
• Managing Humans by Rands in Repose
• The Bad Child's Book of Beasts by Hilaire Belloc
  Belloc's cautionary tales and bestiary of observed pathologies are essential for office life.

Soundtrack for this note

File under: software, technology, complexity, data, metadata, design, architecture, best practices, coordination, organization, social, work, office, observation, decisions, process, programming, versioning, humour, toli,

Blues of an Omnivorous Reader

Wherein I reproduce a couple of apparently futile musings on the announced death of Google Reader. I've always preferred that my public utterances be available in a feed and Google Plus by design doesn't provide one. Further I received a transatlantic call yesterday from The Parents who are alarmed and incredulous that such an important and incredibly useful tool - an essential part of their experience of the web, would be summarily executed, as it were. They raised a point that I hadn't considered as I've been thinking about Plan B. It is easy to say that there are other feed readers, but almost all the proposed alternatives will not be free if they operate at Google's scale, and this presents complications if you live in a country like Ghana which is a black sheep in terms of e-commerce. Although my parents are undoubtedly willing and able to pay for such a service, they will be locked out because PayPal, Google Wallet and most credit card payment systems are not available to Ghanaians. I'm going to have do the dance of getting their credentials, signing them up for Feedly, Newsblur or what have you and entering in my payment details. Now that's a workable arrangement for my family, but what about others in similar circumstances? I had posited, with some amount of hyperbole, that this was a massive destruction of consumer surplus but I didn't know the half of it.

1487 Feeds (or Blues of an Omnivorous Reader)

A belated, and certainly, redundant comment to this thread.

I suppose I am an outlier in terms of usage as Google Reader is my single most used web application (more than even search and email), but I'll have to repeat the question I've been asking myself for a couple of days now: how does one handle 1487 different content sources efficiently across the up to 10 devices that I might use daily?

The corollary question also comes to mind: how did I come to aggregate that many sources of data?

The answer to both those questions is Google Reader.

Chris Wetherell's thesis was that "feed reading is inherently polymorphic" and he would know as he designed Google Reader.

As I'm having to now export said 1487 feeds to try out various alternatives, I've been thinking through my usage of Reader and I must agree with him on that front, because what I see is a wide variety of sources organized however I want.

Some are blogs, some are news sites, some are topical, some sui generis, some polymath, some from people I disagree with, some are the web output of friends, some from family, some are photo blogs, some are podcasts, comics, or comments. Some are on current affairs, some on Africa (although that tag seems to encompass anything non-US), some on technology, some tagged as general interest, some pure entertainment.

Some are high traffic, some are infrequently updated. An example: a friend recently emerged after a 2 year disappearance in the new parent cone of silence. On his return to blogging, he seems to be no longer interested in writing about technology. No problem: simply tag him or add him to a different folder (feeds can be multiply categorized).

Some are work related. Some are very narrowly focused. Some are alerts that I want to monitor e.g. feeds in Flickr and Delicious (tags or people or groups), Google alerts. For a while I could easily monitor almost everything of interest about Ghana since writing on Ghana and indeed the Ghanaian web footprint has been so small.

Some are forums. Some are for research often more speculative than not (e.g. the two months when I mused about a dissertation on two-sided markets in technology). Some are for archiving purposes (e.g. various synthetic feeds of sorts), and of course there are the vanity feeds (e.g. the defunct Technorati watch lists or Blogdigger searhes, Friendfeed etc.) which are all filed in the Egotism folder).

Anything that could generate a feed is fair game and has an equal opportunity.

Some are full feeds (which I prefer), some only offer headlines and some have article summaries but I've found that using Reader to scan headlines and/or summaries is still more useful as a starting point to distill and decide what to open in a new tab.

The huge variety boggles and has matched my varying interests over time.

Humanity Critic can lie next to Maciej Ceg?owski who lies next to Malcolm Gladwell. Google Reader democratizes the process of information consumption.

All of these sources could be sliced and diced, categorized any which way and sorted however I want.

Each folder can be read differently since Reader accommodates different reading styles. Skimming is easy if need be. You could have a river of news with whatever filtering that you want. You could decide to focus on a particular source or topic and deeply explore.

Sort also is important, some things should be read in order e.g. Doonesbury, for others recency is more important.

Sort by "magic" used to be almost uncanny in surfacing what I found most interesting. The algorithm was changed a couple of years ago and it has suffered greatly (some of the higher traffic sites are overwhelming things) and the All Items view hasn't been as useful ever since in bubbling up interesting content. In mitigation, I've since segregated the high traffic but low signal feeds into different folders where they can be easily triaged and marked read to regain sanity. The loss of the old sharing features (and enforced move to Google+) was hard to take, and perhaps contributed to the decline in effectiveness of sort by magic. One click starring and sharing was the most wonderful thing while it lasted.

Starring and synchronization of unread state across multiple devices are of course the killer features.

The high information density is unmatched on the desktop and mobile. (Indeed the mobile web app is better than the Android app for what it's worth). Keyboard shortcuts for navigation and the visual design are the icing on the cake.

Others have explicated much of these design traits and at length.

A few other things:

• An invaluable debugging tool for feed development.
  It's hard to remember how long it took to develop robust tools to produce and consume feeds, not to mention how contentious the process was. Google Reader was a major driver and was at the heart of this process. Mark Pilgrim's myth of RSS compatibility is one iconic example in this vein. It was hard work. Now in 2013, every software product can produce a valid Atom or RSS feed, so much so that we can almost retire the heroic feed validator service. Atom stores proliferate everywhere even if only under the covers. Feeds are essential web infrastructure and like all good infrastructure they are only noticed in their absence.
• Archiving
  Google Reader's archives are second to none: so long as at least one person subscribed to a feed, its content was archived. This obviously includes archives of now-defunct blogs. For example, the only archive of Teju Cole's Modal Minority blog that eventually morphed into his novel Open City is to be found in Google Reader, the Internet Archive doesn't have a copy. Imagine a future, or even current, student with a mind to write a dissertation on the most critically praised blog-turned-novel that we have seen. Well that student has 3 months to get cracking or there will be no discoursing on "The artist as urban exiled soul: witnessing the Bush years through episodic flânerie". Links are ephemeral you see.
  The archiving features have saved many a misconfigured blog's data.
• A Public Service
  In the same way that people in Iran and other countries were using sharing in Google Reader to do interesting things to route around censorship and political repression (something that came to light when Google+ forcibly became the "one true way" to share from Reader), the impending loss will be felt in other countries. It's hard to arbitrarily filter Google, it's too useful a bundle of services. Filtering Feedly? or Newsblur? Come on, let's be serious.

Arggh.. this was meant to be a one line comment namely:

Google Reader is the single most useful tool I've found "for organizing the world's information".

Not to be facetious, but asking one to articulate its value and what in its design can lead to the above sentiment is really sad. It's hard to believe that it's not obvious. It's hard to believe that it has to be articulated. Really. It's hard to believe.

Fairy Tale Endings

Responding to Ben Hyde's first take on the matter: Fairy tale is the right metaphor and Google Reader's is a cautionary tale akin to the Tower of Babel. Coordination costs is the issue and what we are seeing is a vast consumer surplus being dissipated.

And it hurts even if you saw the blow coming. I made my peace with the loss of sharing features a couple of years ago and the enforced push to Google+, I made my peace with many other losses, but now I'm been in mourning for the web.

Grievous bodily harm is being done to the web ecosystem. For a good decade, tools and infrastructure were built that wrangled the web with a view of feeds as an ideal medium for information dissemination and consumption. Google Reader's role was crucial and necessary.

Unlike Les Orchard, I didn't go so far as to actually write books or move home to work on the teams that built some of these building blocks (Delicious, Flickr, Reader) but I similarly evangelized and worked on them. It's a shame.

In retrospect, Mark Fletcher (of Bloglines fame) cashed out at the right time, from what I can tell his life post-Bloglines is a lot of safaris. All power to him.

Looking forward more broadly, I can see the outlines of how the story of Amazon will ultimately play out. For a moment that will seem all too brief in retrospect, consumers had use of a vast consumer surplus as the web was built and options value flowed in their direction. Then we heard about "Spring Cleaning"...

I feel sad writing this note in Google+.

Soundtrack for this note

• Donny Hathaway - A Dream
  "It was a dream"
• Whitney Houston - Why does it hurt so bad
• The Spinners - It's a shame

File under: blues, web, technology, feeds, adoption, destruction, economics, strategy, Google, architecture, design, information, consumption, Google Reader, observation, history, toli

Crawl Before You Walk

I've been working for the past 18 months in the "emerging web technologies" group at IBM. When I first joined, I was struck by a certain difference in attitude compared to my previous experiences at Lotus. "Eh?", I'd almost hear my colleagues say silently whenever I'd start discussing the merits of any technology. When they responded, there'd be a quite jaundiced sound in their voices. You'd notice the slightly arched eyebrow and perhaps a tightening of the mouth. It wasn't quite incredulity but it was fairly close to wariness. I speculated that this tendency was born of years evaluating countless Next Big Things and was simply the posture of experts and futurologists: "What are you selling this time? Tell me something new...". They've seen standards come and go and the various fashions that sweep the technology world. The group is a little guerrilla outpost of pragmatism, if you will, within Big Blue.

I am fairly cynical about buzzwords and the mechanics of technology adoption, but even my experiences of the Lotus side of things hadn't made me that skeptical. In any case I mention this because I was recently merrily surfing away and came upon the following bullet item in a laundry list of proposed features for Java Server Faces (JSF) 2.0.

Allow for bookmarkable JSF pages. More broadly, if HTTP GET can be used, it should be used.

Quite simply I caught myself saying "Eh? Eh!". My normally sedate eyebrows even arched violently. I couldn't help it.

I haven't closely followed the technology known as JSF, but I think if you wanted to quickly evaluate its prospects as an 'emerging web technology', you could do well to simply dwell on what I've highlighted. Its history, its rate of adoption, its maturity and perhaps even its future prospects are fully encapsulated in that most remarkable passage.

It is striking that it is only in its fifth year of existence that a framework for building web applications is considering allowing bookmarking and might even use the HTTP GET method where applicable.

How can one possibly develop successfully for the web without identifying important resources? That is the first plank of the REST elevator pitch. If you can't bookmark, if you can't cache, if you are invisible to intermediaries, you are willfully ignoring the web.

Technology toli readers might remember an intervention and case study on internalizing the REST style that focused on much the same aspect in the case of WebSphere Portal and its first 3 years of existence. The benefits that I outlined once the portal embraced the web are much the same that will accrue to JSF. Ed Burns and company seem to have staged a REST intervention on JSF. Head nods to them.

My first encounter with JSF was when it was announced at JavaOne in 2002. The name seemed funny ("faces" instead of interfaces?) but I'd seen worse. My snap judgement: yet another framework for building web interfaces, oh well, the more competition the better, we'll see how it pans out.

I note that I've only mentioned JSF once: outlining how its abstractions leaked mightily when it came to html buttons and forms. Three years ago, we had been forced to use it in our forms work because it was deemed "strategic" in the IBM Software Group. I didn't have the clout to affect the decision and, well, I worked to get something to ship to customers. Inwardly however I discounted that project's prospects of success based on my experience with the first releases of JSF. A demo that looks good will get you in the door; an application that works well will actually get deployed and used by Real People™.

In the same piece, I referenced the reaction to the release of Google Web Accelerator and the way it highlighted the large number of web frameworks that were ignoring the basic principles of the web architecture. It seems that those frameworks that were deployed on the broad web and whose flaws were publicly exposed in that episode took that lesson to heart. Their creators have begun internalizing the web style to its core. To take an obvious example, the latest releases of Ruby on Rails seem to be almost religious in the fervour for the gospel of REST.

I suspect that JSF wasn't widely deployed outside of 'enterprises' two years ago hence the outcry that many other frameworks faced to "fix your damn code" wasn't heard. It appears that the message has since been conveyed and that fateful item speaks volumes. REST it is. Sun is going full bore towards embracing the web. And about time too.

Laissez-faire Dynamics

I have described REST, the web style, as being laissez faire distributed computing.

This is a blessing in that its laissez faire approach has the virtue of encouraging participation by as many people as possible. Indeed its embodiment in HTTP might be one of the most successful cases of technology adoption we have seen.

To pursue an argument by analogy on the other hand, I'll note that markets regularly fail. There are the temptations of monopolies seeking rents and that the economics of affluence and attention apply. There are bad faith actors, and you need to create spaces for market lubricants: price discrimination, market makers, arbitrage etc. Income inequality, trickle down economics are widely seen when we fetishize all things laissez faire. Still, there is almost no market which is fully laissez faire; we have laws and regulations even in the freest of markets.

The reigning tension of the low end theory is between participation and control. REST, in favouring participation almost to a fault, enforces very minimal regulations. HTTP is abused daily and advocates of the web style would catch hernias if they dwelled on the extent of the abuse and didn't simply get on with things. You'll only see gentle prodding in my handwaving take at Arrested Development

• Identify all important resources
• Use the correct verb
• Respect browser conventions
• Layer Stripping and Radical Simplification

There's no coercion in these suggestions. REST is all carrot and no stick. Now the thing is that once you've imbibed the web style and embraced its minimal restrictions, you don't look back and all those externalities begin to accrue. Based on the urgency of that bullet point, I am fully confident that the next few releases of JSF will be bonafide Next Big Things ®.

Most of us had to crawl before we walked, or fall off the bike a few times before we learned how to ride. I only wish that certain technologies would fail faster or at least blunt their noses earlier. Contact with a mass audience on the web seems to be essential in this respect and this presents a dilemma.

Big companies have credibility by virtue of heft and inertia when it comes to purchasing decisions - the technologies they promote get in the door almost by default. Those who sell to big companies also know that, so long as they make the right noises about the strategic direction they are going in, they will get the benefit of the doubt. I would argue that this is only true in times of affluence. When we are all minding our purses, we look more closely at these things and will use what is expedient. Contrast the spread of web native applications like wikis to vertically integrated teamrooms and content management systems. Moreover, the web radically levels the playing field. In the decade or more since the web was adopted as the preferred platform, we have witnessed an inversion of emphasis due to its vastly wider audience and scale.

I'm reminded of the words of a fellow traveller.

I suppose I'd call myself a pragmatist.

I do whatever works. I strive to fail faster. It's the results that matter, and simplicity (stripping layers) is celebrated because time is precious. We don't have time to waste on complexity and buzzwords; we're already behind on inventing the future.

Waiting 5 years before you adopt the native architecture of the web is almost inexcusable. The web won't (and didn't) wait that long. Others will route around you and their dynamism will be adopted by the marketplace. Now it's a game of catch up on that elusive thing known as mindshare and ultimately on cold cash, rueing all those missed opportunities.

The perplexing challenge of the web remains: how do we encourage things to fail faster? Perhaps we should all learn to say "Eh?" more often.

Crawl Before You Walk, a Playlist

A soundtrack for this note

• The Pharcyde - Passing Me By
  The humourous message of this song from The Pharcyde's hugely influential debut album is that obsolescence awaits if opportunities go wanting. The chorus is wist itself: "she keeps on passing me by".
• Was (Not Was) - Walk the Dinosaur
  Fun soul music that made the pop charts: "everybody walk the dinosaur".
• Count Basie - Sleepwalker's Serenade
  It appears that the web style is exploding things at Sun and other big companies. It is only fitting that we end with a swinging track from the essence of swing, the big band at its tightest, the horns, the driving bass, not to mention the Count's percussive piano, provide the counterpoint to the ironic serenade of the sleepwalker. The album title: The Complete Atomic Basie. Eh!
  
  
  

File under: rest, technology, adoption, jsf, frameworks, design, architecture, web, http, forms, java, j2ee, platforms, software, rejection, pragmatism, best practices, toli

The Low End Theory of Networks

Ethan Zuckerman has been pondering generativity and aggregation, prompted by Jonathan Zittrain's paper, The Generative Internet and its implications. Many of these themes have been stewing in my head for quite some time so I thought I'd finally join in the conversation with some of my armchair punditry. As seems to be my custom, a one paragraph comment somehow turned into this note. Sadly I don't quite have a toli code to contribute to the fun, and I've already narrated a couple of gospels recently. Thus I'll switch tack and change the frame. This time I give you a theory: the Low End Theory.

• Control versus Participation
• The Rumsfeld Taxonomy of Networks
• The Low End Theory of Networks
• Postscript and Soundtrack

Control versus Participation in Networks

It's interesting that the lawyers, from Lawrence Lessig on, are weighing in on these network things and it's about time... Generativity huh? Zittrain argues a legal case built on abundant economic evidence (and one hopes the linguists would also weigh in). It's a nice restatement of the End-to-End principle in terms palatable to lobbyists. And the argument is much like Lawrence Solum and Minn Chung's paper The Layers Principle: Internet Architecture and the Law from a few years ago.

Reed, Saltzer and Clark described the End-to-End principle quite simply in terms of systems design. The heart of the matter is this oft-overlooked sentence

Functions placed at low levels of a system may be redundant or of little value when compared with the cost of providing them at that low level.

They knew when they were writing that this notion had wider applicability than the telecommunication networks that were their initial focus hence they labeled their work "end-to-end arguments in system design". The costs that are borne by the "system" are a generativity tax if we use Zittrain's terminology.

In economic terms, if you read the system as a market, restating this principle turns it into a matter of preserving options value. Black and Scholes have a lot to say on this front.

I see this same notion everywhere in the software engineering that I practice. The most successful system in distributed computing has been the web which, by design (pdf) and to a fault, falls back on minimalist protocols and data formats to handle coordination costs and the human factor.

But there are also tensions at work in the design of any system, and hidden assumptions or vested interests at work.

There will always be a difference between the value of a system to its users and its value to its operators and this is perfectly expressed in networks. At issue in this discussion is who gets to see their utility maximized. As Martin Geddes puts it

If there’s one lesson of the Stupid Network, it’s that there’s a massive increase in consumer surplus. The value to users diverges from that to owners. You can’t measure user value by looking at industry revenue.

A priori you have no idea what options are possible at the edge of your market. This means that the design of the core of your system is a political decision. Like an options trader you are gambling on outcomes in the marketplace and attempting to manage risk. Your design decision is anything but neutral despite what the so-called "network neutrality" proponents say (even if I do agree with their endpoint market outcome).

We know that price discrimination is an optimal pricing strategy from many standpoints and, in market utility terms, it makes sense to optimize system design to enable some modicum of price discrimination. The evidence throughout history, however, is that participants in a market hate price discrimination and favour uniform or predictable pricing. For example, flat rate pricing made the fortunes of AOL, and block pricing is remaking the cell-phone industry. By analogy, Paris Metro pricing is not widespread in transportation systems and on the internet because of these explicit user preferences.

Part of the challenges the US airline companies currently face is that we aren't sympathetic to them. One major reason for our disdain is the pricing games that are played with plane tickets. We value fairness even though we are creatures of a rough jungle and there's likely an anthropological basis for our sense of indignation at price discrimination. Perhaps our past ancestors were optimistic planners by nature. However the presence of price discrimination in a market, coupled with flexibility of pricing, and the liquidity of relative transparency allows for the existence of middlemen and the possibility of arbitrage and disintermediation. On the whole, these are good market outcomes but, like many "options", they can't be characterized up front.

Where there are networks, we will find power laws that apply. Similarly where there are markets, we will have sharp elbows and monopolistic temptations as we scramble for the loot. There are echoes of F. Scott-Fitzgerald's notion that "the rich are very different from you and me". These factors are found in every part of the technology industry and, when combined, they play out in the Great Game of consolidation and lobbying. The big network operators will happily oblige in their strategies since lots of advantages accrue to the early movers and great powers, and they can leverage consumer inertia. In the internet we have many intermediaries at the network level; firewalls, middleboxes and the domain name system are potential, and actual, Great Powers (see the Verisign Tax ® we all pay and the "offers you can't refuse" that content delivery networks like Akamai make).

Moving up a level, intermediaries are acknowledged up front in the web architecture as design constraints. I have cast the web's embrace of visibility to intermediaries as Caesar's Tax Collector Principle and I think it is fitting, taxes are good, on the whole, and especially estate taxes - they keep the trains running and the streets clean at least in my neck of the woods.

The Rumsfeld Taxonomy of Networks

If we take Donald Rumsfeld's taxonomy of knowledge as the starting point for our analysis of networks, we find the following.

Unknown Unknowns

Legislators and vested interests tend to worry about "unknown unknowns" and, like Rumsfeld, will focus on threat models and such. This is a matter of governance and regulation; among the core competencies of most governments are the management of information and regulation. Excessive focus on these leads almost inevitably to data mining, intimidation, the co-option of a complaisant although ostensibly independent press, spying by the NSA , movie-plot security and the like, your basic Global Wars on Everyone.

Unknown Knowns

The "unknown knowns" are our unconscious biases that frame of the discussion of networks. The metaphors that different sides use are instructive and perhaps even my casting the conversation in terms of "sides" betrays my viewing this arena as a game of sorts (hopefully not Mortal Kombat or something). Others may view the networks scene as a Clash of Civilizations which would lead to apocalyptic wars and the End of History if one follows Manifest Destiny.

Known Knowns

The "known knowns" in networks have been visibly demonstrated in the economic verve and activity that is taking place on the internet. Even if the four horsemen of the internet ™ were oversold, we can still point to the ongoing transformation wrought by the Four Horsemen of the Web. As an engineer, I tend to think in terms of protocol, hence my nomination of The HUHXtable Quartet (HTTP, URI, HTML, XML) towards that designation. Your mileage may vary and various prognosticators seem to be weighing in with speculations about the identity of these horsemen. With corporations being legal humans, they have assembled an intriguing cast of characters to liven up the debate.

These same historical forces were at work in the history of communication and transportation networks (pdf) and, although we celebrate the Wright Brothers and Henry Ford, who is really celebrating Samuel Homfray, Richard Trevithick, and the various others who played a role in the history of railroads or even, more recently, the extraordinary innovation brought about by the lowly shipping container?

Known Unknowns

The sweet spots in our analysis are the "known unknowns". These are the things that make venture capitalists salivate, and monopolists turn to paranoia and worse. It's that seductive notion of the startup in the garage that can change the world, of the eBay or Craigslist economy, of the manufactured serendipity and furious and creative energy that has been on display in the past decade or so on the web. This is where that Long Tail notion comes into play, to shamelessly mine another meme. There is the sense that all you need is an idea, and good connections. The fully leveraged network can be a great leveler and promoter of the innovative forces in a system.

Sidenote: This bit about manufacturing serendipity on the road to riches meshes well with the American Dream ™ and it is worth commenting on a little. Paul Graham, in his hermetically-sealed world, apparently believes that a start-up culture is a particularism of the United States (or condensation as he puts it). What a peculiar notion. While he's sleeping soundly at night, after a lullaby of unshakable Manifest Destiny, I'm sure that some Teutonic engineering will suddenly emerge (and he misreads the German economy so completely that he's a front runner for the huhudious awards 2006), or would it be an easterly wind that blows in from Korea or thereabouts that will give him fits early in the morning. I would have thought that the lesson was that the unknowns by definition are unknowable and, if you go ahead, following the Rumsfeld example, and discount even the knowns, you end up traveling in a lightly-armored Humvee on the road from Baghdad airport with a convoy of outsourced mercenaries. Good luck on that front.

The Low End Theory of Networks

But back to my topic... I'll quote Martin Geddes again since he is, as ever, eminently quotable

the end-to-end principle is really an appeal to preserve option value in a world of rapid technology change and innovation. By resisting this force, you’re either betting you can suppress rival distribution channels for competing innovation, or you can yourself be a lead innovator.

Thus Zittrain's pitch about generativity versus responsibility boils down to a tension between architectures of control and architectures of participation (using a much broader sense of participation than the current buzzword).

Stated another way, and since I traffic in coinages, indulge me if you will with Koranteng's first postulate of networks:

The End-to-End Principle in networks is another incarnation of the Low End Theory.

Recall if you will the core tenets of the Low End Theory which follows the Rule of Four as any good code should. Such is the mantra that schoolgirls the world over are reciting as they study Technology Adoption and System Design 101, and it is worth dwelling on:

• Ruthlessly leverage disruptions in the system
• Lower coordination costs through layer stripping
• Favour participation over control
• Temper the human factor to encourage adoption

As should be clear, each tenet of the Low End Theory encourages externalities to accrue in the system in support of preserving options value. Combined, they harness the collective energies of participants and harness innovation. We see these building blocks at work in hardware, in distributed computing and in the various Great Games that are taking place in the technology world. In this respect, I've suggested previously that REST, the web style, was the Low End Theory of Distributed Computing and have also written about The Low End Theory in Hardware. I don't see any reason why networks shouldn't be able to join in the fun, hence my current synthesis.

I like that Ethan is shrewdly focusing on the question of aggregation - that is also a restatement of the examples of eBay and Amazon. A good marketplace will surely allow for middlemen and aggregators, that's the bit about favouring participation. Aggregation however is only one of the styles that are likely to prevail in the market. Astute participants can focus on different areas and make hay.

As an example, it pays to identify the disruptions that underlie the evolution of the system, if you can hitch a ride at the right point, you can build mansions in Redmond. As to the second plank of our game theory, layer stripping restates the end-to-end notion of overlay systems, intelligence at the ends, and the daily reality of leaky abstractions. Similarly the human factor is a double-edged sword. On the one hand, there is the incoherence of the Tower of Babel which is the bit about coordination costs. On the other hand there are network effects to be gained in communication and group forming that argue for emphasizing community features in the system. From hunter-gatherers on, anything that enabled collaboration has given selective advantages in our evolutionary systems. We want to encourage the sharing of knowledge and information but encounter considerable costs in doing so. The ongoing dilemma we face is about how to build systems recognize the people along with the processes.

Operational efficiency is something that companies like Wal-Mart and Dell are good at, and operational efficiency at the margins is all the rage in the staid banking sector. As Willie Sutton quipped about robbing banks, it's "Cause that’s where the money is". Thus there are other ways to prevail in this marketplace. Rather than optimizing your processes around innovation you can optimize around operations, billing, search, storage, creating markets, or lubricating whatever friction there is in the system. To take an obvious example, the economics of peering is an interesting problem to tackle. Briefly stated it's Animal Farm all over again: some peers are more equal than others. Or call it real estate: location, location, location.

Similarly we are only beginning to realize the social implications of mobility in networks, of intermittent connectivity, of periodic synchronization and of the fluidity of our communal relations. Mobility is thus another of the major disruptions at work in the network marketplace and one that many users have voted on with cold cash all over the world. I'll note anecdotally that many colleagues of mine who were working on desktop collaboration software just years ago are now firmly ensconced at Nokia.

Where there is the realm of the social, there is the realm of conversation and its corollary, the market. To raise the tenor of this conversation, I'll repeat the insight of those 20th century philosophers, the Pet Shop Boys, and their master treatise:

There's lots of opportunities

You've got the brawn
I've got the brains
Let's make lot of money

Thus there is room in the network game for the innovation of Juniper and Skype and all the companies weighing in on the disruption of the internet protocols, and the operational effectiveness of say equipment manufacturers like Nokia and Samsung, and the more astute network operators. Everyone has their own exemplars on this last front: phone, cable, cell, fiber, copper, fixed wireless, wi-fi, ethernet... The TCP/IP suite that is the core of the internet has succeeded and scaled because the abstractions used embraced transparency and existing systems. The incredible ascendancy of ethernet over three decades (and of late wi-fi) is a testament to the value of simplicity and uniformity in network system design, and the concomitant scale and leverage that the low end mass market provides. Like the personal computer, these technologies are canonical disruptions and those who embrace their Fung Wah Bus aesthetic are inexorably moving up-market.

Some have suggested that good starting points for determining strategy are tomes like The Gorilla Game or The Innovator's Dilemma, I believe the jury is still out and the MBA types are best positioned to expound on their merit. I would point to Jim Gray's Distributed Computing Economics as my favourite starting point in determining to the best strategy. In any case, the game is on. In the interim, we are simply picking and choosing which of the hard problems we want to address. Still game theory remains theory; in real life games we have that nagging human factor.

Engineers are currently hobbled in our advocacy of the internet because we don't have good instruments for determining metrics for things like resilience and adaptability that could inform the decisions of policy makers, lawyers, economists, and those who really matter: Mr. and Mrs. Big Money. Politicians everywhere like statistics and large numbers in what passes for their policy debates, especially in this current silly season of nostalgia. We do have large numbers in the internet, the users, but unfortunately we only have waffly options pricing to throw around as statistics. I would hazard that muttering "options value pricing for networks" doesn't quite cut it when compared to the red meat of the corn and sugar lobbies or the pork of the military industrial complex.

One needs to dangle some glitter when we talk to congressmen, or votes or something. Thus I'm an alchemist searching for Black Gold. They told me it was a goldfield; it turns out that it's an abandoned coal mine in Pennsylvania, a dud concession. Still I've heard there are good things Out West on the frontier, that area called the internet, that web style I need to adopt. Thus the Low End Theory is a work in progress and one could easily get taken to the cleaners by the loaded dice of the other players.

Everybody wants their cut and, if you can tilt the playing field at whatever level you reside at in the network ecosystem, it makes sense to do so, especially if you are judged every quarter by the baying hounds of Wall Street. If you take the market view, the lobbying capabilities of large corporations, a long history of regulation in this sector, and governments' existential need to pass laws or exercise control or compliance of some sort (whether in the name of security, expedience or getting things done) are going to weigh heavily.

We engineers have been lucky to have built enough of the internet and web infrastructure before the lawyers, lobbyists and governments realized what was there. The underlying architecture has proven sound enough to have scaled several orders of magnitude and looks set for several more as the next few billions of humanity come on board. The fact that the internet (after three decades) and the web (in its second decade) are now treated as infrastructure says it all, and this is a Good Thing ®.

The language is also important, and I continue to wince at George Gilder's revolutionary talk although he is now (slightly) chastened. My reading is that we need to emphasize a sedate, gray-suited discourse to keep the bankers and traders happy and the loud "content" industry at bay. Infrastructure should be boring despite the breathless prose we see in the business rags. Technology was made prematurely sexy in the dot com era; the implications of technology are what are important, not the technology itself. The markets for cement and most other commodities don't raise eyebrows and nor should networks. The logical lesson of the end-to-end principle is that communication networks are about connectivity, everything else is gravy.

I'll suggest then that normalcy is what engineers should aim for in this discussion. The Generative Internet is a good contribution to the debate as it is rooted in an understanding of the engineering insight of the end-to-end principle and the way platforms are developed and evolve. The dissection of the personal computer industry is instructive too as an argument by analogy in the technology world. The exploitation of Moore's Law and the black gold of silicon, the sweet spot of the mass-market where Intel and now AMD have been able to make hay, the platform effects that Microsoft has been able to leverage in the mass market etc., all these continue to drive that Great Game.

The weakest part of Zittrain's paper is where he declares that the end-to-end argument needs to be superseded and that "end-to-end does not fully capture the overall project of maintaining generativity". The primary reason for this postulate is the rhetorical strawman he constructs of end-to-end neutrality. Those advocating what they call "network neutrality" are simply being cute. They are implying, for quite pragmatic and rhetorical reasons (read effectiveness of lobbying), that there are no costs to neutrality; that neutrality is value neutral. It's a nice trick as far as framing a debate goes, but it is a trick nevertheless and it should be discounted accordingly.

Odlyzko has noted that spending on communication services (and especially the connectivity component) is huge, dwarfing many other segments of the economy (and before Rumsfeld's folly it was even trending towards the amount spent on national defense in the United States). Those vested interests and a $300 billion dollar sector will buy lots of slick rhetoric and astute framing. Zittrain carries on to suggest that the inevitable endpoint of the end-to-end argument is a network teeming with, on the one hand, viruses and spam and, on the other, silos and walled gardens. Having raised the spectre of these undoubted bogeymen that obviously need regulation, we then require new paradigms and frameworks.

This is either misguided or flawed, depending on where you stand. The end-to-end argument, as restated in the low-end theory, stands up well to these charges. It is an argument about lowering coordination costs. Nowhere is it acknowledged in the original paper that there are no costs to be borne, or that there is any such thing as end-to-end neutrality. To take a concrete example of engineering expediency at work consider the implementation of congestion control in the TCP/IP suite of protocols. This is characterized by some purists as a layering hack. The upshot of current practice is that we have repeated injunctions for other network applications to be "TCP friendly" in order to preserve the commons and the kind of congestion collapse that was observed as the internet began to see increased usage in the 1980s. We can and do embed functions inside of the network systems that we develop and sometimes we even cross layers if necessary, this is simply pragmatism at work.

Thus the very premise of the end-to-end argument is that this is a matter of tradeoffs and decisions about who bears costs. Engineering decisions are never neutral, the low end theory is political and is competing with different styles in the marketplace. Its emphasis on favouring participation over control simply aims to tilt the marketplace in a direction that encourages externalities to accrue to the end rather than the core. Similarly layer stripping as a design principle in the core is about reducing complexity. In other words, it suggests a strategy for those who run the network about how they can reduce their operating costs and accrue their value in the marketplace.

Lastly I'll note that everybody has to deal with gremlins and parasites and, as Cory Doctorow has noted, "all complex ecosystems have parasites"; they are transaction costs in every marketplace. Things always fall off the back of a lorry, the banking sector tolerates a certain level of fraud etc. Further, these transactions costs are acknowledged upfront in the end-to-end argument. Also, the empirical evidence throughout human history is that silos and walled gardens (from the Berlin Wall to CompuServe) are unsustainable in the long run and that it is shrewd to bet on participation over control (and I hope Burma and North Korea don't give the lie to my optimistic outlook). Still this prognostication is only a small part of Zittrain's remit and perhaps detracts from his wider argument.

I'll acknowledge here that linguistically the generative internet is a good coinage, and perhaps it even works better than End-to-End when pitched to the average congressman. This suggests to me that the enduring value of Zittrain's analysis is likely to be in the branding of the debate. Still as the lawyers and economists gear up and build up their legal frameworks and paraphernalia of economic models to throw at us about the design of networks, we engineers should confront them with prosaic notions of building communities and enabling conversations and marketplaces. I hope the Low End Theory can similarly enrich our vocabulary in this light.

We do have secret weapons in this debate... My grandmother is a surprisingly fierce and effective lobbyist when she puts her mind to it, on topics that sometimes mystify me. Her desire to interact with her progeny and to get on the web to converse with us should not be underestimated. The evidence is clear that she'll even tolerate any amount of spam and the ever-present gremlins and parasites that prevail - in moderation of course. More generally there is the desire to reconnect with old friends (and perhaps old flames), the socialization impulse that lurks in all of us. This is why I try at every opportunity to advocate pragmatism and that we strive to build Sexy Mom Factor Software. Where soccer moms are the demographic the politicians go after, in networks we need to encourage the Grandma Lobby like those cunning financial folks who went after Scottish Widows.

To conclude, participation is winning out over control for the moment, but it is the eternal struggle and I am hoping that the current ascendancy on the internet is not a temporary respite. As with democracy in the Great Game of Politics, eternal vigilance must be our watchword. And to paraphrase he of blood, sweat and tears

End-to-End systems, or Stupid Networks if you like, are the worst form of networks except for all those others that have been tried.

The story is the same as it ever was in the Great Game of Networks and I argue here that the Low End Theory is King, or President, if like me you are a republican (with the lowercase r).

Postscript

I've decided that I like the freedom I've found living within the constraints of a series hence I'll cast this note as part 1 of The Great Game of Networks Series which itself is an offshoot of The Great Game of Technology Series which I hereby retroactively announce. The latter series started last year with some musings on The Great Game of Chips.

Soundtrack

The Low End Theory soundtrack once again comes courtesy of A Tribe Called Quest's 1991 album.

This time we'll hum along to Jazz (We've Got) and nod our head's to Ron Carter's bass:

Stern firm and young with a laid-back tongue
The aim is to succeed and achieve at the age of 21...

[The internet hit the prime time around the age of 21 with the arrival of the web]

So push it, along, trails, we blaze
Don't deserve the gong, don't deserve the praise...

[innovation and manufactured serendipity continues]

A brand new twist with the homie-alistic
So low-key that ya probably missed it...

[low end should be low key infrastructure]

Competition, dem Phifer come sideway
But competition, dem must come straightway

[violators need not apply, we can see you coming]

I take off my hat to other crews that intend to rock
But the Low End Theory's here, it's time to wreck shop

We've got the Jazz (x4)

So peace to that crew, and peace to this crew
Bring on the tour, we'll see you at a theatre nearest you

Next: Disruptions in Networks. Ergo, what's there to leverage?

Print version

File under: networks, end-to-end, systems, Great Game, technology, architecture, design, economics, control, participation, coordination, strategy, adoption, policy, innovation, distribution, aggregation, incentives, arbitrage, politics, observation, analysis, whimsy, web, internet, evolution, resilience, adaptability, essay, Low End Theory, toli

On Coordination Costs and Human Factors

I reproduce here what I thought would be a short email that I fired off this afternoon discussing the web style. I hope it might gain from wider scrutiny...

• On Coordination Costs
• The Human Factor
• The Gospel of REST
• A Data Digression

On Coordination Costs

In an atypically verbose exposition, Matchmaker Roy Fielding propounded thusly:

Don't get carried away. You won't find a constraint about "nouns" anywhere in my dissertation. It talks about resources, as in resources, because that is what we want from a distributed hypermedia system (the ability to reuse those information sources through the provision of links). Services that are merely end-points are allowed within that model, but they aren't very interesting because they only amount to one resource. The really interesting services provide many resources.

Note that this is just another way of restating Reed's law in relation to Metcalfe's law.

Similarly, Layer Stripper Bill de hÓra recently flirted in the same manner:

Some people when faced when a distributed programming problem, think 'I know, I'll expose a new service interface.'

Now they have n-squared problems.

View Sourcerer Joe Gregorio almost in passing shined a light on some cobwebs (on caching and authentication on the web):

"In the process of implementing httplib2 I also discovered some rough spots in HTTP implementations."

Putting these together and handwaving as usual...

It strikes me that the first two statements are all about changing the frame and making economic arguments for REST (Representational State Transfer), namely constraining system design to resources, identifiers and uniform interfaces in order to lower coordination costs... ergo the web style as an enabler of Reed's Law.

I hadn't seen such explicit harkening to Metcalfe and Reed in the past even though there has always been this notion of REST as incorporating end-to-end principles. I too have argued in this vein about a complexity and integration argument for REST.

In a similar thread Benjamin Carlyle put it thusly

Uniformity is key. Simplicity at the network level is key. Managing state transparently as resources is important for its own reasons

Intuitively, the argument about applying architectural constraints to get payoffs in terms of leverage has a lot of appeal to engineers. It seems however that we need some economists to weigh in here with some options pricing theory or other to give more heft to these arguments. Often, decisions about software systems are not made by engineers but rather by financiers and it helps to speak their language.

Instead of Reed's notion of group forming, it seems the argument by analogy for REST is about application integration and the barriers that obtain in that sphere.

Now I like the large numbers that we can throw out in these reformulations. The thing is that Andrew Odlyzko says (pdf) that both Metcalfe and Reed's law are bunk despite their evident appeal and ability to dazzle venture capitalists.

In that paper he sets out to get quantitative measurements and hard data and puts the value of communication networks at nlog(n) which is nothing to sneeze at, better than Sarnoff's linearity but far less than Metcalfe and Reed's estimates. Now Odlyzko was measuring the utility of the internet which encompasses more than the web and you can argue, following Sam Ruby, that the email and peer-to-peer styles, whether expressed as, Bittorrent, Skype or usenet, are the true winners in his numbers... Also you could argue with his methodology and I have my own quibbles, but it's a brave man who takes on Odlyzko... Thus I'll take his numbers in my handwaving, acknowledging after all that the web was the key enabler and popularizer of the internet.

The Human Factor

This gets me to the third quote from Joe Gregorio, namely the perennial rough spots and implementation quirks that are our daily bread as engineers trying to design and produce systems for the web.

We see daily abuse of HTTP and there are annoying glitches with the libraries and implementations that exist and what is deployed in the real world. Perhaps this is because REST is the web style rather than the programming model and consequently enforces very few prescriptions. We are only now seeing good frameworks geared toward REST; historically, HTTP client and server libraries have been minimalist.

Now it seems to me that this is about the place where theory meets practice and we get into the realm of pragmatism and leverage. In the wild we see

• the difficulties of interoperation
• differing interpretations of specifications if indeed specs are read
• backward compatibility constraints e.g. for leverage and adoption, HTTP 1.1 had to accomodate some of the pitfalls in HTTP 1.0
• difficulties in authoring structured data
• configuration and deployment issues (mime types, content negotiation etc.)
• competition with other styles, the web style exists in a marketplace of architectures
• vested interests and economic models e.g. limits imposed by shared hosting providers, asymetry of some broadband networks etc

With this in mind, I wonder if I can come up with a stab towards Koranteng's postulates on coordination costs

1. There is a natural dampening factor in the utility of distributed computing
   
   We can use Odlyzko's numbers as the lower bound in practice of network effects and Reed's law as the theoretical limit (with Metcalfe being a great popularizer).
   
   I happen to be reading Graham Greene's The Human Factor and, looking through some of the issues that hinder adoption, many of them could be summarized as comprehension or human variability hence I'll characterize the issue as the human factor. All that is left is to augment with some Black-Scholes options thinking and financial derivatives to package to CEOs
2. the human factor in technology adoption is sizable and its effect can be measured. Moreover I would argue that it should be recognized as an explicit architectural constraint in the design on software systems.
3. In the realm of distributed computing, this human factor is bounded by Odlyzko's limit and Reed's law.
   Mathematicians can derive the correct coefficient for me... 1/nlog(n) ?

The rest as they say is advocacy and implementation details...

We are operating with imperfect specifications, imperfect frameworks and imperfect implementations. REST as laissez faire distributed computing doesn't acknowledge these costs as architectural constraints but rather seems to go about it by encouraging best practices and hoping that, by existence proof, people will come to it... One can look at the high level requirements that have been articulated

• Simple protocols for authoring and data transfer
• textual formats for protocol and some exchanged hypermedia
• Sensitive to user-perceived latency
• Mark Baker's talk about "principled sloppiness" (i.e. "must-ignore style extensibility")

We don't tend to enforce many of these things in the deployed protocols. I wonder what other best practices can lower coordination costs and whether they can be encoded in protocol to remove the human factor...

Anyway food for thought...

The Gospel of REST

If anything this enables me to add Reed's insight to my nascent taxonomy (or is it theology?) of the web style which some may have come across... namely:

There's a tag: REST

There's a slogan: the web style

There's a Holy Book: Architectural Styles and the Design of Network-based Software Architectures

There's a Reverend: HTTP

There's a choir: the HUHXtable quartet (HTTP, URI, HTML, XML)

There are Four Horsemen: GET, POST, PUT, DELETE

There are prophets: (you know who you are)

There are pillars: Resource Modeling, Idempotency etc.

There are priests and tax collectors: the caching and other intermediaries. Ergo "Render unto Caesar that which is Caesar's" recast as the notion of "giving visibility to intermediaries".

There are angels and demons: a band of Apaches and various HTTP libraries which are alternately sources of delight and exasperation.

There's a Messiah: the browser (which comes with various pretenders: Firefoxes, Great Explorers, Viking Operas and Fruity Safaris).

There are red herrings: url opacity etc.

There are false gods: WS-*, crusty old architectures of appropriation etc.

There's the wilderness and prodigal children: WebDAV?

There's Mary Magdalene and the disciples: HTML and Forms.

There's immaculate conception: the virtuous XML.

There are worldly travellers: the three mobile kings JavaScript, Java Applet and ActiveX (some discredited) and a Flashy pretender.

There are scrappy offspring: Atom, RSS and Atompub.

There are gruesome Philistines: implementation details such as Structured Data, Character Encoding and Security.

There are elevator pitches, Cliff Notes and ballads: Sir Tim's lullaby of Web Architecture 101 is quite reasonable as a Song of Solomon

There's myrrh and frankincense: the web as conversation engine

And now there's the Promised Land ©: Reed's Law as the proverbial milk and honey

The parts I'm missing are the Apocrypha and Gnostic gospels (with Judas in the news this week)... but those should be forthcoming... As will the eventual accommodation by Rome as the official religion but then Bill de Hóra has noted that we are almost there.

[Update] Ernie Prabakar suggests that "Lo-Rest" works as apocrypha and that "SOAP is really gnostic - it focuses on the divinity of XML, to the denial of its incarnation in HTML." One wonders who Rome is in the technology world, perhaps Microsoft like he suggests... I'd note in passing that I've heard in corridors that the IBM Software Group Architecture Board is "looking at REST" anew. That's got to qualify as progress... Pretty soon I'll be able to publish an official gospel from my muddy trenches. Looking further down the line, there will likely be a split between the Catholic and Orthodox churches as the empire suffers from navel gazing and an East/West axis of discontent and eventually there'll be Martin Luther and the Reformation... I wonder whether I'll live see to see the Protestants of REST and if I'll recognize them.

A Data Digression

I noticed last year that Roy Fielding produced a white paper about JSR 170 (Java Content Repository) for his day job (pun intended) applying principled constraints to the modeling of content repositories.

I've been curious about the surprising inertia behind that specification having played with IBM implementations in the past few years. Perhaps however, the immaturity of implementations in that space is a tribute to some of these arguments about coordination costs applied to the marketplace of data (relational, object relational, XML, SQL, XPath, XQuery, ActiveRecord, ODMA, Spring, Hibernate, SDO etc).

I wonder if the Atom store dream is the way to go, namely rather than apply the constraint of an API and a language, Java, in a world in which we have a Tower of Babel of languages and persistence frameworks, it makes more sense to focus on wire protocol (as in Atom Publishing protocol) and wire format say Atom. In other words the greater payoff would be not in establishing a programming model (the JCR) but rather in moving to Atompub which is agnostic on the underlying programming model and lowers the coordination costs by stripping a layer of comprehension from the mix. All this of course is modulo the quirks of compound documents, media collections etc...

The web worked because it was an overlay system that acknowledged existing systems and encoded much of its benefits in protocol rather than API. At the current stage of development in the software industry, it appears that the combination of protocol and data formats rather than API is the more effective approach to lowering coordination costs and dampening the human factor.

File under: REST, design, architecture, technology, coordination, economics, web, human factors, adoption, comprehension, costs, theory, systems, software, leverage, optimization, language, data, jcr, jsr170, toli