Sunday, March 19, 2006

The REST Elevator Pitch

The Pitch

I somewhat share Ryan Tomayko's concern for a shorter hand language for describing REST (Representational State Transfer), but I believe that the REST elevator pitch is pretty succinct already. It packs a lot of insight into the web architectural style of which dissertations have been written. The language used in technical debates is always important and rather than devolving into the fuzziness of "highs and lows", I'll suggest here that, as a matter of advocacy, one should stick to the elevator pitch.

To recap, the REST elevator pitch is:
  • Identification Of Resources
  • Manipulation Of Resources Through Representations
  • Self-Descriptive Messages
  • Hypermedia As The Engine Of Application State
In its most common application, the pitch will lead you to the HUHXtable Quartet (HTTP / URI / HTML / XML) if we handwave away the additional content types that the web style enables you to negotiate. It is quite clear that applications of all kinds have been built on these fundamentals for both human and machine interaction.

The thing is that each plank of the pitch brings its own benefits, benefits that perhaps can accrue independently. Still you get the most bang for the buck at the internet scale by the principled combination of all of them.

In my advocacy I tend to start with resource identification as the most important plank. The reason for this is that everybody can understand the value of identification. My grandmother understands the value of a bookmark to my photos. Heck we are even seeing the social benefits of ostensibly selfish bookmarking. The importance of resource identification cannot be understated and it leads us to the primacy of the URI.

The second plank, manipulation of resources through representations, is the bread and butter of HTTP, where we have an evolving set of standard formats of hypermedia exchanged using a small set of verbs.

Unpacking the HTTP proposition leads us to the four horsemen of the web: GET / POST / PUT / DELETE

Of course you can do very well if you pick up only 2 of those verbs, GET and POST. The good Reverend HTTP is a pretty tolerant fellow and won't begrudge those that don't believe in the rule of four, although in admittedly imperfect analogies, there's CRUD (Create / Retrieve / Update / Delete) in the database world and, as far as this layered internet goes there's TCP / IP / Datalink / Physical Layer (with say Ethernet or "Wi-Fi" 802.11b/a/g)

On the other hand, if you want to be a good citizen in the town of Deadwood and indeed if you want to survive in the wild west of the web, you shouldn't abuse HTTP. HTTP abuse leads to user confusion, headaches for systems designers and a tragedy of the commons. I consider HTTP a deliberately minimalist compromise, laissez-faire distributed computing if you will.

In social settings making your intentions known can help smooth interactions. We may start with small talk but we eventually manage to discern where others stand - well most of the time at least. We all operate with imperfect information but being straightforward is often a great policy. I'm sure there's some game theory or behavioural economics that can point to the value of transparency. Perhaps it is trite in light of Machiavelli and Sun-Tzu to believe that disclosure is the best policy but at least in markets transparency tends to be the font of aggregated wisdom. On the internet scale, this translates into the notion of giving visibility to intermediaries. Arguing again by analogy, the internet is an agreement, a network of networks. You can do whatever you want on your network but to interoperate with others, a small set of primitives, the TCP/IP standards in this case, have been agreed upon. I would suggest that the web style is similarly such an overlay system. REST is an architectural style which by principled design, is optimized to lower coordination costs.

Acknowledging the benefits of idempotency, of doing things safely, will lead one to making sure the use of GET is safe. Idempotency is not just about safety however; in clearly signaling your intentions, it is also about replayability. Thus idempotency is not just about the GET method. That is the wider argument for also using that oft-neglected duo, PUT and DELETE. Tunneling everything through POST (and the completely aberrant case GET) may be convenient but it adds unnecessary opacity and unpredictability about one's application. Unpredictability in the world leads to invasions and worse I might add. The use of those four core HTTP verbs helps intermediaries understand and participate more fully in the ecosystem.

The last two planks of the REST elevator pitch, self-descriptive messages, and especially that last one, hypermedia as the engine of application state, are attempts at addressing the issues of state, caching and evolving systems amongst other things.

Now of course in the preceding paragraphs, I've handwaved away entire dissertations that have been written about architectural styles and systems design, tomes written with language more precise and concepts more subtle than I've outlined. Good samples and running code are often also a determining factor in which style or application is adopted thus the view source imperative is also one of the reasons that web has caught on so quickly.

The Hard Problems

I've recently been thinking about defining the hardest problems I've encountered in software engineering, my cursory top 10 list:
  • State
  • Caching
  • Latency
  • Concurrency
  • Search
  • Metadata
  • Persistence
  • The Holy Grail Of Extensibility
  • Structured data
  • Character encoding
Now I'm not a database person so I handwaved away all of those data peoples' worries in one word: persistence. Of course I should probably add a few more issues and indeed you probably have your own notions. Indeed I settled just two days ago on those last two, structured data and character encoding, when I characterized them as being the gruesome twosome of computer science. Perhaps also there's the issue of discovery and bootstrapping of a system but that could be considered implementation details. There are those who mention security and reliability as other hard problems and they certainly are. Still I'll also handwave those away not as technological issues but as transaction costs; the economic benefits of ubiquity and leverage are key here. It seems that humans are quite prepared to tolerate gremlins and parasites in their systems (social, biological and technological) in moderation of course.

What is interesting is that almost all of these issues are important even if you aren't dealing with distributed computing. On the internet, which is all about distributed computing however, these are crucial problems. Now addressing these problems is a pretty tall order I must admit, but it seems to me that the web style has concrete answers to almost all of these constraints. Furthermore REST also offers a couple of those very valuable system characteristics: resilience and adaptability, almost as externalities.

Search for example seems to be addressed by valuing the importance of resource identification, the URI, that leads to PageRank and other algorithms. If you start with a viewpoint that URIs are cheap, and if you model your resources appropriately, you can deal with concurrency quite reasonably. State, caching and latency are all to do with performance which is always a bear. To that I'd argue that the commerce that is taking place daily on the internet seems to be a good indication that things are acceptable even if they could be improved. Metadata is the eternal headache of course but HTTP will give you headers and soul to help a little.

Discovery is addressed by URI as identification, the notion of hypermedia (either link headers or references in the media) and perhaps the current best practices of introspection resources. Persistence and structured data, I'll argue, are being addressed, warts and all, by the relational database, the seemingly-unstoppable spread of XML and, I believe, the current ascendancy of feeds. On this last subject, the wiring of the web, I am cautiously optimistic that the Atom store dream could be an answer; i.e. the combination of feed format (say Atom/RSS) and a wire protocol - which perhaps the Atom Publishing Protocol or similar could be. On the question of extensibility, last year I started a response that I felt was needed due to Dion Hincliffe's unbelievable assertion that "extensibility was the Achilles heel of REST". It always takes me a year to respond to minor footnotes but if there's one thing I'll do in the next few months, I'll be sure to write about REST and the Holy Grail of Extensibility.

But anyway, that is getting ahead of myself. I've been told to save my musings for "the book" or to stick to writing the running code that will "show, rather than tell" yet I keep falling for this web as conversational engine. Still there are undoubted social benefits of these exchanges and perhaps conversations like these are another virtue and externality of the web style. In any case, consider this my long winded response, Ryan: in matters of technology adoption and systems design, let's stick to the elevator pitch.

Further reading:

The Sequel

[Update - March 27, 2006] It seems as if everyone is getting into shaping the pitch and the language has been quite interesting.
  • Danny Ayers remarked that we should go back to the source and points at Tim Berners-Lee's pitch from last year
    Web architecture 101
    • Things are denoted by URIs.
    • Use them to denote things.
    • Serve useful information at them.
    • Dereference them.
    That indeed is about as brief a pitch as some executives will be able to handle and it covers the essentials.
  • Tim Bray's pitch is fairly pithy too, and for the last plank he uses
    You expect to ship a lot of URIs around in the bodies of requests and responses, and use them in operations that feel like link following.
    Now that's quite a mouthful in a presentation but it works fine in an elevator. I think "dereference them" is more terse and captures much of the same insight.

    As an engineer who is interested in precision in language, I tend to prefer Roy Fielding's formulation of "hypermedia as engine of application state". It's short and alludes to both the implementation and the area of focus of the web style. But your mileage may vary and I think it depends on the audience. We should be interested in a pitch that works because the advocacy struggle isn't over even if some are already thinking about the future issues. In this light, it was good to see Joe Gregorio again emphasizing the Show me the code route he embarked on last year. That View Source Imperative is the secret ingredient as far as adoption goes.
Note that everyone seems to conform to the Rule of Four as far as bullet points in the pitch go. Interestingly enough, Roy Fielding used to go with a five point pitch until he merged what were his previously separate definitions of "resource" and "URI" into the phrase "Identification Of Resources". I've proposed my own four point plan as a model for ArRESTed Development or The Low End Theory.

I concur of course with the endorsement of the formulation of REST - the web style and not just because I seem to have juice (of the google flavour) on that front having internalized that phrase long ago in my interventions, but because it gives us a tag, REST, combined with a catchphrase. As we know, every movie needs a tag line. Especially in Deadwood: "6 million ways to die, choose one". That is the B-movie of distributed computing. We want to move beyond grindhouse pulp fiction and get some of those glamourous, golden Hollywood trophies. And if that fails there's always the Gospel of REST.

The Elevator Soundtrack

  • Outkast - Elevators (Me & You)
    The chorus of this laidback piece of Atlanta folklore could well be a stand-in for the inclusiveness of the web style and its architecture of participation.
    Me and You
    Your Momma and Your Cousin Too
    Rollin' Down The Strip On Vogues
    Comin' Up Slammin' Cadillac Doors
  • [Update] A hat tip to the good doctor, Ernie Prabhakar, who points to a real life Huxtable Quartet. When I first started using HUHXtable, I was thinking of the Cosby Show and of comfy sweaters, thus it's great to hear that the Slide Huxtable Quartet is itself a tribute to that show and to the "Play It Again, Russell" episode in which Cliff's father, trombonist Russell "Slide" Huxtable (Earl Hyman) hosts a jam session. REST has even got a house band.
  • Miles Davis - Ascenseur Pour L'Echafaud
    I've always loved Miles Davis's cool soundtrack to one of the great film noirs which saw a great print released last year. It's atmospheric and gets the job done - and with verve. I am always upset that ascenseur is translated as lift rather than elevator which is more heady to my ears. I'll conclude by suggesting that the web style will save you from heading up the Elevator to the Gallows.

File under: , , , , , , , , , , , , , , , , , , ,


Anonymous said...

Excellent trip Koranteng and I agree for the most part. Often times in conversation I find myself refering to the "low-REST" subset, not necessarily to pitch it or recommend it, but rather just to acknowledge its existence or to point out that REST, in some concrete form, has been proven to be highly adoptable. I wouldn't include high/low REST terminology if I was introducing the concept to someone but I find that it's useful keeping discussions going with people who aren't new to the topic. I don't like the idea that conversations are stopping because we lack the vocabulary to have them.

PS: I don't know who's telling you to "save it for the book" but it sure isn't me :) Take a page out of Paul Graham's book (no pun intended) and turn the blog into the book verbatim.

Anonymous said...

First, sorry, but that elevator pitch is totally unintelligible to me.

Second, is this whole current Hi vs. Lo crisis solely concerned with whether or not PUT is used (who DELETEs?)?

Finally, it's correct that the format of the payloads can vary. For GETs, you might use URIs, querystrings, YAML or XML. For responses, XML or JSON might be approriate. Whatever.

Koranteng said...

Responding to "Anonymous"'s queries.

1. I hope that some of the other pitches I've pointed out by the two Tims, Berners-Lee and Bray, are easier to understand. They certainly use less flowery language than me but it is just variations on a theme.

If you are more inclined to show rather than tell, you can do no worse than look at Joe Gregario's Restful Web column. They are all essential but this one gets at the heart of the matter. Paul Prescod, Mark Baker and Sam Ruby have great material also. Alternatively look at Roy Fielding's works.

If not, I would try the presentation I delivered last year in which I tried to distill the style. Particularly this slide where I tried to boil down everything into the essentials. I called this model, ArRESTed Development, (I need to find a better name for it). The salient features of this model of development in the web style are:

* Resource Modeling
* Every important resource should be identified
* Use the right verb
* Don't abuse HTTP
* Your users will hate you if you do
* Life is easier with the web
* Respect browser conventions
* Bookmark, Back button, Refresh
* Layer Stripping & Radical Simplification

That last bullet is simply a plea for reducing complexity in software

2. The Hi-Lo formulation means something different depending on whoever brandishes it which is why I believe it is a rathole and red herring. Ryan took it to mean something quite subtle but others are far less discriminating. The less said the better.

3. hypermedia of course can be in any format as you note.