The REST Elevator Pitch

• The Pitch
• The Hard Problems
• The Sequel
• The Elevator Soundtrack

The Pitch

I somewhat share Ryan Tomayko's concern for a shorter hand language for describing REST (Representational State Transfer), but I believe that the REST elevator pitch is pretty succinct already. It packs a lot of insight into the web architectural style of which dissertations have been written. The language used in technical debates is always important and rather than devolving into the fuzziness of "highs and lows", I'll suggest here that, as a matter of advocacy, one should stick to the elevator pitch.

To recap, the REST elevator pitch is:

• Identification Of Resources
• Manipulation Of Resources Through Representations
• Self-Descriptive Messages
• Hypermedia As The Engine Of Application State

In its most common application, the pitch will lead you to the HUHXtable Quartet (HTTP / URI / HTML / XML) if we handwave away the additional content types that the web style enables you to negotiate. It is quite clear that applications of all kinds have been built on these fundamentals for both human and machine interaction.

The thing is that each plank of the pitch brings its own benefits, benefits that perhaps can accrue independently. Still you get the most bang for the buck at the internet scale by the principled combination of all of them.

In my advocacy I tend to start with resource identification as the most important plank. The reason for this is that everybody can understand the value of identification. My grandmother understands the value of a bookmark to my photos. Heck we are even seeing the social benefits of ostensibly selfish bookmarking. The importance of resource identification cannot be understated and it leads us to the primacy of the URI.

The second plank, manipulation of resources through representations, is the bread and butter of HTTP, where we have an evolving set of standard formats of hypermedia exchanged using a small set of verbs.

Unpacking the HTTP proposition leads us to the four horsemen of the web: GET / POST / PUT / DELETE

Of course you can do very well if you pick up only 2 of those verbs, GET and POST. The good Reverend HTTP is a pretty tolerant fellow and won't begrudge those that don't believe in the rule of four, although in admittedly imperfect analogies, there's CRUD (Create / Retrieve / Update / Delete) in the database world and, as far as this layered internet goes there's TCP / IP / Datalink / Physical Layer (with say Ethernet or "Wi-Fi" 802.11b/a/g)

On the other hand, if you want to be a good citizen in the town of Deadwood and indeed if you want to survive in the wild west of the web, you shouldn't abuse HTTP. HTTP abuse leads to user confusion, headaches for systems designers and a tragedy of the commons. I consider HTTP a deliberately minimalist compromise, laissez-faire distributed computing if you will.

In social settings making your intentions known can help smooth interactions. We may start with small talk but we eventually manage to discern where others stand - well most of the time at least. We all operate with imperfect information but being straightforward is often a great policy. I'm sure there's some game theory or behavioural economics that can point to the value of transparency. Perhaps it is trite in light of Machiavelli and Sun-Tzu to believe that disclosure is the best policy but at least in markets transparency tends to be the font of aggregated wisdom. On the internet scale, this translates into the notion of giving visibility to intermediaries. Arguing again by analogy, the internet is an agreement, a network of networks. You can do whatever you want on your network but to interoperate with others, a small set of primitives, the TCP/IP standards in this case, have been agreed upon. I would suggest that the web style is similarly such an overlay system. REST is an architectural style which by principled design, is optimized to lower coordination costs.

Acknowledging the benefits of idempotency, of doing things safely, will lead one to making sure the use of GET is safe. Idempotency is not just about safety however; in clearly signaling your intentions, it is also about replayability. Thus idempotency is not just about the GET method. That is the wider argument for also using that oft-neglected duo, PUT and DELETE. Tunneling everything through POST (and the completely aberrant case GET) may be convenient but it adds unnecessary opacity and unpredictability about one's application. Unpredictability in the world leads to invasions and worse I might add. The use of those four core HTTP verbs helps intermediaries understand and participate more fully in the ecosystem.

The last two planks of the REST elevator pitch, self-descriptive messages, and especially that last one, hypermedia as the engine of application state, are attempts at addressing the issues of state, caching and evolving systems amongst other things.

Now of course in the preceding paragraphs, I've handwaved away entire dissertations that have been written about architectural styles and systems design, tomes written with language more precise and concepts more subtle than I've outlined. Good samples and running code are often also a determining factor in which style or application is adopted thus the view source imperative is also one of the reasons that web has caught on so quickly.

The Hard Problems

I've recently been thinking about defining the hardest problems I've encountered in software engineering, my cursory top 10 list:

• State
• Caching
• Latency
• Concurrency
• Search
• Metadata
• Persistence
• The Holy Grail Of Extensibility
• Structured data
• Character encoding

Now I'm not a database person so I handwaved away all of those data peoples' worries in one word: persistence. Of course I should probably add a few more issues and indeed you probably have your own notions. Indeed I settled just two days ago on those last two, structured data and character encoding, when I characterized them as being the gruesome twosome of computer science. Perhaps also there's the issue of discovery and bootstrapping of a system but that could be considered implementation details. There are those who mention security and reliability as other hard problems and they certainly are. Still I'll also handwave those away not as technological issues but as transaction costs; the economic benefits of ubiquity and leverage are key here. It seems that humans are quite prepared to tolerate gremlins and parasites in their systems (social, biological and technological) in moderation of course.

What is interesting is that almost all of these issues are important even if you aren't dealing with distributed computing. On the internet, which is all about distributed computing however, these are crucial problems. Now addressing these problems is a pretty tall order I must admit, but it seems to me that the web style has concrete answers to almost all of these constraints. Furthermore REST also offers a couple of those very valuable system characteristics: resilience and adaptability, almost as externalities.

Search for example seems to be addressed by valuing the importance of resource identification, the URI, that leads to PageRank and other algorithms. If you start with a viewpoint that URIs are cheap, and if you model your resources appropriately, you can deal with concurrency quite reasonably. State, caching and latency are all to do with performance which is always a bear. To that I'd argue that the commerce that is taking place daily on the internet seems to be a good indication that things are acceptable even if they could be improved. Metadata is the eternal headache of course but HTTP will give you headers and soul to help a little.

Discovery is addressed by URI as identification, the notion of hypermedia (either link headers or references in the media) and perhaps the current best practices of introspection resources. Persistence and structured data, I'll argue, are being addressed, warts and all, by the relational database, the seemingly-unstoppable spread of XML and, I believe, the current ascendancy of feeds. On this last subject, the wiring of the web, I am cautiously optimistic that the Atom store dream could be an answer; i.e. the combination of feed format (say Atom/RSS) and a wire protocol - which perhaps the Atom Publishing Protocol or similar could be. On the question of extensibility, last year I started a response that I felt was needed due to Dion Hincliffe's unbelievable assertion that "extensibility was the Achilles heel of REST". It always takes me a year to respond to minor footnotes but if there's one thing I'll do in the next few months, I'll be sure to write about REST and the Holy Grail of Extensibility.

But anyway, that is getting ahead of myself. I've been told to save my musings for "the book" or to stick to writing the running code that will "show, rather than tell" yet I keep falling for this web as conversational engine. Still there are undoubted social benefits of these exchanges and perhaps conversations like these are another virtue and externality of the web style. In any case, consider this my long winded response, Ryan: in matters of technology adoption and systems design, let's stick to the elevator pitch.

Further reading:

• REST - the web style
• A REST Intervention
• On Iframes, WSRP, REST and Portals
• The Unloved HTML Button and Other Folktales
• The Gospel of REST
• The Economics of REST

The Sequel

[Update - March 27, 2006] It seems as if everyone is getting into shaping the pitch and the language has been quite interesting.

• Danny Ayers remarked that we should go back to the source and points at Tim Berners-Lee's pitch from last year

  Web architecture 101

  • Things are denoted by URIs.
  • Use them to denote things.
  • Serve useful information at them.
  • Dereference them.

  That indeed is about as brief a pitch as some executives will be able to handle and it covers the essentials.
• Tim Bray's pitch is fairly pithy too, and for the last plank he uses

  You expect to ship a lot of URIs around in the bodies of requests and responses, and use them in operations that feel like link following.

  Now that's quite a mouthful in a presentation but it works fine in an elevator. I think "dereference them" is more terse and captures much of the same insight.
  
  As an engineer who is interested in precision in language, I tend to prefer Roy Fielding's formulation of "hypermedia as engine of application state". It's short and alludes to both the implementation and the area of focus of the web style. But your mileage may vary and I think it depends on the audience. We should be interested in a pitch that works because the advocacy struggle isn't over even if some are already thinking about the future issues. In this light, it was good to see Joe Gregorio again emphasizing the Show me the code route he embarked on last year. That View Source Imperative is the secret ingredient as far as adoption goes.

Note that everyone seems to conform to the Rule of Four as far as bullet points in the pitch go. Interestingly enough, Roy Fielding used to go with a five point pitch until he merged what were his previously separate definitions of "resource" and "URI" into the phrase "Identification Of Resources". I've proposed my own four point plan as a model for ArRESTed Development or The Low End Theory.

I concur of course with the endorsement of the formulation of REST - the web style and not just because I seem to have juice (of the google flavour) on that front having internalized that phrase long ago in my interventions, but because it gives us a tag, REST, combined with a catchphrase. As we know, every movie needs a tag line. Especially in Deadwood: "6 million ways to die, choose one". That is the B-movie of distributed computing. We want to move beyond grindhouse pulp fiction and get some of those glamourous, golden Hollywood trophies. And if that fails there's always the Gospel of REST.

The Elevator Soundtrack

• Outkast - Elevators (Me & You)
  The chorus of this laidback piece of Atlanta folklore could well be a stand-in for the inclusiveness of the web style and its architecture of participation.

  Me and You
  Your Momma and Your Cousin Too
  Rollin' Down The Strip On Vogues
  Comin' Up Slammin' Cadillac Doors

• [Update] A hat tip to the good doctor, Ernie Prabhakar, who points to a real life Huxtable Quartet. When I first started using HUHXtable, I was thinking of the Cosby Show and of comfy sweaters, thus it's great to hear that the Slide Huxtable Quartet is itself a tribute to that show and to the "Play It Again, Russell" episode in which Cliff's father, trombonist Russell "Slide" Huxtable (Earl Hyman) hosts a jam session. REST has even got a house band.
• Miles Davis - Ascenseur Pour L'Echafaud
  I've always loved Miles Davis's cool soundtrack to one of the great film noirs which saw a great print released last year. It's atmospheric and gets the job done - and with verve. I am always upset that ascenseur is translated as lift rather than elevator which is more heady to my ears. I'll conclude by suggesting that the web style will save you from heading up the Elevator to the Gallows.

File under: technology, rest, systems, software, design, architecture, advocacy, leverage, adoption, composition, tradeoffs, http, uri, xml, html, atom, web, language, strategy, toli

On Iframes, WSRP, REST and Portals

• On Prescience
• On Iframes, WSRP, REST and Portals

I'm still getting lots of good feedback to my presentation on REST - the web style (background) which I will summarize shortly. One of the more curious responses was to something that I didn't even dwell on in the presentation; indeed I think I skipped past it in my concern to get the big picture articulated. I suppose it was the proximity of yet another acronym, WSRP, to a couple of slides about complexity in the IBM technology stack, and a very public case study bemoaning the fact that Websphere Portal only added bookmarking after 3 years that raised a little concern. This was part of my subsequent response to one of those comments, slightly edited for clarity.

On Prescience

I fully agree, we should celebrate any and all who contributed to us internalizing the web style in WebSphere Portal. We should be very explicit about the pain we and our customers suffered and the evident gains that came once we learned our lesson.

The notion that it was still a struggle to get that simple feature (the ability to bookmark) included in the WebSphere Portal 5.1 release is actually a commentary on our previous attitude to the web style.

I would add to the list of those to be celebrated, those screaming Cassandras who were incredulous that we could ship a web product while ignoring url addressability and even tried to prototype some rudimentary addressability and sneak it in back when we had commit privileges to the core. If only we had had more nerve or had been able to better communicate the importance of this point... I know it normally takes 3 releases for a product to become useful, but sometimes magic can happen in the first release and with that comes thought leadership (Glory) and perhaps also some money (Gold).

On Iframes, WSRP, REST and Portals

Leading with the iframe invocation argument was always the weakest argument of those who I'll cast as the "K-stationers" in the early days of WebSphere Portal. After all, and like you note, even though iframes may provide

• easy integration of existing content
• multi-threaded download in the browser to lower the latency for the user
• caching closer the client (indeed in the client)
• offloading the portal by removing a layer of indirection in the portlet -> portal server -> client browser route
• introducting the possibility of moving aggregation to the client (leveraging Moore's law and bandwidth increases to the client)
• further recasting the "Portal Server" to a "Portlet server"(in other words there should be no difference between a "remote" and a local portlet) etc . That is the lesson of uniform interfaces in the web (e.g. URIs)

Now that's a pretty strong "weak argument" but we should acknowledge that Iframes also have downsides. Iframes mess with user expectations especially with respect to focus and some browser conventions are a little upset by them (e.g to bookmark, you need to right click on the frame in question). Browsers did solve the issues with the navigation history after a few years, but there was a little confusion in the interim with back button and refresh etc).

6 years on, I hope I have learnt to recognize the stronger aspects of an argument about systems design. What we didn't recognize or adequately evangelize was the value of the other aspect of an approach that more fully embraced the web, namely:

Basing the application semantics of the portal directly on URIs and the use of those 4 verbs that are the core of the web.

To explain, HTTP is an application-transfer protocol. It is something that works at the application layer. It has its native notion of how CRUD like operations are done (GET, POST, PUT, DELETE).

The REST proposition again:

• Identification Of Resources
• Manipulation Of Resources Through Representations
• Self-Descriptive Messages
• Hypermedia As The Engine Of Application State

I haven't kept abreast of WSRP in the past 5 years and thus my characterization of it was perhaps a little careless, but that's what you do when you make a presentation that borders on the provocative. In mitigation, I'll confess to being what I've termed a Layer Stripper.

Let me handwave a little... It strikes me that building an application protocol with CRUD like functionality that will typically (or even solely) be used over HTTP can lead to a little impedence if one is not careful.

To take my Technical Arteriosclerosis Terminator scalpel to WSRP, I would only ask:

• Do the CRUD like operations in WSRP map one to one with HTTP's native semantics? In other words, is the moral equivalent of a PUT in the WSRP world implemented as a PUT when the "transport" is HTTP?
• Is this true of at least those 3 verbs at the core of the Web (GET? POST? DELETE?). (I'll assume that WSRP adds perhaps more verbs in its applications semantics).
• Are the security primitives going to be leveraging HTTP's native security features or are we rolling our own?
• Are there any other "transports" other than HTTP? Is there a CORBA mapping for WSRP?

RFC 3205 On the use of HTTP as a Substrate should be required reading here.

I'll suggest that if we were starting from scratch we might do the following

• Start with Resource Modeling and ask: what are the core resources that need to be identified in a remotely invoked portlet?
• What representations need to be returned? For this we'd probably pick some standard Portal XML vocabulary or, for browser rendering, an XHTML format, for pervasive rendering some other Markup Language
• What are the operations that need be performed on said resources?
• What HTTP verbs are appropriate for each operation when we manipulate those resources?

Joe Gregario's Show Me The Code column has been going through this exercise with the example of a bookmarking service - well worth reading. A remote portlet may have more complex semantics but one should go through this to clarify our design and architecture. I assume that WSRP implementors went through such a process as that spec was standardized.

The result would be something like simple URI commands returning representations and manipulating them using well-worn HTTP semantics. I very much hope that is what WSRP has evolved into. We'd get:

1. Visibility to HTTP intermediaries, first the caching proxies in the web sense. (like the page caching in the portal)
2. Since we'd be using URIs, someone may think up new uses for our remote portlets and be able to easily embed them in their application through simple hyperlink and the use of the appropriate verbs etc. Third party Glue Layer people could "remix" our portlets through pipelining and filtering or whatever else those people do (I'll note that IBM recently bought Gluecode)

With this approach there is no API per se, the interface contract is explicitly manifested in the exchanged hypermedia and associated operations on the resources.

"What? No API?" "This RESTful stuff has no substance". "Where is the value add?"

That indeed is the paradox of the web style. The "API" is typically HTTP/URI/HTML/XML.

The fourth plank of the REST proposition is a difficult thing to internalize, I certainly couldn't express it adequately 6 years ago.

"Hypermedia as the engine of application state"

Ponder that for a few weeks.

[Update] James Snell adds: "HTTP is the programming model".

Some have suggested that one would additionally require some service description language, perhaps WSDL, for such a service. On that, the jury is still out. In the example I highlighted in the presentation, Google Maps didn't publish any WSDL. Craigslist didn't publish any WSDL. They simply had a clean URI api with well-defined parameters that were obvious to any developer with a week to spare and idle curiousity satisfied by the View Source impulse. It was obvious what verbs to apply: POST or GET, PUT (they don't delete their maps, but if they did, that is how one would proceed). Third party intermediaries simply figured out that url hierarchy and the associated verbs, the identified resources and what formats the representations were retuned in. Then they manufactured serendipity. Now there is no binding contract and things can get broken at any time but it wouldn't be in Google's interest in their competition with Mapquest or Terraserver to break people that compose applications on their platform and brings eyeballs and mindshare to their company.

This then is my argument for changing the frame of the debate:

There's a complexity and layering argument that naturally falls out of the REST viewpoint which should guide the applications that one builds.

The web is about identifying important resources and exchanging representional state. There are certain constraints that are made to enable global scope and evolution. Being on the web, is being an active participant in this scheme of things.

Identifiers lead almost to having the location field as the command line.

Resource modeling as your starting point sets you in direct consonance with this notion and exposes you to ease of composition.

e.g. Jon Udell's Library Lookup project

An information architecture that starts with hypemedia lends itself to the construction of simple interaction designs for both humans and machines, and there are huge amounts of tools readily available for this.

The question is then one of leverage; leverage in economic terms is all about making externalities work for you. The endpoint of this approach to system design is what I call a virtuous cycle of managed serendipity.

One response to this piece was that actually WSRP stood up quite well to those questions I had hand-waved in its direction, apparently it was not too complex, it addressed wider issues and use-cases etc. I think that is great and hopefully the leaks have been plugged in that spec so that it can be mapped reasonably to HTTP; I guess the argument is that it is not an unnecessary layer. I'm still unclear if WSRP is ever used with a "non-HTTP transport" or security, I'll dig into that at some point. In any case, the point of my talk was the same scrutiny should be brought to bear on all of IBM's technology stack and unnnecessay layers shed and/or leaky abstractions plugged to make them work better with the web style.

A later follow-up, privately-addressed, asked:

"Isn't a "portlet" fundamentally a presentation-level object? Why is the concept of "portlet remoting" not considered an oxymoron? Shouldn't we be talking instead about the simple expression of web services as data (with, perhaps, media-type detection as an enabler of polymorphism"

There was lots more and quite fulminant at that, I'd have to say that's all true and expressed more concisely than I did. It's good to know that there are like-minded people around. I didn't mention content-negotiation in my response but that is another aspect built into the web style that can be leveraged but that is constantly being reinvented at higher levels and layers. There's lots more to say but that's the lay of this technology world... In evangelism the first step is to get people sensitized, after that it's best to proceed one argument at a time.

---

P.S. I've been told there's been too much technology toli of late so I'll try to push some literary, musical or African toli to the head of the queue, it can be more fun and oftentimes more rewarding.

File under: technology, web, REST, systems, design, architecture, tradeoffs, adoption, leverage, composition, iframe, wsrp, WebSphere, ibm, portal, portlets, aggregation, software, complexity, uri, http, xml, html, xhtml, lotus, IBM, toli