On Iframes, WSRP, REST and Portals
I'm still getting lots of good feedback to my presentation on REST - the web style (background) which I will summarize shortly. One of the more curious responses was to something that I didn't even dwell on in the presentation; indeed I think I skipped past it in my concern to get the big picture articulated. I suppose it was the proximity of yet another acronym, WSRP, to a couple of slides about complexity in the IBM technology stack, and a very public case study bemoaning the fact that Websphere Portal only added bookmarking after 3 years that raised a little concern. This was part of my subsequent response to one of those comments, slightly edited for clarity.
I fully agree, we should celebrate any and all who contributed to us internalizing the web style in WebSphere Portal. We should be very explicit about the pain we and our customers suffered and the evident gains that came once we learned our lesson.
The notion that it was still a struggle to get that simple feature (the ability to bookmark) included in the WebSphere Portal 5.1 release is actually a commentary on our previous attitude to the web style.
I would add to the list of those to be celebrated, those screaming Cassandras who were incredulous that we could ship a web product while ignoring url addressability and even tried to prototype some rudimentary addressability and sneak it in back when we had commit privileges to the core. If only we had had more nerve or had been able to better communicate the importance of this point... I know it normally takes 3 releases for a product to become useful, but sometimes magic can happen in the first release and with that comes thought leadership (Glory) and perhaps also some money (Gold).
Leading with the iframe invocation argument was always the weakest argument of those who I'll cast as the "K-stationers" in the early days of WebSphere Portal. After all, and like you note, even though iframes may provide
Now that's a pretty strong "weak argument" but we should acknowledge that Iframes also have downsides. Iframes mess with user expectations especially with respect to focus and some browser conventions are a little upset by them (e.g to bookmark, you need to right click on the frame in question). Browsers did solve the issues with the navigation history after a few years, but there was a little confusion in the interim with back button and refresh etc).
6 years on, I hope I have learnt to recognize the stronger aspects of an argument about systems design. What we didn't recognize or adequately evangelize was the value of the other aspect of an approach that more fully embraced the web, namely:
To explain, HTTP is an application-transfer protocol. It is something that works at the application layer. It has its native notion of how CRUD like operations are done (GET, POST, PUT, DELETE).
The REST proposition again:
I haven't kept abreast of WSRP in the past 5 years and thus my characterization of it was perhaps a little careless, but that's what you do when you make a presentation that borders on the provocative. In mitigation, I'll confess to being what I've termed a Layer Stripper.
Let me handwave a little... It strikes me that building an application protocol with CRUD like functionality that will typically (or even solely) be used over HTTP can lead to a little impedence if one is not careful.
To take my Technical Arteriosclerosis Terminator scalpel to WSRP, I would only ask:
RFC 3205 On the use of HTTP as a Substrate should be required reading here.
I'll suggest that if we were starting from scratch we might do the following
Joe Gregario's Show Me The Code column has been going through this exercise with the example of a bookmarking service - well worth reading. A remote portlet may have more complex semantics but one should go through this to clarify our design and architecture. I assume that WSRP implementors went through such a process as that spec was standardized.
The result would be something like simple URI commands returning representations and manipulating them using well-worn HTTP semantics. I very much hope that is what WSRP has evolved into. We'd get:
With this approach there is no API per se, the interface contract is explicitly manifested in the exchanged hypermedia and associated operations on the resources.
That indeed is the paradox of the web style. The "API" is typically HTTP/URI/HTML/XML.
The fourth plank of the REST proposition is a difficult thing to internalize, I certainly couldn't express it adequately 6 years ago.
Ponder that for a few weeks.
[Update] James Snell adds: "HTTP is the programming model".
Some have suggested that one would additionally require some service description language, perhaps WSDL, for such a service. On that, the jury is still out. In the example I highlighted in the presentation, Google Maps didn't publish any WSDL. Craigslist didn't publish any WSDL. They simply had a clean URI api with well-defined parameters that were obvious to any developer with a week to spare and idle curiousity satisfied by the View Source impulse. It was obvious what verbs to apply: POST or GET, PUT (they don't delete their maps, but if they did, that is how one would proceed). Third party intermediaries simply figured out that url hierarchy and the associated verbs, the identified resources and what formats the representations were retuned in. Then they manufactured serendipity. Now there is no binding contract and things can get broken at any time but it wouldn't be in Google's interest in their competition with Mapquest or Terraserver to break people that compose applications on their platform and brings eyeballs and mindshare to their company.
This then is my argument for changing the frame of the debate:
One response to this piece was that actually WSRP stood up quite well to those questions I had hand-waved in its direction, apparently it was not too complex, it addressed wider issues and use-cases etc. I think that is great and hopefully the leaks have been plugged in that spec so that it can be mapped reasonably to HTTP; I guess the argument is that it is not an unnecessary layer. I'm still unclear if WSRP is ever used with a "non-HTTP transport" or security, I'll dig into that at some point. In any case, the point of my talk was the same scrutiny should be brought to bear on all of IBM's technology stack and unnnecessay layers shed and/or leaky abstractions plugged to make them work better with the web style.
A later follow-up, privately-addressed, asked:
P.S. I've been told there's been too much technology toli of late so I'll try to push some literary, musical or African toli to the head of the queue, it can be more fun and oftentimes more rewarding.
File under: technology, web, REST, systems, design, architecture, tradeoffs, adoption, leverage, composition, iframe, wsrp, WebSphere, ibm, portal, portlets, aggregation, software, complexity, uri, http, xml, html, xhtml, lotus, IBM, toli
On Prescience
I fully agree, we should celebrate any and all who contributed to us internalizing the web style in WebSphere Portal. We should be very explicit about the pain we and our customers suffered and the evident gains that came once we learned our lesson.
The notion that it was still a struggle to get that simple feature (the ability to bookmark) included in the WebSphere Portal 5.1 release is actually a commentary on our previous attitude to the web style.
I would add to the list of those to be celebrated, those screaming Cassandras who were incredulous that we could ship a web product while ignoring url addressability and even tried to prototype some rudimentary addressability and sneak it in back when we had commit privileges to the core. If only we had had more nerve or had been able to better communicate the importance of this point... I know it normally takes 3 releases for a product to become useful, but sometimes magic can happen in the first release and with that comes thought leadership (Glory) and perhaps also some money (Gold).
On Iframes, WSRP, REST and Portals
Leading with the iframe invocation argument was always the weakest argument of those who I'll cast as the "K-stationers" in the early days of WebSphere Portal. After all, and like you note, even though iframes may provide
- easy integration of existing content
- multi-threaded download in the browser to lower the latency for the user
- caching closer the client (indeed in the client)
- offloading the portal by removing a layer of indirection in the portlet -> portal server -> client browser route
- introducting the possibility of moving aggregation to the client (leveraging Moore's law and bandwidth increases to the client)
- further recasting the "Portal Server" to a "Portlet server"(in other words there should be no difference between a "remote" and a local portlet) etc . That is the lesson of uniform interfaces in the web (e.g. URIs)
Now that's a pretty strong "weak argument" but we should acknowledge that Iframes also have downsides. Iframes mess with user expectations especially with respect to focus and some browser conventions are a little upset by them (e.g to bookmark, you need to right click on the frame in question). Browsers did solve the issues with the navigation history after a few years, but there was a little confusion in the interim with back button and refresh etc).
6 years on, I hope I have learnt to recognize the stronger aspects of an argument about systems design. What we didn't recognize or adequately evangelize was the value of the other aspect of an approach that more fully embraced the web, namely:
Basing the application semantics of the portal directly on URIs and the use of those 4 verbs that are the core of the web.
To explain, HTTP is an application-transfer protocol. It is something that works at the application layer. It has its native notion of how CRUD like operations are done (GET, POST, PUT, DELETE).
The REST proposition again:
- Identification Of Resources
- Manipulation Of Resources Through Representations
- Self-Descriptive Messages
- Hypermedia As The Engine Of Application State
I haven't kept abreast of WSRP in the past 5 years and thus my characterization of it was perhaps a little careless, but that's what you do when you make a presentation that borders on the provocative. In mitigation, I'll confess to being what I've termed a Layer Stripper.
Let me handwave a little... It strikes me that building an application protocol with CRUD like functionality that will typically (or even solely) be used over HTTP can lead to a little impedence if one is not careful.
To take my Technical Arteriosclerosis Terminator scalpel to WSRP, I would only ask:
- Do the CRUD like operations in WSRP map one to one with HTTP's native semantics? In other words, is the moral equivalent of a PUT in the WSRP world implemented as a PUT when the "transport" is HTTP?
- Is this true of at least those 3 verbs at the core of the Web (GET? POST? DELETE?). (I'll assume that WSRP adds perhaps more verbs in its applications semantics).
- Are the security primitives going to be leveraging HTTP's native security features or are we rolling our own?
- Are there any other "transports" other than HTTP? Is there a CORBA mapping for WSRP?
RFC 3205 On the use of HTTP as a Substrate should be required reading here.
I'll suggest that if we were starting from scratch we might do the following
- Start with Resource Modeling and ask: what are the core resources that need to be identified in a remotely invoked portlet?
- What representations need to be returned? For this we'd probably pick some standard Portal XML vocabulary or, for browser rendering, an XHTML format, for pervasive rendering some other Markup Language
- What are the operations that need be performed on said resources?
- What HTTP verbs are appropriate for each operation when we manipulate those resources?
Joe Gregario's Show Me The Code column has been going through this exercise with the example of a bookmarking service - well worth reading. A remote portlet may have more complex semantics but one should go through this to clarify our design and architecture. I assume that WSRP implementors went through such a process as that spec was standardized.
The result would be something like simple URI commands returning representations and manipulating them using well-worn HTTP semantics. I very much hope that is what WSRP has evolved into. We'd get:
- Visibility to HTTP intermediaries, first the caching proxies in the web sense. (like the page caching in the portal)
- Since we'd be using URIs, someone may think up new uses for our remote portlets and be able to easily embed them in their application through simple hyperlink and the use of the appropriate verbs etc. Third party Glue Layer people could "remix" our portlets through pipelining and filtering or whatever else those people do (I'll note that IBM recently bought Gluecode)
With this approach there is no API per se, the interface contract is explicitly manifested in the exchanged hypermedia and associated operations on the resources.
"What? No API?" "This RESTful stuff has no substance". "Where is the value add?"
That indeed is the paradox of the web style. The "API" is typically HTTP/URI/HTML/XML.
The fourth plank of the REST proposition is a difficult thing to internalize, I certainly couldn't express it adequately 6 years ago.
"Hypermedia as the engine of application state"
Ponder that for a few weeks.
[Update] James Snell adds: "HTTP is the programming model".
Some have suggested that one would additionally require some service description language, perhaps WSDL, for such a service. On that, the jury is still out. In the example I highlighted in the presentation, Google Maps didn't publish any WSDL. Craigslist didn't publish any WSDL. They simply had a clean URI api with well-defined parameters that were obvious to any developer with a week to spare and idle curiousity satisfied by the View Source impulse. It was obvious what verbs to apply: POST or GET, PUT (they don't delete their maps, but if they did, that is how one would proceed). Third party intermediaries simply figured out that url hierarchy and the associated verbs, the identified resources and what formats the representations were retuned in. Then they manufactured serendipity. Now there is no binding contract and things can get broken at any time but it wouldn't be in Google's interest in their competition with Mapquest or Terraserver to break people that compose applications on their platform and brings eyeballs and mindshare to their company.
This then is my argument for changing the frame of the debate:
There's a complexity and layering argument that naturally falls out of the REST viewpoint which should guide the applications that one builds.
The web is about identifying important resources and exchanging representional state. There are certain constraints that are made to enable global scope and evolution. Being on the web, is being an active participant in this scheme of things.
Identifiers lead almost to having the location field as the command line.
Resource modeling as your starting point sets you in direct consonance with this notion and exposes you to ease of composition.
e.g. Jon Udell's Library Lookup project
An information architecture that starts with hypemedia lends itself to the construction of simple interaction designs for both humans and machines, and there are huge amounts of tools readily available for this.
The question is then one of leverage; leverage in economic terms is all about making externalities work for you. The endpoint of this approach to system design is what I call a virtuous cycle of managed serendipity.
One response to this piece was that actually WSRP stood up quite well to those questions I had hand-waved in its direction, apparently it was not too complex, it addressed wider issues and use-cases etc. I think that is great and hopefully the leaks have been plugged in that spec so that it can be mapped reasonably to HTTP; I guess the argument is that it is not an unnecessary layer. I'm still unclear if WSRP is ever used with a "non-HTTP transport" or security, I'll dig into that at some point. In any case, the point of my talk was the same scrutiny should be brought to bear on all of IBM's technology stack and unnnecessay layers shed and/or leaky abstractions plugged to make them work better with the web style.
A later follow-up, privately-addressed, asked:
"Isn't a "portlet" fundamentally a presentation-level object? Why is the concept of "portlet remoting" not considered an oxymoron? Shouldn't we be talking instead about the simple expression of web services as data (with, perhaps, media-type detection as an enabler of polymorphism"There was lots more and quite fulminant at that, I'd have to say that's all true and expressed more concisely than I did. It's good to know that there are like-minded people around. I didn't mention content-negotiation in my response but that is another aspect built into the web style that can be leveraged but that is constantly being reinvented at higher levels and layers. There's lots more to say but that's the lay of this technology world... In evangelism the first step is to get people sensitized, after that it's best to proceed one argument at a time.
P.S. I've been told there's been too much technology toli of late so I'll try to push some literary, musical or African toli to the head of the queue, it can be more fun and oftentimes more rewarding.
File under: technology, web, REST, systems, design, architecture, tradeoffs, adoption, leverage, composition, iframe, wsrp, WebSphere, ibm, portal, portlets, aggregation, software, complexity, uri, http, xml, html, xhtml, lotus, IBM, toli
