On rich web applications, AlphaBlox and Oddpost

I had been meaning to comment about the past few posts over at Loosely Coupled but for some reason the comment entry fields don't show up in Mozilla or Firefox. The idea of having to launch Internet Explorer just to enter a comment didn't sit well with me, but I guess I will do it this once. This is part of a fairly deep vein of conversation in recent months on web applications, rich clients, the browser as a platform, or the location field as the new command line. A certain critical mass seems to have been reached, maybe it's the maturity of the browsers competing with Internet Explorer (especially Mozilla), the long history of security holes in MSIE or the publicity surrounding GMail and OddPost. I might as well join in on the commentary with a view from the trenches.

I work at Lotus software, IBM and was part of the tangled Halfbrain, Alphablox, IBM, Oddpost story at least last year when I worked on the "Simple Browser Productivity Components" - the spreadsheet, presentation and rich text editors that are now bundled with WebSphere Portal, tied to a document management system out of the box. For 6 months, my job was to 'port' these components to Mozilla; IBM for obvious reasons wants everything it ships to work on all platforms and browsers. The spreadsheet and presentation were licensed from AlphaBlox.

I wrote about some of my experiences at length last year in a discussion about "Applications vs. W3C DOM" and the adequacy of standards and browser support for rich web applications. Here's some further history and commentary:

The Halfbrain folks made a quite sensible decision that browser-based productivity applications would be a good thing, after all they, like everyone else, were starting to 'live' in the browser. More to the point, there's a market for good componentry and focused applications - lots of companies have developed spreadsheets, grids, charts, and various other editors and components. You may not make as much money upfront selling these components as shrink-wrapped software, like say a copy of Microsoft Office, but you can certainly tie it to services, other value-add features or even advertising and make money in other ways.

Sidenote: IBM has tried before to crack this market with Lotus eSuite (Java Office Suite for thin clients) which I also worked on after transfering from Lotus Freelance Graphics. Smaller companies have fared better because they understand that this is a niche market that one has to nurture and they didn't have the kind of insane revenue expectations that IBM had for that product (echoes of MS Office). 5 years on I hope/pray/(think?) that the company now understands what these kinds of products/technologies can achieve as part of its Portal and Workplace product portfolio and competitive strategy. One aspect of which is Linux. (cross my fingers since that's were the salary is coming from).

In any case, the Halfbrain folks didn't know any better, forged ahead, and miraculously a year later they managed to get a DHTML spreadsheet working. Everybody who sees it goes "Wow! How did you do that? In Javascript? With the then awful state of tooling for browser development?". It simply works; the UI looks, behaves and feels essentially like Excel, selecting cells, columns, drag and drop etc. Technically I can't say enough about that achievement (especially when I think back to the effort involved in the eSuite spreadsheets and presentation - there was still far more tooling available, and of couse, a rich programming model even in the less capable early 1.0 versions of Java than in the DHTML world). Keep in mind that it's a full spreadsheet engine, complete with calculation engine, dependency-checking, macro language. They were crazy enough to even at first target IE4. The programmers were are all new to Javascript development, but they were very disciplined and persevered. They lived on the bleeding edge of DHTML, had to invent things at times and workaround countless browser bugs. They talked somewhat with the MSIE team who made a lot of fixes for them in IE 5 and especially 5.5. As a very complex DHTML application, I'm sure it was a good test bed for Internet Explorer. In much the same way the folks at Opera last year jumped at a chance to test some of their DOM support with the speadsheet and presentation..

AlphaBlox, which does business analytics, sees this demo and buys HalfBrain. It makes perfect sense for them, everything is moving to the web. They integrate it with their packages, add conversion services from Excel, beef it up and offer it as part of their products.

Having learnt from the experience of getting the spreadsheet working, and realizing that what they have is essentially a very capable, general purpose DHTML library, one of the guys (now at OddPost/Yahoo) then heroically decides: why not do a Powerpoint clone? He goes ahead and writes a presentation editor !!!! This involves not just drag and drop, selection handling etc but most importantly text editing - ie. when you add your bulleted lists you need to edit the text. He managed to get this working even without using contentEditable support - it's a hack but users are non the wiser. It worked since IE 5.5 was up to scratch as a DHTML platform. This was an even greater bravura performance. Also, with greater facility with JavaScript, the code was smaller, more object-oriented. AlphaBlox also added some level of Powerpoint import capability.

The Javascript libraries for the spreadsheet and presentation amounted to around 60,000 lines of code, much of it shared and like any good DHTML toolkits had infrastructure APIs for menuing, windowing drag and drop, selection handling etc - the kinds of things that the DOM is missing since it's document and not screen-oriented. Sure, performance on very large spreadsheets may be a little poky and the recalc engine may need to be tuned, but those making really extensive use of spreadsheets will always stick to Lotus 123, Excel or more specialized packages. Similarly the drawing primitives of the presentation editor are not what you'd have in Illustrator but I think they are good enough. These kinds of editors target a different audience, they are 'worse' by definition. Of course 'worse is better', something that Christensen knows well.

Some of the HalfBrain folks left AlphaBlox with the idea to write yet another browser version of a productivity application - email and formed Oddpost - which Yahoo bought last week.

I don't really need to comment much about IBM's goals which are self-evident. Amy Wohl had a decent article from a while back about IBM's strategy Reinventing The Office: IBM Workplace. I'm sure the story and strategy has been tweaked and revised since. IBM, with a gap in its componentry, and looking to beef up its portal story helped with the development of the Midas component in Mozilla, the rich text editor, and then licensed the AlphaBlox spreadsheet and presentation. This is where I and others come in and that was last year's palava.

As far as I can tell, given the evolution of these products, the code got better, more object oriented, and more compact. Also the state of the browser has evolved, removing the need for most of the hacks of yesteryear. If I was starting the spreadsheet from scratch with the current capable browser platforms, it wouldn't be as much of a pipe-dream as back when HalfBrain started. (Joel per contra says 'never rewrite from scratch'). Given what I know about the effort it took on the Mozilla port, if Yahoo is so inclined, it should take a 6-8 months or so to have Oddpost working well in Mozilla, maybe less since there's obviously more staff. It would be perhaps a little longer in Safari or Opera since their DOM support (especially Level 2 Range) is weaker.

I talked to the Oddpost folks a few times last year; they were happy to hear that their first born babies were still being tended to - commiserating with me about some of the hacks they had had to implement earlier. With the advent of XMLHttp in MSIE, and IE 6 being a more robust platform for them, I think things were looking good for them as history has shown. Also since their app was their regular email client, they were eating their own dogfood and have every motivation to make it work.

I suspect that IBM could (and maybe should) have acquired them and perhaps there was the kind of usual flirtation/evaluation and general testing of the waters for building business relationships. With all the various mail clients and various technologies that a large company like IBM has, and the trendy push for Eclipse-based components, 'Javascript' components were a harder sell I would guess.

If the Oddpost code is anything like the AlphaBlox code I had to modify, they could probably jump start their porting effort considerably by using a lot the code I had to write or certainly the 10 or so common design patterns I encountered in doing this work. In a sense there is definitely a missed opportunity for IBM, we've missed out on an application compares favourably to many others. Of course, there would have been a difficulty in crafting a marketing message about how it fit in with the rest of the vast IBM software portfolio (read Domino, iNotes or any of the various mail apps we have). Again, I am glad that they've found a home at Yahoo, Oddpost is good technology and a good response to GMail. IBM acquiring Alphablox right on the heels of Yahoo taking up Oddpost is just another twist to this tale and likely just a coincidence.

Back to web applications. Mozilla is now the best platform for doing such development. It has the best standards support, is cross-platform and has the best devloper tools: DOM Inspector and Venkman, and perhaps even mindshare. In fact (market share be damned), it makes sense to first write yout rich web application for Mozilla. Your code will be cleaner to start off, will be better structured and will most likely work in other browsers - but that's my opinion, others say target MSIE first, just look at marketshare! - I would rather ignore that and get the compelling app.

As of Mozilla 1.5, XMLHTTPRequest support was well baked, and people have been using it successfully. I haven't tried Safari but I assume that it is not as good as Mozilla, though fast improving. Back then Opera was missing XMLHttpRequest, ContentEditable and DOM Level 2 Range support. We got some level of Opera support for free once the Mozilla work was done for the editors. The spreadsheet was functional and one could view presentations but not edit them because of the lack of range support and scriptable textareas.

The "rich web application" strategy is a very powerful appproach to development - and one we first used when I was when working on Lotus K-station (the post-mortem on that is for another day). It entails complete leverage of the browser which, after all, is the ubiquitous client. If the browser adds features you inherit them automatically. A short description of what I think is needed:

A client side framework for managing 'widgets'; a 'widget' is construed as a parameterized blob that produces markup (either in-line html or iframe-based). The data model is pushed to the client, the page is stitched together on the client, augmented by chrome and a code layer handles drag and drop, preview mode, incremental rendering and client side caching etc.

The idea is to fetch an HTML skeleton, decide what content you need, fetch that (as XML), and cache it wherever you get a chance. Render incrementally.

The pattern is simple:

Database <-> XML (Optional) <--> Javascript Object Bindings <--> UI Bindings (HTML) + UI management code

And:

It's the Latency, stupid.

When dealing with distributed applications, it's the issue of latency that will determine which applications will rule. Users ultimately want applications that are 1. fast to load 2. capable and 3. intuitive. They want all these at the same time. This is where making increased use of the DOM should shine compared to most simple html based UIs. Of course, you have to work hard when writing DOM apps to figure out where in your architecture you can do things incrementally and where to cache. But that's what engineering is all about and how one would get paid.

In K-station the 'widget' was a portlet, your portal pages was the drawing canvas that the framework managed, and you could navigate from page to page, incrementally building your pages in memory and switch back and forth instantaneously. With client-side caching, all you're doing is toggling the css display property. In the presentation editor, the analog of the widget was a drawing object (text, image, group etc). GMail and Oddpost follow the same pattern and it is the incremental rendering and caching that distinguishes them in their performance characteristics and makes them 'feel' like desktop apps.

The major missing infrastructure piece in rich web applications is going offline and synchronizing with good security. But that's a story for another day.

See also:

• On GMail and DHTML architecture again
• Deadwood and the Web Application Leap

File under: DHTML, IBM, software, technology, toli, web, javascript, design, architecture, history, oddpost, REST, DOM, programming, best practices, scripting