Friday, July 16, 2004

On rich web applications, AlphaBlox and Oddpost

I had been meaning to comment about the past few posts over at Loosely Coupled but for some reason the comment entry fields don't show up in Mozilla or Firefox. The idea of having to launch Internet Explorer just to enter a comment didn't sit well with me, but I guess I will do it this once. This is part of a fairly deep vein of conversation in recent months on web applications, rich clients, the browser as a platform, or the location field as the new command line. A certain critical mass seems to have been reached, maybe it's the maturity of the browsers competing with Internet Explorer (especially Mozilla), the long history of security holes in MSIE or the publicity surrounding GMail and OddPost. I might as well join in on the commentary with a view from the trenches.

I work at Lotus software, IBM and was part of the tangled Halfbrain, Alphablox, IBM, Oddpost story at least last year when I worked on the "Simple Browser Productivity Components" - the spreadsheet, presentation and rich text editors that are now bundled with WebSphere Portal, tied to a document management system out of the box. For 6 months, my job was to 'port' these components to Mozilla; IBM for obvious reasons wants everything it ships to work on all platforms and browsers. The spreadsheet and presentation were licensed from AlphaBlox.

I wrote about some of my experiences at length last year in a discussion about "Applications vs. W3C DOM" and the adequacy of standards and browser support for rich web applications. Here's some further history and commentary:

The Halfbrain folks made a quite sensible decision that browser-based productivity applications would be a good thing, after all they, like everyone else, were starting to 'live' in the browser. More to the point, there's a market for good componentry and focused applications - lots of companies have developed spreadsheets, grids, charts, and various other editors and components. You may not make as much money upfront selling these components as shrink-wrapped software, like say a copy of Microsoft Office, but you can certainly tie it to services, other value-add features or even advertising and make money in other ways.

Sidenote: IBM has tried before to crack this market with Lotus eSuite (Java Office Suite for thin clients) which I also worked on after transfering from Lotus Freelance Graphics. Smaller companies have fared better because they understand that this is a niche market that one has to nurture and they didn't have the kind of insane revenue expectations that IBM had for that product (echoes of MS Office). 5 years on I hope/pray/(think?) that the company now understands what these kinds of products/technologies can achieve as part of its Portal and Workplace product portfolio and competitive strategy. One aspect of which is Linux. (cross my fingers since that's were the salary is coming from).

In any case, the Halfbrain folks didn't know any better, forged ahead, and miraculously a year later they managed to get a DHTML spreadsheet working. Everybody who sees it goes "Wow! How did you do that? In Javascript? With the then awful state of tooling for browser development?". It simply works; the UI looks, behaves and feels essentially like Excel, selecting cells, columns, drag and drop etc. Technically I can't say enough about that achievement (especially when I think back to the effort involved in the eSuite spreadsheets and presentation - there was still far more tooling available, and of couse, a rich programming model even in the less capable early 1.0 versions of Java than in the DHTML world). Keep in mind that it's a full spreadsheet engine, complete with calculation engine, dependency-checking, macro language. They were crazy enough to even at first target IE4. The programmers were are all new to Javascript development, but they were very disciplined and persevered. They lived on the bleeding edge of DHTML, had to invent things at times and workaround countless browser bugs. They talked somewhat with the MSIE team who made a lot of fixes for them in IE 5 and especially 5.5. As a very complex DHTML application, I'm sure it was a good test bed for Internet Explorer. In much the same way the folks at Opera last year jumped at a chance to test some of their DOM support with the speadsheet and presentation..

AlphaBlox, which does business analytics, sees this demo and buys HalfBrain. It makes perfect sense for them, everything is moving to the web. They integrate it with their packages, add conversion services from Excel, beef it up and offer it as part of their products.

Having learnt from the experience of getting the spreadsheet working, and realizing that what they have is essentially a very capable, general purpose DHTML library, one of the guys (now at OddPost/Yahoo) then heroically decides: why not do a Powerpoint clone? He goes ahead and writes a presentation editor !!!! This involves not just drag and drop, selection handling etc but most importantly text editing - ie. when you add your bulleted lists you need to edit the text. He managed to get this working even without using contentEditable support - it's a hack but users are non the wiser. It worked since IE 5.5 was up to scratch as a DHTML platform. This was an even greater bravura performance. Also, with greater facility with JavaScript, the code was smaller, more object-oriented. AlphaBlox also added some level of Powerpoint import capability.

The Javascript libraries for the spreadsheet and presentation amounted to around 60,000 lines of code, much of it shared and like any good DHTML toolkits had infrastructure APIs for menuing, windowing drag and drop, selection handling etc - the kinds of things that the DOM is missing since it's document and not screen-oriented. Sure, performance on very large spreadsheets may be a little poky and the recalc engine may need to be tuned, but those making really extensive use of spreadsheets will always stick to Lotus 123, Excel or more specialized packages. Similarly the drawing primitives of the presentation editor are not what you'd have in Illustrator but I think they are good enough. These kinds of editors target a different audience, they are 'worse' by definition. Of course 'worse is better', something that Christensen knows well.

Some of the HalfBrain folks left AlphaBlox with the idea to write yet another browser version of a productivity application - email and formed Oddpost - which Yahoo bought last week.

I don't really need to comment much about IBM's goals which are self-evident. Amy Wohl had a decent article from a while back about IBM's strategy Reinventing The Office: IBM Workplace. I'm sure the story and strategy has been tweaked and revised since. IBM, with a gap in its componentry, and looking to beef up its portal story helped with the development of the Midas component in Mozilla, the rich text editor, and then licensed the AlphaBlox spreadsheet and presentation. This is where I and others come in and that was last year's palava.

As far as I can tell, given the evolution of these products, the code got better, more object oriented, and more compact. Also the state of the browser has evolved, removing the need for most of the hacks of yesteryear. If I was starting the spreadsheet from scratch with the current capable browser platforms, it wouldn't be as much of a pipe-dream as back when HalfBrain started. (Joel per contra says 'never rewrite from scratch'). Given what I know about the effort it took on the Mozilla port, if Yahoo is so inclined, it should take a 6-8 months or so to have Oddpost working well in Mozilla, maybe less since there's obviously more staff. It would be perhaps a little longer in Safari or Opera since their DOM support (especially Level 2 Range) is weaker.

I talked to the Oddpost folks a few times last year; they were happy to hear that their first born babies were still being tended to - commiserating with me about some of the hacks they had had to implement earlier. With the advent of XMLHttp in MSIE, and IE 6 being a more robust platform for them, I think things were looking good for them as history has shown. Also since their app was their regular email client, they were eating their own dogfood and have every motivation to make it work.

I suspect that IBM could (and maybe should) have acquired them and perhaps there was the kind of usual flirtation/evaluation and general testing of the waters for building business relationships. With all the various mail clients and various technologies that a large company like IBM has, and the trendy push for Eclipse-based components, 'Javascript' components were a harder sell I would guess.

If the Oddpost code is anything like the AlphaBlox code I had to modify, they could probably jump start their porting effort considerably by using a lot the code I had to write or certainly the 10 or so common design patterns I encountered in doing this work. In a sense there is definitely a missed opportunity for IBM, we've missed out on an application compares favourably to many others. Of course, there would have been a difficulty in crafting a marketing message about how it fit in with the rest of the vast IBM software portfolio (read Domino, iNotes or any of the various mail apps we have). Again, I am glad that they've found a home at Yahoo, Oddpost is good technology and a good response to GMail. IBM acquiring Alphablox right on the heels of Yahoo taking up Oddpost is just another twist to this tale and likely just a coincidence.

Back to web applications. Mozilla is now the best platform for doing such development. It has the best standards support, is cross-platform and has the best devloper tools: DOM Inspector and Venkman, and perhaps even mindshare. In fact (market share be damned), it makes sense to first write yout rich web application for Mozilla. Your code will be cleaner to start off, will be better structured and will most likely work in other browsers - but that's my opinion, others say target MSIE first, just look at marketshare! - I would rather ignore that and get the compelling app.

As of Mozilla 1.5, XMLHTTPRequest support was well baked, and people have been using it successfully. I haven't tried Safari but I assume that it is not as good as Mozilla, though fast improving. Back then Opera was missing XMLHttpRequest, ContentEditable and DOM Level 2 Range support. We got some level of Opera support for free once the Mozilla work was done for the editors. The spreadsheet was functional and one could view presentations but not edit them because of the lack of range support and scriptable textareas.

The "rich web application" strategy is a very powerful appproach to development - and one we first used when I was when working on Lotus K-station (the post-mortem on that is for another day). It entails complete leverage of the browser which, after all, is the ubiquitous client. If the browser adds features you inherit them automatically. A short description of what I think is needed:

A client side framework for managing 'widgets'; a 'widget' is construed as a parameterized blob that produces markup (either in-line html or iframe-based). The data model is pushed to the client, the page is stitched together on the client, augmented by chrome and a code layer handles drag and drop, preview mode, incremental rendering and client side caching etc.

The idea is to fetch an HTML skeleton, decide what content you need, fetch that (as XML), and cache it wherever you get a chance. Render incrementally.

The pattern is simple:

Database <-> XML (Optional) <--> Javascript Object Bindings <--> UI Bindings (HTML) + UI management code
And:

It's the Latency, stupid.

When dealing with distributed applications, it's the issue of latency that will determine which applications will rule. Users ultimately want applications that are 1. fast to load 2. capable and 3. intuitive. They want all these at the same time. This is where making increased use of the DOM should shine compared to most simple html based UIs. Of course, you have to work hard when writing DOM apps to figure out where in your architecture you can do things incrementally and where to cache. But that's what engineering is all about and how one would get paid.

In K-station the 'widget' was a portlet, your portal pages was the drawing canvas that the framework managed, and you could navigate from page to page, incrementally building your pages in memory and switch back and forth instantaneously. With client-side caching, all you're doing is toggling the css display property. In the presentation editor, the analog of the widget was a drawing object (text, image, group etc). GMail and Oddpost follow the same pattern and it is the incremental rendering and caching that distinguishes them in their performance characteristics and makes them 'feel' like desktop apps.

The major missing infrastructure piece in rich web applications is going offline and synchronizing with good security. But that's a story for another day.

See also:

File under: , , , , , , , , , , , , ,

11 comments:

Anonymous said...

well said.

Anonymous said...

The latency will tend toward zero and computing will become even more centralized. IOW that problem can be solved.

The problem with rich web apps is ultimately their variety of interface: they become increasingly arcane until the user requires training. In contrast, navigation in a well-designed web application is apparent to a knowledgeable browser user. So in the end it's time and money spent on training and learning that will keep rich web apps out of the marketplace and that will propel the use of simple web apps.

Anonymous said...

AFAICT, XMLHTTPRequest is subtly broken in newer IE installations. Perhaps it's due to security settings. I'm not sure. In any event it's not well maintained my MS and is likely to be further marginalized as they push XAML onstage.

Anonymous said...

Great post. What do think is the future for offline web apps. Eg an open version of iNotes - where content can be available offline?

Anthony
http://xminc.com/mt/

Ben Brown said...

You mention what you think is needed in a client-side DHTML app framework, and I totally agree. But have you (or anyone for that matter) know of anything similar that already exists? There are numerous DHTML 'toolkits' available, but with your experience, can you say that any of these are useful? Has anyone had the chance to poke around with bindows.net? how about netwindows.org?

Koranteng said...

Ahh... the state of DHTML toolkits: this is something I've been meaning to post about - will get round to it eventually... I've rolled my own toolkit 3 times, it's getting better each time but I haven't been able to convice anyone at IBM to invest the time to release it, document it opensource it or invest more in it - I tend to use parts of it on every project I've done of late or point folks to it when they come with DHTML web development questions.
I know of 3 other toolkits and widget sets within IBM some with very rich functionality: the problems seem to be 1. not well factored, these toolkits tend to drag in more than one needs for a given application 2. politics about the widget set (designers change their minds monthly about the look and feel - if your current demo deviates from the look dujour, god help you) 3. politics about the so-called "programming model" (IBM sells lots of development tools, has a large investment in java, J2EE - people can shut down your project by claiming that [your particular technology/toolkit etc] "doesn't follow the 'strategic' IBM software group programming model". This is often said with a veneer of confidence that barely disguises the handwaving but it's a highly effective way to kill projects 4. management hasn't been convinced that the toolkit is valuable in and of itself. sigh... If you manage to make it past this gamut of criticism you often have to tack on some 'programming model' to your toolkit (so force people to use Eclipse and those lovely tools that IBM and others sell). If you can afford it, try the Odyssey Browser Framework - ships with websphere portal.
If you're less well heeled, there seem to be a bunch of sourceforge projects with varying degrees of maturity.

The primary developer on netwindows.org has some of the deepest DHTML knowledge I've seen so that particular toolkit is very high quality... my only concern was that he seemed to assume that everyone was equally as smart as him... If you can overcome the initial learning curve, it is definitely worth it. If you're coming from

JSlib.mozdev.org probably has the most developer eyes on it but doesn't go out of its way to get MSIE support right.

In the interim, roll your own I say...

Anonymous said...

hello there

Paulo Eduardo Neves said...

Nice post. One of my greatest problems is to test DHTML applications. Till now, my best option is jsunit, but I'd like to know what other people use.

Anonymous said...

Nice article. I was just at Lotusphere 2005 and attended Thomas Gumz's session on the Domino Administration Web Application which uses a Javascript toolkit called DFC001 (or something like that) - DFC stands for Domino Foundation Classes. Have you seen this toolkit and if so how does it compare to some of the others you mentioned? (I thought it looked really nice).

Nakul said...

Thanx a ton for the article. Since long I have been thinking of dirtying hands with XML and javascript. your article actuated it to the level that i can start working.

also thanx for http://groups.yahoo.com/group/wdf-dom/message/4136%3fthreaded%3d1

Anonymous said...

Don't forget about the original WebOS.com guys. Shervin Pishevar, Drew Morris, Fredrik Malmer and Erik Arvidsson at WebOS were the pioneers of the WebOS space way back in 1999 and 2000. Unfortunately, they had idiotic investors who had no idea what they were sitting on. WebOS was the first AJAXian API complete with an app they called "Maggie" that created a true desktop-like application (it was their Hyperoffice product done just look outlook but through the browser only in DHTML/JS with the WebOS API). This was years before OddPost. They also had the first desktop evironment complete with windowing capabilities. Rumours are some of the old gang are planning a WebOS-redux and relaunching WebOS.com. Should be interesting to see what comes out.