On The Limitations Of Notes On The Web
In the week after Lotus announces lots of innovation in the forthcoming version of Lotus Notes, with lots of great screenshots and undoubted enhancements, it may seem weird to post a short article I had written a few weeks ago on the past limitations of said product in my internal IBM blog. I do this because I've been taking a historian of science's approach to my writing on technology and its effects. I am always interested in how organizations work and how they make decisions. And a certain amount of dirty laundry airing is healthy if only to pat ourselves on the back to show that now we have learnt our lessons and that there is a lot of introspection and thought in the company. Also my focus is not on the Notes client which has always sold itself, but on rather the web interfaces that product has had since the advent of Domino, its first embrace of the web. As you may know I am an unabashed believer in the web as the ubiquitous client. This piece should give some context to these three slides from my recent REST presentation. They weren't quite a twisting of the dagger but more a challenge to the development organization. I believe there's Joy in Repetition and with a hip-hop sensibility remix myself constantly. So here's the original version before the sample.
Joe Turic and others are pondering Lotus Notes vrs Gmail and coming down in favour of the latter.
The complaints are about "Why is the email program that we use internally so limited?" and some of it has to do with storage quotas and the fact that we have very low ones (even as the platform handles 49 GB mailboxes just fine as Ed Brill keeps pointing out when comparing it to Microsoft Outlook). Most of the reason for this hence is not technical but rather I suspect a hint about the direction of email retention policy.
Later David Crumpton asks "Does Notes offer a Webmail interface?"
And of course it does, and it even has two, the regular webmail interface and iNotes, the prettier dhtml version which should give a snappier performance with lower latency for the user. A third way of getting your notes mail in a browser is through a portlet in WebSphere Portal.
The question is, why did these web interfaces not get widely adopted?
Let's start with the question of investment.
Plainly put, not everyone at IBM has bought into the web, html and http.
I would count Lotus/IBM's efforts over the past 7 years as half-hearted although now we are supposedly hardcore about the web, Linux and open source in general. The thing to take away is that, at the point that judicious investment could have been made, and leadership in thought, if not in software products, was possible, we blinked.
IBM does middleware and services and moves slowly due to a sense of innate conservatism. I think this is fair, there are parts of IBM that are bleeding edge and one has to get buy-in before the rest of the elephant moves. As someone who was advocating the web almost from the beginning, it has been a long road to convince others to buy-in. The fact that we are now moving (and quickly) is great but I think it is cause for rueing missed opportunities.
Jim Gray's Distributed Computing Economics (pdf) makes a devastating case for the relative innefficiency of many in the industry that haven't embraced the web style.
Up until 2000, I believe the operational staff at Yahoo was 8 and included the founder Jerry Yang.
Google has embraced redundancy and written software to work around human fallibility and hardware reliability. Google File System is not too shabby, if may say so. We have efforts in the same arena whether it is the Grid and autonomic computing work and say the move to VOIP but are we eating that dogfood?
Are we prepared to change our corporate structure and have profits come more from well-designed and usable software as opposed to consultantware? Are we prepared to eat our babies and make products that have much lower operational costs?
Tim Bray had this to say $46,213,000,000.00
Now that Godfather Bray is at Sun, perhaps there's a little pleasure in tweaking us. But the question still remains. Have we bought into Radical Simplification?
It takes time for the effects of technology to be felt and for people to get comfortable with the web. We have now had 10 years of Moore's Law in the Datacenter and so we have examples and success stories to pick from. It was a more uncertain world 5-10 years ago and perhaps as a corporation we were hedging our bets. When IBM moves it's a sign that something is ready to take off.
One answer is that usability was a problem in webmail interfaces that are used daily. In the browser world 5-7 years ago, you had to handle that beast called Netscape 4.7 which meant that it was a case of lowest common denominator in the user interface and frequent browser round trips (read poor user interaction experience due to latency). It was controversial when iNotes was developed and proclaimed that it only supported MS Internet Explorer 5.0 and above only. The product manager who managed to get this dispensation approved deserves to be elevated to the highest levels. Many other managers didn't even bother fighting that battle. I'll hazard that even though IBM could have written something like GMail 7 years ago (and I certainly had a framework from K-station that would have made it possible with some work) it would have been a tough sell internally.
I've written about what was possible back then On Rich Web applications, AlphaBlox and Oddpost and On Gmail and DHTML architecture again
Joel Spolksy says that "it is easier to write code than to read code". This is what the Not Invented Here attitude betrays. We didn't have anything like community source where others might discover and find out about components that have solved similar problems.
(sidenote: when one asks about spending time to document one's component, one is often told that it is not mature enough, that we'd have to spend time supporting new users, we can't afford to slow down to support the one current users etc. There's a tendancy to keep everything close to our chests. Even if others will be able to help and flesh out flaws in our designs, it is thought that exposure to inchoate thinking will be a fatal flaw. This is a mindset that has to be overcome and on this, I am not so sure that it will happen, certainly I appear to me a lone voice in my parts of the wood)
See also Why Specs matter and the Law of Leaky Abstractions
Back to the subject of web interfaces for Notes... No one really invested in making great browser voodoo happen even though we had lots of great frameworks internally. Indeed I still see people grapple with problems that I have fixed before and keep digging up my old code to point them to. Ideally I would have been able to start an incubator at some site and others would have discovered it. Instead everyone is re-writing the same code all over. We haven't codified best practices or if we have, we haven't publicised them sufficiently.
Braindead "corporate standards" are another aspect of the problem. Back in 1999 when iNotes was just about usable and we were developing the Notes portlet for Lotus K-station (and subsequently WebSphere Portal), we were told that
We certainly paid a heavy price for this risk aversion.
Now of course we worked around all of these obstacles and made the best of things, lots of people toiled to get performance fairly reasonable (the CORBA api is now (slightly) more coarse-grained, we developed Collaboration Services for Portal so that others could reuse our code and hard-learnt lessons - now the most downloaded aspect of WebSphere Portal etc), but our efforts were hobbled in many ways. We certainly could have had more things going our way.
Summing up, first impressions matter, just ask Malcolm Gladwell (Blink)...
I f the pilot deployments of the various webmail interfaces internally had been reasonable (instead of awful experiences for the executives and other audiences who tried them), we would have been able to get more buy-in, more investment and with a bit of luck might have made comparable interfaces to the snappy Gmail that not only us but also our users and customers would have loved.
It is true that many things had to go right for this vision to have transpired but let's admit that we shot ourselves in the proverbial foot and also acknowledge that it was a failure of imagination and ultimately a failure of technical leadership.
Looking at today's opportunities and the various directions in which Lotus/IBM can go in, I only hope that we don't Get on the wrong Bus this time.
File under: technology, software, web, Lotus, notes, IBM, history, strategy, organizations, adoption, decisions, iNotes, K-station, WebSphere, portal, history, toli
On The Limitations Of Notes/Domino On The Web
Joe Turic and others are pondering Lotus Notes vrs Gmail and coming down in favour of the latter.
The complaints are about "Why is the email program that we use internally so limited?" and some of it has to do with storage quotas and the fact that we have very low ones (even as the platform handles 49 GB mailboxes just fine as Ed Brill keeps pointing out when comparing it to Microsoft Outlook). Most of the reason for this hence is not technical but rather I suspect a hint about the direction of email retention policy.
Later David Crumpton asks "Does Notes offer a Webmail interface?"
And of course it does, and it even has two, the regular webmail interface and iNotes, the prettier dhtml version which should give a snappier performance with lower latency for the user. A third way of getting your notes mail in a browser is through a portlet in WebSphere Portal.
The question is, why did these web interfaces not get widely adopted?
Let's start with the question of investment.
1. Investment and Adoption
Plainly put, not everyone at IBM has bought into the web, html and http.
I would count Lotus/IBM's efforts over the past 7 years as half-hearted although now we are supposedly hardcore about the web, Linux and open source in general. The thing to take away is that, at the point that judicious investment could have been made, and leadership in thought, if not in software products, was possible, we blinked.
IBM does middleware and services and moves slowly due to a sense of innate conservatism. I think this is fair, there are parts of IBM that are bleeding edge and one has to get buy-in before the rest of the elephant moves. As someone who was advocating the web almost from the beginning, it has been a long road to convince others to buy-in. The fact that we are now moving (and quickly) is great but I think it is cause for rueing missed opportunities.
2. Innefficient IT structure at IBM
Jim Gray's Distributed Computing Economics (pdf) makes a devastating case for the relative innefficiency of many in the industry that haven't embraced the web style.
"Megaservices like Yahoo!, Google, and Hotmail have relatively low operations staff costs. These megaservices have discovered ways to deliver content for less that the milli-dollar that advertising will fund. For example, in 2002 Google had an operations staff of 25 who managed its two petabyte (2^15 bytes) database and 10,000 servers spread across several sites.
Hotmail and Yahoo! cite similar numbers - small staffs manage ~300 TB of storage and more than ten thousand servers."
Up until 2000, I believe the operational staff at Yahoo was 8 and included the founder Jerry Yang.
Google has embraced redundancy and written software to work around human fallibility and hardware reliability. Google File System is not too shabby, if may say so. We have efforts in the same arena whether it is the Grid and autonomic computing work and say the move to VOIP but are we eating that dogfood?
Are we prepared to change our corporate structure and have profits come more from well-designed and usable software as opposed to consultantware? Are we prepared to eat our babies and make products that have much lower operational costs?
Tim Bray had this to say $46,213,000,000.00
I looked up the answer to the question: What is IBM's consulting revenue? In 2004, IBM's gross revenue was $96B, of which $46B was Global Services, i.e. consulting. I see that basically as testimony to how our profession, the IT profession, has failed our customers. Nothing against IBM; in fact, as solution-providers go, my experience is that IBM GS is pretty good. But if you see IBM as a microcosm of the industry, it shouldn't cost $46B in consulting to deploy $50B worth of technology. It's not going to be easy to get there, and it's going to take a long time, but we just have to focus on making things simpler.
Now that Godfather Bray is at Sun, perhaps there's a little pleasure in tweaking us. But the question still remains. Have we bought into Radical Simplification?
3. Technology Adoption Takes Time
It takes time for the effects of technology to be felt and for people to get comfortable with the web. We have now had 10 years of Moore's Law in the Datacenter and so we have examples and success stories to pick from. It was a more uncertain world 5-10 years ago and perhaps as a corporation we were hedging our bets. When IBM moves it's a sign that something is ready to take off.
4. Fragility of the browser platform
One answer is that usability was a problem in webmail interfaces that are used daily. In the browser world 5-7 years ago, you had to handle that beast called Netscape 4.7 which meant that it was a case of lowest common denominator in the user interface and frequent browser round trips (read poor user interaction experience due to latency). It was controversial when iNotes was developed and proclaimed that it only supported MS Internet Explorer 5.0 and above only. The product manager who managed to get this dispensation approved deserves to be elevated to the highest levels. Many other managers didn't even bother fighting that battle. I'll hazard that even though IBM could have written something like GMail 7 years ago (and I certainly had a framework from K-station that would have made it possible with some work) it would have been a tough sell internally.
I've written about what was possible back then On Rich Web applications, AlphaBlox and Oddpost and On Gmail and DHTML architecture again
5. Transparency and Visibility
Joel Spolksy says that "it is easier to write code than to read code". This is what the Not Invented Here attitude betrays. We didn't have anything like community source where others might discover and find out about components that have solved similar problems.
(sidenote: when one asks about spending time to document one's component, one is often told that it is not mature enough, that we'd have to spend time supporting new users, we can't afford to slow down to support the one current users etc. There's a tendancy to keep everything close to our chests. Even if others will be able to help and flesh out flaws in our designs, it is thought that exposure to inchoate thinking will be a fatal flaw. This is a mindset that has to be overcome and on this, I am not so sure that it will happen, certainly I appear to me a lone voice in my parts of the wood)
See also Why Specs matter and the Law of Leaky Abstractions
Back to the subject of web interfaces for Notes... No one really invested in making great browser voodoo happen even though we had lots of great frameworks internally. Indeed I still see people grapple with problems that I have fixed before and keep digging up my old code to point them to. Ideally I would have been able to start an incubator at some site and others would have discovered it. Instead everyone is re-writing the same code all over. We haven't codified best practices or if we have, we haven't publicised them sufficiently.
6. Braindead "corporate standards"
Braindead "corporate standards" are another aspect of the problem. Back in 1999 when iNotes was just about usable and we were developing the Notes portlet for Lotus K-station (and subsequently WebSphere Portal), we were told that
- It was against "corporate best practices" for the http port to be enabled on Domino servers (because it increased the load on the server). Now it seemed weird for an HTTP server like Domino not to turn on its HTTP port, but let's leave that aside.
- If we got past that ridiculous plank, we were told that if the http port was to be enabled, that "corporate security standards" meant that it had to be SSL enabled, hence https had to be used. The idea is that the same HTTP basic authentication which we use in webmail and everywhere else in the web (other than when we're purchasing something) is too insecure for IBM mail (I've looked at my mail and perhaps only 3% should be confidential). Was it really so bad to just use good old HTTP?
We certainly paid a heavy price for this risk aversion.
- We found that using SSL made thing 3-10 times slower. The server load of processing SSL was insuperable not to mention that that it degraded performance on the wire. Hence iNotes which was already pushing the dtml envelope at that point was nigh unusable. The same concern for server load raised in the first objection was in fact exacerbated.
- As far as the portlet in K-station and WebSphere Portal went, where we were aggregating Domino's xml on a portal server and then marshalling it to the user, this was even worse because there was an extra level of indirection (mail server to portal server to the user's browser)
- On servers where HTTP was disabled, one had to use CORBA (DIIOP and again the SSL variant of this), this wasn't tuned since the DIIOP api was essentially the same as the local notes api, meaning that we couldn't do coarse-grained calls, and instead had to call many apis in a row to get the data that one http request would have obtained.
Now of course we worked around all of these obstacles and made the best of things, lots of people toiled to get performance fairly reasonable (the CORBA api is now (slightly) more coarse-grained, we developed Collaboration Services for Portal so that others could reuse our code and hard-learnt lessons - now the most downloaded aspect of WebSphere Portal etc), but our efforts were hobbled in many ways. We certainly could have had more things going our way.
Summing up, first impressions matter, just ask Malcolm Gladwell (Blink)...
I f the pilot deployments of the various webmail interfaces internally had been reasonable (instead of awful experiences for the executives and other audiences who tried them), we would have been able to get more buy-in, more investment and with a bit of luck might have made comparable interfaces to the snappy Gmail that not only us but also our users and customers would have loved.
It is true that many things had to go right for this vision to have transpired but let's admit that we shot ourselves in the proverbial foot and also acknowledge that it was a failure of imagination and ultimately a failure of technical leadership.
Looking at today's opportunities and the various directions in which Lotus/IBM can go in, I only hope that we don't Get on the wrong Bus this time.
File under: technology, software, web, Lotus, notes, IBM, history, strategy, organizations, adoption, decisions, iNotes, K-station, WebSphere, portal, history, toli
No comments:
Post a Comment