The Joy Of Small Things

• The Joy Of Small Things
• Life Is Sweet
• A Soundtrack of Small Things

Happiness is noticing blinking lights on your answering machine just past midnight in your hot apartment, bleary eyed as you try to open a window to let out that bee that somehow made it past the nets or alternatively dodge mosquitos to avoid the West Nile Blues.

Happiness occurs when you check your messages and hear that your favourite aunt has given birth to a new cousin, a baby girl.

Happiness is forgetting all about the bees and mosquitos and screaming to wake your neighbours up.

Happiness is the 3 hour conversation you then have with your parents back home about life, family, change and everything else.

Happiness is hearing about your grandma so overjoyed that she broke out in voluminous prayer. Your symptoms are genetic.

Happiness is going to sleep with a smile on your face.

Happiness is the dream you have of blowing off work the next day to catch the Fung Wah bus to head to New York.

Akua Abigail has arrived a few weeks early but is wonderfully healthy and her mum is similarly content.

Although we are thoroughly modern, having your first child in your forties is still extraordinary and there had been some trepidation all around.

Happiness will be the mad rush to New Jersey to welcome mother and daughter when they are discharged from the hospital over the weekend.

Happiness is being in England, Germany, France or Ghana (in addition to being thoroughly modern, we are a thoroughly dispersed family) and hearing said news and joyfully negotiating with travel agents to change flight plans handwaving away the prospect of paying several thousand of dollars for tickets to see your daughter or niece.

I first heard about my aunt's "loaded condition" a few months ago on the day I made my own modest proposal and a dear cousin stepped into matrimony. Thus the entire family has been smiling ever since and anticipating a wonderful end of July. Thank goodness I caught a few glimpses of her pregnancy and we all have bits to add for the historical record when I went to visit her. But back to happiness...

Happiness is remembering the various churches you passed in Brooklyn on that same day. New York's finest on display.

Happiness is recalling the name of the church you passed on your way to help a fellow bachelor out in Ikea and that confusing place we call the modern department store:

The Institutional Church of God in Christ. inc.

Happiness is trying to be a personal shopper and attempting to convince someone whose apartment has only a bed, computer, iPod, and beer in the fridge that it's worth it to spruce up since The Parents would be landing in just a few weeks to stay with him.

Happiness is your chutzpah when you, who are only a couple of weeks removed from confirmed bachelorhood,

have the gall to expound to your cousin on "thread counts" and explain that "some people" (you don't qualify who) may feel better in the morning if they slept on 450 thread count sheets rather than utilitarian prison-ware.

Happiness is recalling when you asked that young black woman in Target if "there were any deals on sets", her subsequent laugh and response that

"This is a family store!"

since she mistook your enthusiasm for sets of bedsheets with garden-variety sexual propositions.

Happiness was the spontaneous laughter you all shared. She had just herself propositioned Mr Bachelor with a Target credit card with "An additional 15% off today's purchase" that finally tipped the deal and would ensure the success of your home decorating campaign.

Happiness is knowing that she lost the pen that you offered her for the credit card application your cousin and her filled out.

Happiness has been writing for the past few months with the replacement Target pen she gave you.

Happiness is the decor of furniture in Pierre Deux, which reminded you of the most awful florid French and English houses.

Happiness is the curious looks you exchanged with your cousin as you walked into Pierre Deux as you saw the bright colours and the knowledge that in future you'll be able to crack him up with that codeword.

Happiness is making phone calls to your other cousin now ensconced in Richmond Virginia for her to launch her browser and reserve a car at Newark Airport since both of you guys were not air travelers and apparently you can no longer just walk up to those rental locations and pick a car.

Happiness is remotely directing the web transaction from New Jersey and discussing car options knowing that you were at the mercy of this babe in Virginia, sweet talking her to make sure she picked a large sedan.

Happiness is her astute interior decorating advice about which shops to attack in your bachelor intervention.

"Door Store, Ikea, Secaucus outlets and Target".

She pronounces Target as if it was some french word, that soft "g".

Happiness is leaving Ikea having only persuaded your cousin to buy a third of the things on his list

"I don't want to buy anything too permanent if I'll just be moving soon"

and seeing some normally unemployed guys on the roadside wearing signs promising a

Closing Down Furniture Sale!!!
Everything Must Go!

Happiness is following the signs and gesticulating to the 5 or 6 "Breadcrumb Guys" and being directed to the back of a furniture warehouse, it turns out that it was Corts Rental Furniture. Oh joy: bargain-basement rates on items that literally fell off the back of a truck.

Happiness is avoiding that business of making eye contact with the clientele of fellow desperados that had similarly been drawn by the flashy (fleshy?) advertisements. It reminds you of teenage expeditions to some "special shops" in Soho in the Red Light district of London when a bunch of you managed to get away from chaperones during school trips. You both feel a little dirty about being titillated by this "product".

Happiness is the sight of the gruesome furniture therein and the jokes you exchanged with one of the brothers who worked there when you enquired about a dining room table.

"It's seen better days".

Happiness is cracking up and quipping that

It's been to war... It's just come back from Iraq or Afghanistan.

Happiness is all the people in that dank warehouse breaking up and beginning to comment on the forthcoming draft, and Dubya's grudge match with evildoers who "tried to kill his dad".

Someone started calling the furniture Weapons of Mass Destruction.

At that we had to leave, those verbal Scud missiles hit too close to home, passing right by our Patriot (Act) defenses and Star Wars shields.

Happiness is getting lost in the twilight zone of New Jersey on the way to visit your aunt. Did you know that there were 4 adjoining towns within a 2 mile radius that all have streets with the same name - and not just one street but 5 or 6 streets with the same name and layout and all near the same rail tracks? Did you know that neither Google Maps nor Mapquest had cottoned on to the strange archeology of the Garden State?

Happiness is the increasingly frantic phone calls as you were lost and drove right to her address only to not recognize the house that was there.

Hmm... Everything sure looked familiar.

Happiness is your pregnant aunt, after an hour of this business, heading to the Bank of America parking lots of the wrong 3 of the 4 towns in a bid to rescue both of you hapless bachelors from your Garden State misery in the streets of Teaneck, Bogota and the like.

Stay where you are. I'll find you.

Really?

Happiness is your cousin's cell phone. Both you and your aunt don't have one so of course you can't call her up while she's looking for you in the wrong towns.

Happiness is the huge plate of jollof rice and chichinga (suya the Nigerians would call it, and others more generally kebab) that you wolfed down when you somehow finally made it to her house. Google Maps had suggested 30 minutes for your 2 1/2 hour expedition. Home cooking made up for the difference and your aunt dotes on you.

Happiness is hearing your aunt arrange for your cousins in France to join you and gatecrash a wedding in London that you just mentioned you were about to attend next weekend.

Happiness is remembering the conversation you'd had just weeks earlier with your friend Kweku in which you told him to expect that Ghanaians would "always gatecrash weddings" and commiserating about the madness of wedding preparation, something you've just begun thinking about.

Happiness is realizing that, after a mere 5 minutes of trans-Atlantic conversation, it looks as if you will be leading a party of French gatecrashers to said wedding and imagining Kweku and Zai hyperventilating.

Happiness is your aunt asking for Kweku's mum's phone number.

"I don't want to talk to Kweku, he'll be too busy with the wedding. I want to talk to his mum."

Indeed.

Happiness is going to a bus stop to wait for a New Jersey transit bus to return to New York's Port Authority.

Happiness is realizing that she had looked at the wrong column on the schedule for Bus 168 so that you will have an hour to wait on this Sunday evening.

Happiness is the pair of you sitting at the bus stop and simply chatting about whatever comes to mind. That waiting hour becomes one of your fondest memories of your aunt in retrospect.

Happiness is discovering that you have a copy of your musings on Inauguration à L'Africaine and laughing with her as she reads it and you recount on the even more absurd items you didn't write about.

Happiness is seeing your aunt's permanent outrage and also when she encourages you to continue doing what you do:

"You should send it to the president. He should sack all of them. And you've written it in the nicest way."

Thus I'll modify Arundhati Roy's formulation and write a short piece about

The Joy Of Small Things

Life is Sweet

My favourite movie on this theme is of course Life is Sweet, Mike Leigh's brilliant exercise in celebrating small things from 1990. Delusions of restaurant grandeur, a slice of family life, dance classes, encounters with spoons and broken limbs, chocolate fetishes, eating disorders, sexual confusion, teenage angst, and most of all inimitable and life-affirming laughs. The decor of some of the rooms in this British middle class nirvana seemed to have come straight from Pierre Deux. It's a real pity it hasn't yet been released on DVD.

A Soundtrack of Small Things

As usual a playlist for this joint

• Anita Baker - Sweet Love
  There you have it.
• Incognito - Step Into My Life
  Positivity is the thing.
• Price Nico Mbarga - Sweet Mother
  Ever since 1976, this has been the quintessential Highlife joint. Afrobeat was never more danceable or happy. Its lyrics are a celebration of African motherhood.

  Sweet Mother I No Go Forget You
  For The Suffer Wey You Suffer For Me.

• Cameo - Attack Me With Your Love

  Baby

• Curtis Mayfield - Sweet Exorcist
  Soul music's premier falsetto.
• Bunny Mack - Let Me Love You

  My Sweetie, My Sugar

  My long search for a good copy of this continues...
• Chaka Khan - Sweet Thing
  How to start everyone singing in 4 bars.
• Finlay Quaye - Your Love Gets Sweeter

  Love gets sweeter every day
  Love gets sweeter every way

• Prince & The New Power Generation - Sweet Baby
  Like a lullaby
• Mary J. Blige - Sweet Thing
  Puffed up hip-hop soul
• Loose Ends - Mr Bachelor

  You should have told me

  From The Real Chuckeeboo, I need to do a Loose Ends appreciation at some point; they are truly London's finest.
• U2 - Where The Streets Have No Name
  a.k.a. New Jersey

See also: New York Trip

File under: life, happiness, children, babies, cousins, aunts, family, scramble, conversation, airlines, buses, plans, Small Things, New York, New Jersey, whimsy, funny, travel, journalism, storytelling, shopping, joy, food, jollof, toli

Manifest Destiny

A savaging is in order.

I'm already in a vicious mood and more on that shortly but I have to say that I almost had a case of airplane rage while flying back from San Francisco last Sunday. There I was, reading what started out as a reasonable essay by Michael Ignatieff, Who Are Americans to Think That Freedom Is Theirs to Spread?, when I came across the following:

Never has America been more alone in spreading democracy's promise.

I didn't even know where to start with that last sentence and I still don't. I've written recently about a supremely supine media, but here we have hubris beyond belief from the academy. Still let me give it a little context, lest I be accused of nitpicking his point.

Other democratic leaders may suspect Bush is right, but that doesn't mean they are joining his crusade. Never have there been more democracies. Never has America been more alone in spreading democracy's promise.

The cheek of it. The self-righteous solipsist.

Ignatieff used to write similarly long, ambitious pieces for The Observer in the 90s. I can imagine him content with the ring of those heady sentences but he really should be taken to task. There is a certain class of supposedly "Public Intellectuals" who are simply pernicious because of the palliative cover they provide for Rovian rogues who probably can't believe their luck that they can point to ideological justification for their misdeeds in the pages of the Grey Lady and from Ivory Towers I'm quite familiar with. And from that sentence on, the Great Game of misdirection and bromides proceeds apace.

Now the deification of Reagan is one thing (although there was a little shock on my part in reading the ensuing pablum about Ronnie's supposed prescience), but that deserves a separate post and I've long promised The Governor of Redmonk something on that front. Per Ignatieff, Reagan is supposedly responsible for "the emergence of democracy promotion as a central goal of United States foreign policy". The repeated misadventures in Latin America, Africa, the Middle East and almost everywhere else are conveniently skipped over. The dissonance of soaring cowboy rhetoric, garden-variety Iran/Contra criminality, defence industry sinecures, or Savings and Loan cronyism and plain hypocrisy in the Realpolitik that Reagan's cohorts practiced are not commented upon, nor is it anywhere acknowledged that those swarthy developing world types incurred considerable collateral damage when they served as battlegrounds and proxies in that awful Cold War.

In any case, it's understandable that we want to ascribe superhuman attributes to B-movie players, and we should have a little respect for genuinely popular "Great Men" who were in tune with the zeitgeist. The rush to beatify John Paul II is similarly par for the course.

Still Ignatieff's conflation of the Iraq blackhole with the supposed project to spread democracy is even harder to take, but that too is just a post-hoc rationalization of ongoing, ill-conceived follies, or rectfication of errors as Billmon would put it. I understand we'll hear yet another turn of the damage mitigation in the much leaked presidential address on the "resolve" required to "stay the course".

I had just finished reading a piece in the San Francisco Chronicle, Optimism fades, but war goes on, which sought out the opinions of those who are bearing the weight of the current imperial overstretch. On the ground in Iraq, the soldiers, who by inclination are not normally the most reflective of types, no longer have the WMD plank of yore as motivation, hence they are falling to either the "Saddam was a bad guy" plank or to the "better we fight over there than in Manhattan" party line. And yes, Uday feeding his people to the lions in his private zoo is awful but we won't mention Rumsfeld visiting Uday's father in the 80s selling him weapons and turning a blind eye to the gassing of those pesky Kurds, all in the name of access to oil or of "containing Iran". There is no moral equivalence but there is a certain axis of hypocrisy at work.

Despite the seeming lack of progress against the insurgency - as evidenced by the rising casualty count of both Americans and civilian Iraqis - it is those scenes that make Wood, a career officer who has been in the army for 19 years, a firm believer in his mission.

Even with all the setbacks, he adds, the war must continue to be waged in Iraq. Otherwise, it will be waged on American soil, he says.

The soldiers don't even know that back home the rhetoric has gone past The Evildoers or even The Neighbour's Grass Theory to the "we can't allow our folly to fail" routine. What a wonderful world I suppose, to mess up other people's gardens.

"But yet, but yet", Ignatief writes. There must be some good to find in all of this, something to make us feel good about what has come to pass. So then we turn to this:

While Americans characteristically oversell and exaggerate the world's desire to live as they do, it is actually reasonable to suppose, as Americans believe, that most human beings, if given the chance, would like to rule themselves. It is not imperialistic to believe this. It might even be condescending to believe anything else.

Now if anything has been oversold in his essay it is precisely his previous exercise in rhetorical overstretch.

Never has America been more alone in spreading democracy's promise.

Really?

America alone? I beg your pardon.

Democracy promotion is not the preserve of Americans, it is a universal struggle. Further it is a struggle that is undergoing many setbacks currently because of proverbial Quiet Americans (and would it that Dubya and co were quiet). That same laudable Reagan was for breaking sanctions on the apartheid regime in the name of not encouraging that dastardly communist Nelson Mandela who was busy breaking rocks on Robben Island. Indeed, if I remember correctly, even the democratic institutions of the US need looking after, and some of us are monitoring the situation here with alarm.

In Ghana we get daily lectures from the State Department about governance, corruption and institutions, lectures on the form of democracy in short. The concern is well taken but all too often these are just a cover for securing very favourable access for American firms to our local markets. In terms of actual aid and strengthening of democracy, those sensible Canadians, those quiet Dutch, the Danes or even the Brits who are quietly doing penance for their colonial past are in fact far more effective on the ground in spreading democracy's promise.

The story of Millennium Challenge Accounts is the correct parable in this respect: announce lofty goals to provide cover during election season and actually don't do anything, wrap the "aid" with red tape, rinse with a touch of quid pro quo a la Haliburton and lecture away about the feel good quotient of the benevolent elephant. Levelling the playing field of global affairs is actually anathema to America foreign policy as we have seen - to be fair the French and others are no choirboys. "But yet, but yet".

Still we sit and listen to the lectures because, in our position, we'll take any help we can get even if it is the incidental backwash of self-congratulation. But that's the thing: the rhetoric is all about democracy when the practice is about exceptionalism. As a piece I read on the same flight put it, it all depends on which side is doing the globalizing. The unresolved issues in the American Dream, the dissonance between perception and reality is something that this young country is still dealing with.

The sadness that came over me as I finished the article was much like the disappointment one has felt reading almost anything Thomas Friedman has written in the past 4 years. Others have taken Chutzpah Thomas to task for his most brazen piece of revisionism from last week. And like Dershowitz, his discourse has devolved into the most unseemly blowhard rhetoric.

Now I've read Isaiah Berlin on Liberty, I read the Woodrow Wilson Quarterly, I've read the canon of both the liberals and the conservatives in the west. You think that by reading the same texts that you've taken away a coherent worldview but, truth be told, there are considerable blindspots and much solipsism in these "world"views. And these blindspots manifest themselves in the contortions of apologia and logical inconsistency borne of hurt pride that we've been seeing especially since 9/11. What's worse is that no one calls these people on it and we are supposed to bear their hot air lectures with a smile.

This reminded me of the way many Africans felt on reading Robert Kaplan in The Coming Anarchy back in 1994. Kaplan at least was embarrassed that his travel journalism came to be construed as policy prescriptions for Bill Clinton and Madelaine Albright in dealing with Rwanda and the madness in the former Yugoslavia. Actually, Kaplan was making a more subtle point than Ignatieff or Friedman ever will and when I see neighbour's houses on fire and worse, I can at least empathize with his concern for the fabric of my society. But I can't wash my hands of my home so I differ on the remedies (cut-and-run) prescribed from this side of the Atlantic.

I read the rest of the article half expecting some Social Darwinist claptrap to emerge and it's quite close: Ignatieff's essay is simply Manifest Destiny revisited as farce.

A Manifest Destiny Soundtrack

The sound of manifest destiny has mostly been a soundtrack of irony because the foundations of the ideology simply dissolve away with the slightest hint of scrutiny. Thus I haven't been able to find much worthy art in the affirmation of self congratulation. Consider this then a passive-aggressive musical take on the theme.

• Tears For Fears - Everybody Wants To Rule The World
  Don't we know it.
• Jaguar Wright - Self Love

  Self Love
  Self Preservation

• Steel Pulse - Babylon Makes The Rules (lyrics)

  You've Had All Night And Day To
  Consider And Pray
  You've Brought Fire On My Head And
  Now You Must Pay...
  
  Babylon Makes The Rules
  Where My People Suffer...

  
• Kurtis Blow - If I Ruled The World
  The original critique
• Nas (ft. Lauryn Hill) - If I Ruled The World (Imagine That)
  Eschewing the bling-bling before the bubble.
• Jamiroquai - Manifest Destiny (lyrics)
  
  

  Every child can learn a different thing
  See I tried a million times
  But I had to change my mind
  Sadness was the best thing I could bring
  On the inside he was dead and this is what he said
  As he took away the blue above the clouds
  "Oh, superiority was clearly meant to be
  So destiny will manifest in time"
  
  So where has all the love gone?
  A troubled trail of tears will tell the tale
  Of how I was put down where I don't belong
  Woman, child and man for sale
  For ethical slavery is just an absurdity
  How can you be alive when you are dead
  With these chains of hypocrisy
  The shame of my ancestry
  Forever strained in blood in which you tread

Update: Lord of the Rings File under: hubris, solipsism, cheek, chutzpah, blowhards, intellectuals, politics, academy, social darwinism, USA, Ignatieff, Friedman, america, exceptionalism, perception, narcissism, observation, rant, toli

On an Ambiguous Adventure

A short note to Kenya Hudson who unilaterally declared an end to her blog experiment the aptly-named Ambiguous Adventure...

A couple of comments/observations:

Those who research the blogging phenomenom have noted that:

• The vast majority of blogs are abandonned after 90 days.
• Most bloggers experience an existential crisis at some point. Usually this comes down to the blog not living up to their expectations. There have been numeous high-profile instances of this.

I've noticed, in the past few months, that life has intruded more than a little on your ambitious adventure causing a little matter of ambiguity to come into the picture.

I believe that you're in good company and 10+ months of blogging would be well beyond the call of duty as those statistics would attest. We all blog for different reasons and get pulled in different directions. For my part, my crisis came after 6 months when various insanities at work and other life issues raised their head - some of which continue to this day.

My blogging hiatus and its consequent resolution came from a curious direction and I was surprised that my voice returned, refreshed by a 96 year old woman who calls me Frank and who caused the discovery of a community I didn't know I had.

But to engage you a little, Kenya, I'll simply suggest this: there's a paucity of voices such as yours on the web.

I am hoping that you'll rather consider this period a time out rather than an end. Unilateral acts are a little much in this day and age. Just ask the Croats about declaring independence in the past decade knowing that opportunist Milosevic types were itching to take you on and the costs they have suffered. Or take these recent ones for example which led to this: Zimbabwe Slum Dwellers Are Left With Only Dust. Metaphorically, and with some tongue in cheek, you're leaving your slum readers in the dust a la Robert Mugabe and lobbing some tear gas surreptitiously supplied via Malawi to boot.

Thus I'll also suggest a different track. I have long seen this web thing as a conversation engine and even though some conversations fizzle out, many will pick up the strands and you've started quite a number that we, your readers, are all trying to digest and weave together.

And will you resist the opening about Mugabe's continuing misdeeds (and the sanctions-breaking Malawi connection) that begs for your kind of dissection? If I wasn't spread so thin I'd be weighing in myself on the idea of a government preparing for over a year for the propitious moment to raze the accomodation of and destroy the livelihoods of 300,000+ of its citizens without notice and moving them into camps under the guise of "driving out the rubbish" (and political opposition) - something that reminds me of what happened repeatedly in Nigeria under military rule in the slums of Lagos and other towns; a government so far removed from the everyday concerns of those it is supposed to serve. I've touched on slums and squalor in the past, I wonder what you thought about that piece or my other ramblings.

And since I know you don't confine yourself to Africa, how about this one: Bitter divide over plan to wall in Rio's slums. Should we be building walls around modern day ghettos (to prevent the collateral damage of stray bullets ) or deal with the social ills that give rise to the gangs, Bus 174 episodes and Cities of God?

I'll also ask you some of the questions that I've been asking the technical audiences I've been speaking to since the African toli has been scarce of late:

• How did you get on the web?
• What caused you to land your flag on this here internet?
• What caused you, your friends, or family to live, shop and commune on this here web?
• And what will it take to make you stay?

The internet is full of mysterious wonders. As an example, The Girlfriend and I were emailing links to her graduation pictures to friends and family and all of a sudden we saw some photos from the graduation at Tech, the Kwame Nkrumah University of Science and Technology at Kumasi A friend in Ghana, upon seeing her photos, signed up to Flickr and spent a few minutes uploading their graduation photos at some Internet cafe or other (or maybe it was a couple of hours, internet connections being what they are in Ghana). I can't tell you how wonderful it was to see the colourful glimpses of graduation done Ghana-style among the global graduation tag at Flickr even if only temporarily (the photos were since made private for some reason much like you're heading for the private sphere). And indeed we've already seen some great visions of the Ghana goes Flickr meme. Great sights and the sound of those voices again...

Lastly, a matter of serendipity:

Last summer, I lost my copies of the novel of your eponymous blog, Cheikh Hamidou Kane's L'aventure ambiguë both the original and the decent translation. Well I think I lent them to someone, I'm always giving out my best books and have for example, bought Camara Laye's L'enfant noir and Ken Saro-Wiwa's Sozaboy at least 5 times, not to mention Milan Kundera's Book of Laughter and Forgetting, Julian Barnes' A History of the World in 10 1/2 Chapters or Patrick Chamoiseau's Texaco which speaks cogently to the slum life that is the modern wold.

Rather than heading to Schoenhof's Foreign Books and bankrupting myself on books as I normally do since my wishlist is so huge and I too believe in compulsive book buying, I've been periodically checking Amazon and eBay for a used copy. Thus it was that last Thursday, some kind soul sent me a pristine copy of the book which arrived in a battered envelope (it had been advertised as being in acceptable condition). This internet thing is something: receive one of African literature's greatest works for $4.50 after just 2 clicks?

That I received the book on the same day that you apparently went all formal and declared a case of unsustainable blog development is a cause for some head-scratching on my part.

I'll be re-reading said ambiguous adventure in coming weeks, right after I'm done with Suzan Lori-Park's Getting Mother's Body which you should also read if you haven't already. A friend hipped me to it and its vibrant and cacophonous sound of the African-American experience is asserting itself in my ear whenever I can spare fifteen minutes - far more refreshing than the increasingly sclerotic Toni Morisson. And Lori-Parks has more groove than almost any living writer or playwright; I hope you managed to catch Topdog/Underdog if it passed your town (there's even a DVD about the struggle and creative process behind that great achievement. By the way, did you hear that Terry, and by transposition, Stella now has now lost her groove in a mix of sexual confusion, age and cultural misunderstanding. That could be the next entertainment in our silly season of discontent now that MJ has been released from a hysterical witchhunt.

I've been looking forward to engage you in the conversation about the ambiguous adventure, asking for example how the novel measured up with Mariama Ba's Une si longue lettre which I would have thought was more in your line (which again has a good translation). This short note is turning out to be a long letter... And for the record, what caused you to go with The Cheikh's words in your allusive blog title?

And what about Kwame Gyekye's Tradition and Modernity that I spotted on your reading list a few months back, I read it last year and have alluded to it in passing. We find ourselves mediating tradition and modernity daily and I wonder what the analogues of the blogging world were in the traditional past. Gyekye would have some provocative thought on the issue I'm sure. I also picked up a used copy of his African Cultural Values. All secondary school children should read it and not just Ghanaians or Nigerians, all American kids would do well to get some of his flavour and learn about cultures that have influenced the world but that aren't normally advocated. And why the hell is the book out of print in this country? Isn't he the greatest? A philosopher from the old school full of scholarship and learning and still vitally relevant today. I've been meaning to start that conversation on tradition and modernity for a while. But will I have a forum? And, if so, will you lead the way?

Do take your time before you resurface, I know it's difficult to regain balance. I fear there is much I don't do justice to myself. Still I firmly believe in Gabriel Garcia Marquez's maxim:

Any idea which couldn't stand a few decades of neglect isn't worth anything.

Thus a literal and figurative head nod in your direction Kenya.

I remain subscribed in Bloglines.

File under: Africa, Kenya Hudson, blogging, life, balance, conversation, communities, web, literature, politics, slums, letters, philosophy, ambiguity, adventure, Ghana, writing, toli

On Iframes, WSRP, REST and Portals

• On Prescience
• On Iframes, WSRP, REST and Portals

I'm still getting lots of good feedback to my presentation on REST - the web style (background) which I will summarize shortly. One of the more curious responses was to something that I didn't even dwell on in the presentation; indeed I think I skipped past it in my concern to get the big picture articulated. I suppose it was the proximity of yet another acronym, WSRP, to a couple of slides about complexity in the IBM technology stack, and a very public case study bemoaning the fact that Websphere Portal only added bookmarking after 3 years that raised a little concern. This was part of my subsequent response to one of those comments, slightly edited for clarity.

On Prescience

I fully agree, we should celebrate any and all who contributed to us internalizing the web style in WebSphere Portal. We should be very explicit about the pain we and our customers suffered and the evident gains that came once we learned our lesson.

The notion that it was still a struggle to get that simple feature (the ability to bookmark) included in the WebSphere Portal 5.1 release is actually a commentary on our previous attitude to the web style.

I would add to the list of those to be celebrated, those screaming Cassandras who were incredulous that we could ship a web product while ignoring url addressability and even tried to prototype some rudimentary addressability and sneak it in back when we had commit privileges to the core. If only we had had more nerve or had been able to better communicate the importance of this point... I know it normally takes 3 releases for a product to become useful, but sometimes magic can happen in the first release and with that comes thought leadership (Glory) and perhaps also some money (Gold).

On Iframes, WSRP, REST and Portals

Leading with the iframe invocation argument was always the weakest argument of those who I'll cast as the "K-stationers" in the early days of WebSphere Portal. After all, and like you note, even though iframes may provide

• easy integration of existing content
• multi-threaded download in the browser to lower the latency for the user
• caching closer the client (indeed in the client)
• offloading the portal by removing a layer of indirection in the portlet -> portal server -> client browser route
• introducting the possibility of moving aggregation to the client (leveraging Moore's law and bandwidth increases to the client)
• further recasting the "Portal Server" to a "Portlet server"(in other words there should be no difference between a "remote" and a local portlet) etc . That is the lesson of uniform interfaces in the web (e.g. URIs)

Now that's a pretty strong "weak argument" but we should acknowledge that Iframes also have downsides. Iframes mess with user expectations especially with respect to focus and some browser conventions are a little upset by them (e.g to bookmark, you need to right click on the frame in question). Browsers did solve the issues with the navigation history after a few years, but there was a little confusion in the interim with back button and refresh etc).

6 years on, I hope I have learnt to recognize the stronger aspects of an argument about systems design. What we didn't recognize or adequately evangelize was the value of the other aspect of an approach that more fully embraced the web, namely:

Basing the application semantics of the portal directly on URIs and the use of those 4 verbs that are the core of the web.

To explain, HTTP is an application-transfer protocol. It is something that works at the application layer. It has its native notion of how CRUD like operations are done (GET, POST, PUT, DELETE).

The REST proposition again:

• Identification Of Resources
• Manipulation Of Resources Through Representations
• Self-Descriptive Messages
• Hypermedia As The Engine Of Application State

I haven't kept abreast of WSRP in the past 5 years and thus my characterization of it was perhaps a little careless, but that's what you do when you make a presentation that borders on the provocative. In mitigation, I'll confess to being what I've termed a Layer Stripper.

Let me handwave a little... It strikes me that building an application protocol with CRUD like functionality that will typically (or even solely) be used over HTTP can lead to a little impedence if one is not careful.

To take my Technical Arteriosclerosis Terminator scalpel to WSRP, I would only ask:

• Do the CRUD like operations in WSRP map one to one with HTTP's native semantics? In other words, is the moral equivalent of a PUT in the WSRP world implemented as a PUT when the "transport" is HTTP?
• Is this true of at least those 3 verbs at the core of the Web (GET? POST? DELETE?). (I'll assume that WSRP adds perhaps more verbs in its applications semantics).
• Are the security primitives going to be leveraging HTTP's native security features or are we rolling our own?
• Are there any other "transports" other than HTTP? Is there a CORBA mapping for WSRP?

RFC 3205 On the use of HTTP as a Substrate should be required reading here.

I'll suggest that if we were starting from scratch we might do the following

• Start with Resource Modeling and ask: what are the core resources that need to be identified in a remotely invoked portlet?
• What representations need to be returned? For this we'd probably pick some standard Portal XML vocabulary or, for browser rendering, an XHTML format, for pervasive rendering some other Markup Language
• What are the operations that need be performed on said resources?
• What HTTP verbs are appropriate for each operation when we manipulate those resources?

Joe Gregario's Show Me The Code column has been going through this exercise with the example of a bookmarking service - well worth reading. A remote portlet may have more complex semantics but one should go through this to clarify our design and architecture. I assume that WSRP implementors went through such a process as that spec was standardized.

The result would be something like simple URI commands returning representations and manipulating them using well-worn HTTP semantics. I very much hope that is what WSRP has evolved into. We'd get:

1. Visibility to HTTP intermediaries, first the caching proxies in the web sense. (like the page caching in the portal)
2. Since we'd be using URIs, someone may think up new uses for our remote portlets and be able to easily embed them in their application through simple hyperlink and the use of the appropriate verbs etc. Third party Glue Layer people could "remix" our portlets through pipelining and filtering or whatever else those people do (I'll note that IBM recently bought Gluecode)

With this approach there is no API per se, the interface contract is explicitly manifested in the exchanged hypermedia and associated operations on the resources.

"What? No API?" "This RESTful stuff has no substance". "Where is the value add?"

That indeed is the paradox of the web style. The "API" is typically HTTP/URI/HTML/XML.

The fourth plank of the REST proposition is a difficult thing to internalize, I certainly couldn't express it adequately 6 years ago.

"Hypermedia as the engine of application state"

Ponder that for a few weeks.

[Update] James Snell adds: "HTTP is the programming model".

Some have suggested that one would additionally require some service description language, perhaps WSDL, for such a service. On that, the jury is still out. In the example I highlighted in the presentation, Google Maps didn't publish any WSDL. Craigslist didn't publish any WSDL. They simply had a clean URI api with well-defined parameters that were obvious to any developer with a week to spare and idle curiousity satisfied by the View Source impulse. It was obvious what verbs to apply: POST or GET, PUT (they don't delete their maps, but if they did, that is how one would proceed). Third party intermediaries simply figured out that url hierarchy and the associated verbs, the identified resources and what formats the representations were retuned in. Then they manufactured serendipity. Now there is no binding contract and things can get broken at any time but it wouldn't be in Google's interest in their competition with Mapquest or Terraserver to break people that compose applications on their platform and brings eyeballs and mindshare to their company.

This then is my argument for changing the frame of the debate:

There's a complexity and layering argument that naturally falls out of the REST viewpoint which should guide the applications that one builds.

The web is about identifying important resources and exchanging representional state. There are certain constraints that are made to enable global scope and evolution. Being on the web, is being an active participant in this scheme of things.

Identifiers lead almost to having the location field as the command line.

Resource modeling as your starting point sets you in direct consonance with this notion and exposes you to ease of composition.

e.g. Jon Udell's Library Lookup project

An information architecture that starts with hypemedia lends itself to the construction of simple interaction designs for both humans and machines, and there are huge amounts of tools readily available for this.

The question is then one of leverage; leverage in economic terms is all about making externalities work for you. The endpoint of this approach to system design is what I call a virtuous cycle of managed serendipity.

One response to this piece was that actually WSRP stood up quite well to those questions I had hand-waved in its direction, apparently it was not too complex, it addressed wider issues and use-cases etc. I think that is great and hopefully the leaks have been plugged in that spec so that it can be mapped reasonably to HTTP; I guess the argument is that it is not an unnecessary layer. I'm still unclear if WSRP is ever used with a "non-HTTP transport" or security, I'll dig into that at some point. In any case, the point of my talk was the same scrutiny should be brought to bear on all of IBM's technology stack and unnnecessay layers shed and/or leaky abstractions plugged to make them work better with the web style.

A later follow-up, privately-addressed, asked:

"Isn't a "portlet" fundamentally a presentation-level object? Why is the concept of "portlet remoting" not considered an oxymoron? Shouldn't we be talking instead about the simple expression of web services as data (with, perhaps, media-type detection as an enabler of polymorphism"

There was lots more and quite fulminant at that, I'd have to say that's all true and expressed more concisely than I did. It's good to know that there are like-minded people around. I didn't mention content-negotiation in my response but that is another aspect built into the web style that can be leveraged but that is constantly being reinvented at higher levels and layers. There's lots more to say but that's the lay of this technology world... In evangelism the first step is to get people sensitized, after that it's best to proceed one argument at a time.

---

P.S. I've been told there's been too much technology toli of late so I'll try to push some literary, musical or African toli to the head of the queue, it can be more fun and oftentimes more rewarding.

File under: technology, web, REST, systems, design, architecture, tradeoffs, adoption, leverage, composition, iframe, wsrp, WebSphere, ibm, portal, portlets, aggregation, software, complexity, uri, http, xml, html, xhtml, lotus, IBM, toli

On The Limitations Of Notes On The Web

• Some context
• On The Limitations Of Notes/Domino On The Web

In the week after Lotus announces lots of innovation in the forthcoming version of Lotus Notes, with lots of great screenshots and undoubted enhancements, it may seem weird to post a short article I had written a few weeks ago on the past limitations of said product in my internal IBM blog. I do this because I've been taking a historian of science's approach to my writing on technology and its effects. I am always interested in how organizations work and how they make decisions. And a certain amount of dirty laundry airing is healthy if only to pat ourselves on the back to show that now we have learnt our lessons and that there is a lot of introspection and thought in the company. Also my focus is not on the Notes client which has always sold itself, but on rather the web interfaces that product has had since the advent of Domino, its first embrace of the web. As you may know I am an unabashed believer in the web as the ubiquitous client. This piece should give some context to these three slides from my recent REST presentation. They weren't quite a twisting of the dagger but more a challenge to the development organization. I believe there's Joy in Repetition and with a hip-hop sensibility remix myself constantly. So here's the original version before the sample.

On The Limitations Of Notes/Domino On The Web

Joe Turic and others are pondering Lotus Notes vrs Gmail and coming down in favour of the latter.

The complaints are about "Why is the email program that we use internally so limited?" and some of it has to do with storage quotas and the fact that we have very low ones (even as the platform handles 49 GB mailboxes just fine as Ed Brill keeps pointing out when comparing it to Microsoft Outlook). Most of the reason for this hence is not technical but rather I suspect a hint about the direction of email retention policy.

Later David Crumpton asks "Does Notes offer a Webmail interface?"

And of course it does, and it even has two, the regular webmail interface and iNotes, the prettier dhtml version which should give a snappier performance with lower latency for the user. A third way of getting your notes mail in a browser is through a portlet in WebSphere Portal.

The question is, why did these web interfaces not get widely adopted?

Let's start with the question of investment.

1. Investment and Adoption

Plainly put, not everyone at IBM has bought into the web, html and http.

I would count Lotus/IBM's efforts over the past 7 years as half-hearted although now we are supposedly hardcore about the web, Linux and open source in general. The thing to take away is that, at the point that judicious investment could have been made, and leadership in thought, if not in software products, was possible, we blinked.

IBM does middleware and services and moves slowly due to a sense of innate conservatism. I think this is fair, there are parts of IBM that are bleeding edge and one has to get buy-in before the rest of the elephant moves. As someone who was advocating the web almost from the beginning, it has been a long road to convince others to buy-in. The fact that we are now moving (and quickly) is great but I think it is cause for rueing missed opportunities.

2. Innefficient IT structure at IBM

Jim Gray's Distributed Computing Economics (pdf) makes a devastating case for the relative innefficiency of many in the industry that haven't embraced the web style.

"Megaservices like Yahoo!, Google, and Hotmail have relatively low operations staff costs. These megaservices have discovered ways to deliver content for less that the milli-dollar that advertising will fund. For example, in 2002 Google had an operations staff of 25 who managed its two petabyte (2^15 bytes) database and 10,000 servers spread across several sites.

Hotmail and Yahoo! cite similar numbers - small staffs manage ~300 TB of storage and more than ten thousand servers."

Up until 2000, I believe the operational staff at Yahoo was 8 and included the founder Jerry Yang.

Google has embraced redundancy and written software to work around human fallibility and hardware reliability. Google File System is not too shabby, if may say so. We have efforts in the same arena whether it is the Grid and autonomic computing work and say the move to VOIP but are we eating that dogfood?

Are we prepared to change our corporate structure and have profits come more from well-designed and usable software as opposed to consultantware? Are we prepared to eat our babies and make products that have much lower operational costs?

Tim Bray had this to say $46,213,000,000.00

I looked up the answer to the question: What is IBM's consulting revenue? In 2004, IBM's gross revenue was $96B, of which $46B was Global Services, i.e. consulting. I see that basically as testimony to how our profession, the IT profession, has failed our customers. Nothing against IBM; in fact, as solution-providers go, my experience is that IBM GS is pretty good. But if you see IBM as a microcosm of the industry, it shouldn't cost $46B in consulting to deploy $50B worth of technology. It's not going to be easy to get there, and it's going to take a long time, but we just have to focus on making things simpler.

Now that Godfather Bray is at Sun, perhaps there's a little pleasure in tweaking us. But the question still remains. Have we bought into Radical Simplification?

3. Technology Adoption Takes Time

It takes time for the effects of technology to be felt and for people to get comfortable with the web. We have now had 10 years of Moore's Law in the Datacenter and so we have examples and success stories to pick from. It was a more uncertain world 5-10 years ago and perhaps as a corporation we were hedging our bets. When IBM moves it's a sign that something is ready to take off.

4. Fragility of the browser platform

One answer is that usability was a problem in webmail interfaces that are used daily. In the browser world 5-7 years ago, you had to handle that beast called Netscape 4.7 which meant that it was a case of lowest common denominator in the user interface and frequent browser round trips (read poor user interaction experience due to latency). It was controversial when iNotes was developed and proclaimed that it only supported MS Internet Explorer 5.0 and above only. The product manager who managed to get this dispensation approved deserves to be elevated to the highest levels. Many other managers didn't even bother fighting that battle. I'll hazard that even though IBM could have written something like GMail 7 years ago (and I certainly had a framework from K-station that would have made it possible with some work) it would have been a tough sell internally.

I've written about what was possible back then On Rich Web applications, AlphaBlox and Oddpost and On Gmail and DHTML architecture again

5. Transparency and Visibility

Joel Spolksy says that "it is easier to write code than to read code". This is what the Not Invented Here attitude betrays. We didn't have anything like community source where others might discover and find out about components that have solved similar problems.

(sidenote: when one asks about spending time to document one's component, one is often told that it is not mature enough, that we'd have to spend time supporting new users, we can't afford to slow down to support the one current users etc. There's a tendancy to keep everything close to our chests. Even if others will be able to help and flesh out flaws in our designs, it is thought that exposure to inchoate thinking will be a fatal flaw. This is a mindset that has to be overcome and on this, I am not so sure that it will happen, certainly I appear to me a lone voice in my parts of the wood)

See also Why Specs matter and the Law of Leaky Abstractions

Back to the subject of web interfaces for Notes... No one really invested in making great browser voodoo happen even though we had lots of great frameworks internally. Indeed I still see people grapple with problems that I have fixed before and keep digging up my old code to point them to. Ideally I would have been able to start an incubator at some site and others would have discovered it. Instead everyone is re-writing the same code all over. We haven't codified best practices or if we have, we haven't publicised them sufficiently.

6. Braindead "corporate standards"

Braindead "corporate standards" are another aspect of the problem. Back in 1999 when iNotes was just about usable and we were developing the Notes portlet for Lotus K-station (and subsequently WebSphere Portal), we were told that

1. It was against "corporate best practices" for the http port to be enabled on Domino servers (because it increased the load on the server). Now it seemed weird for an HTTP server like Domino not to turn on its HTTP port, but let's leave that aside.
2. If we got past that ridiculous plank, we were told that if the http port was to be enabled, that "corporate security standards" meant that it had to be SSL enabled, hence https had to be used. The idea is that the same HTTP basic authentication which we use in webmail and everywhere else in the web (other than when we're purchasing something) is too insecure for IBM mail (I've looked at my mail and perhaps only 3% should be confidential). Was it really so bad to just use good old HTTP?

We certainly paid a heavy price for this risk aversion.

• We found that using SSL made thing 3-10 times slower. The server load of processing SSL was insuperable not to mention that that it degraded performance on the wire. Hence iNotes which was already pushing the dtml envelope at that point was nigh unusable. The same concern for server load raised in the first objection was in fact exacerbated.
• As far as the portlet in K-station and WebSphere Portal went, where we were aggregating Domino's xml on a portal server and then marshalling it to the user, this was even worse because there was an extra level of indirection (mail server to portal server to the user's browser)
• On servers where HTTP was disabled, one had to use CORBA (DIIOP and again the SSL variant of this), this wasn't tuned since the DIIOP api was essentially the same as the local notes api, meaning that we couldn't do coarse-grained calls, and instead had to call many apis in a row to get the data that one http request would have obtained.

Now of course we worked around all of these obstacles and made the best of things, lots of people toiled to get performance fairly reasonable (the CORBA api is now (slightly) more coarse-grained, we developed Collaboration Services for Portal so that others could reuse our code and hard-learnt lessons - now the most downloaded aspect of WebSphere Portal etc), but our efforts were hobbled in many ways. We certainly could have had more things going our way.

Summing up, first impressions matter, just ask Malcolm Gladwell (Blink)...

I f the pilot deployments of the various webmail interfaces internally had been reasonable (instead of awful experiences for the executives and other audiences who tried them), we would have been able to get more buy-in, more investment and with a bit of luck might have made comparable interfaces to the snappy Gmail that not only us but also our users and customers would have loved.

It is true that many things had to go right for this vision to have transpired but let's admit that we shot ourselves in the proverbial foot and also acknowledge that it was a failure of imagination and ultimately a failure of technical leadership.

Looking at today's opportunities and the various directions in which Lotus/IBM can go in, I only hope that we don't Get on the wrong Bus this time.

File under: technology, software, web, Lotus, notes, IBM, history, strategy, organizations, adoption, decisions, iNotes, K-station, WebSphere, portal, history, toli

Calling Things By Their Names

So I heard that sanity has returned to these parts. ^[1]

Our marketing folks are now going to call Lotus QuickPlace, QuickPlace instead of whatever bland and insipid name that had been previously mandated. IBM Lotus Team-Workplace-something or-the-other?

Similarly Sametime will now be called Sametime instead of "IBM Lotus Instant Messaging and Web Conferencing". For a while the Lotus in that designation had been deprecated.

This probably portends that our marketing teams now better understand how to sell the Lotus portfolio. A few years ago, the perception from the trenches where I live was that there was a rush away from the land of Lotus simply because (in my estimation) it wasn't understood or didn't mesh well with the compensation structure of our sales teams. Presumably we've adjusted and now understand what these products are capable of instead of dismissing them out of hand.

Now when belts tighten in the IT industry (and right now, it is a deep retrenchment), people fall back on things that work, whether it is the glue, spackle or wrenches that I keep referencing. And much in the Lotus portfolio works even if it isn't pretty sometimes. And it works especially in the SMB space that is in the Long Tail of software. See our recent acquisitions of more Glue Layer People or even my recent hand-waving thoughts on Lotus Notes:

"For those unfamiliar with Notes/Domino, my handwaving elevator pitch is that it is a platform essentially based on the fundamental insight that a huge class of applications can be built based on just a few compositional building blocks: Forms, Views and a standard file format, the note in Notes terms. The brouhahas made about messaging, security, directory services, and all that paraphernalia that marketing people throw about when they pitch the platform to you are all syntactic sugar around the core competency of Forms and Views and the client and server processes that can manage them. A whole cottage industry of business partners are doing very fine thank you building custom and evolvable applications for businesses, small and large, everywhere. The fact that email can be construed as a forms application is just a side benefit and detracts from the real focus of the platform. This is much misunderstood by people whose only encounter with Notes is as a Mail client. It's really just a forms and view app for people and processes. Incidentally this same platform is most likely what is funding my current work and much of the IBM Software Group, even as resources are spent on other "sanctioned" and more "strategic" approaches. C'est la vie."

So that's the good news.

The not so good news:

Coincidentally the [redacted] domain was removed from the internal dns without notice sometime in the last 2 weeks. I was wondering for a few days why I coudn't couldn't post to [redacted product] or vnc into my development servers from home. I thought it was a temporary flaw and added temporary entries to my hosts file. Of course I eventually realized that nothing in the internal [redacted] hierarchy would ever resolve again that this was part of the reason why [redacted product] moved away from [redacted domain]to its current location at [redacted new location]

Some of us who had to jump through hoops to get static ip addresses like [redacted] were a little disgruntled for a good 15 minutes. Mr Feinberg even brought Godwin's law into the mix albeit with a smiley at the end of that invocation.

I guess it was approriate that the hard drive for that twilight zone machine died 2 days later. Cosmic justice or Sign of the times. These are strange days...

Now this was probably just a case of run-of-the-mill miscommunication, but let's remember that simple things like naming bring out the tribal instinct in people with sometimes shocking reactions... A lot of goodwill could well be lost.

Take for example the time 5 years ago when it was not announced that Lotus Development Corporation had been deincorporated and a good 7,000 people suddenly received payslips from the IBM corporation saying that they had received only their thousand or so dollars year-to-date (and it was mid-year). Indeed it was only months later, after prompting by an atypically tough question in an Lotus all hands meeting and seeing the quite hurt and vociferous articulation of pent-up feeling that the question belied that this was acknowledged in a subsequent prickly corporate message. Perhaps we were being babies (after all we were still getting paid), why should we care about trivial things like the name on our paycheck? I didn't attend said meeting but I caught the fallout since I just became "reverse-mentor" to [redacted executive] and had to explain why such things would matter to us "[redacted] folks"...

The identity of a community is to be found in the most unlikely of things. The things that draw people together to form a cohesive whole are not the explicit things that one thinks, it's not a kind of warlike territoriality or dedication to some mission statement or other, it is rather in small insignificant items that the tribal instinct is articulated.

People will rally round the most surprising items (Elian Gonzalez, OJ Simpson, Teri Schiavo) and they quickly become litmus tests. Think of some of the most intractable political conflicts (say Serbia/Croatia/Bosnia or and here I hesitate, I wonder who else will have an Israel/Palestine category on Blogcentral) and wonder whether the political, religious or economic arguments were really the cause of the conflict or if there weren't instead some more mundane hurt feelings that then blossomed into what we have seen and subsequently rationalized by opportunist political or religious hacks.

Or take this example form my part of the woods: for the past 9 years, Northern Ghana has been in turmoil because of an argument at a market stall over a chicken or guinea fowl (the historians will have to figure this out at some point). Two ethnic groups, the Dagbons and the Kokonbas who have lived peacefully together for centuries are now in open dispute. Many, many lives have been lost, and a huge amount of money, goodwill, diplomacy, cajoling and outright bribing has had to to done to try to cool things down and to get that part of the country to begin to contribute again to the rest of the community. This in a place that can ill afford such distractions. Now in my analogy, the guinea fowl is the stand-in for the name or for a dns entry.

All this to say that when nurturing communities, it is best to keep such sensitivities in mind and call things by their names. If not all those Sensational Fruity Delights might come home to roost.

Editors note: I posted this piece on May 18 on my internal blog but now that the information is no longer a rumour but has received confirmation, I decided to remove the embargo since it speaks to the tribal instinct in communities, something that interests me as an anthropologist of social software

File under: language, naming, communities, tribes, patterns, best practices, software, history, strategy, Lotus, quickplace, sametime, notes, IBM, web, conflict, israel, palestine, Ghana, Dagbon, Kokonba, africa, chicken, Guinea fowl, anthropology, sociology, social software, technology, toli

On The Long Tail of Music, Metrics and Recommendations

Responding to Chris Anderson's Bring tha Noise post with some commentary, some metrics from my music collection and pointers to artists who inhabit this Long Tail of music...

• On The Long Tail of Music, Metrics and Recommendations
• Metrics From My Music Collection
• Full Force in the Long Tail

On The Long Tail of Music, Metrics and Recommendations

Some lunch break ego-surfing prompts this piece. Chris "Long Tail" Anderson tries to use real world data to validate some of his theorizing about music trends. Yours truly is one of the guinea pigs for this exercise in Bringing tha noise as he, and Public Enemy and Anthrax would have it. He's trying to figure out the signal to noise ratio in the fringe.

I seem to be a good source of metrics for this kind of thing because my tastes are not too mainstream and also because I augment my writing with lots of allusions and links to music and books that I champion or lean on to underline the slightest point. "Mining the cultural zeitgeist" is how I have described this tendency of mine when Jon Udell's metrics leaped out to me. I also tend to just use Amazon links which helps those seeking statistics. So then, what to say about the Long Tail of music?

First this: there's a difference between

1. the things that I write about, and quote or link to (i.e. what you'll see if you mined this blog)
2. the things I actually listen to or read
3. the things I've bought - my entire collection of music and books

I recently wrote a piece On Recommendation Systems that is in much the same area as this Long Tail discussion.

On the matter of recommendation systems, a couple of comments are in order.

• Leonard Richardson felt a little guilty that he hadn't spent time enhancing Ultra-Gleeper but that's the way things go, we're all stretched thin. Like he suggests, sometimes it's the idea that is important and not the embodiment in code, we see that all the time in technology. If one manages to give some intern somewhere an itch to scratch, as that paper should do, that is a contribution to knowledge and we are all the better for it.
• Robert Jamison seems to have a made a great contribution in ideas (Searching, Sharing and Stumbling), if not in code (kindakarma.com) to the recommendation systems debate and points to a few others of note. I will be playing with said code and services in coming weeks

But back to Anderson and Udell's search for metrics...

One thing I didn't comment on in Jon's post was that he was looking at statistics of bloggers he reads who most frequently cite books on Amazon.com. The problem was that many of the links from this site are about music rather than books. This may have skewed his statistics a little.

In other words, what was required was some sort of link classifier to figure out what a link refers to (whether book, music cd, dvd etc.). The classifier would be like a regexp pre-processor that would figure out that, for example, a link to IMDB is probably about a movie in much the same way as the LibraryLookup project maps ASINS to APACS. I assume All Consuming uses the Amazon api to figure out what type a given item is (book, music, dvd, other). I continue to wonder if one can have a classifier that will determine when a link points to a person... All Consuming is in flux as they transfer to 43 Things and their old REST apis are orphaned in the transition but that product has an interesting take on link classification and tagging.

Items 2 (what I actually listen to) and 3 (what I've purchased) in the above list are also interesting.

I have a partial pass at item 2, what I actually listen to. You can check out my statistics at Audioscrobbler. I installed Audioscrobbler on my home machine and have been playing with it to figure out if its recommendation engine will prove useful, I'll report back in a couple of months. After a couple of weeks, it has me in quite good company on the basis of the 769 tracks I've played on the home computer.

With a music collection as large as I have, my listening habits tend to be

• One third playlist-driven (I have numerous playlists for every mood as befits someone in the grip of musical obsession - thanks Gardner by the way for your kind comments and Autumn song playlist suggestions)
• One third shuffle serendipity
• The other third typically comes into play during shuffle; I'll hear something which will make me think of some other song and I'll do an ad-hoc thematic playlist, the problem is that I don't tend to save those tangential playlists borne of serendipity to feed back into the first category. I'm too lazy for that.

I'm a bit of an audiophile bigot and "can hear the difference" between original cds and even the 224 kbps VBR mp3 encodings I've done of my collection. Also some things have to be listened to loud (and one has to keep the neighbours on their toes every now and then). My computers haven't been hooked up to the big speakers and gleaming stereo system; the computer speakers are some generic $30 muffled fuzz boxes. Thus I need to get a Squeezebox, Audiotron or something although I'm a late adopter in such things. Thus, what is missing in the Audioscrobbler statistics is the music I listen to on my stereo. Still Audioscrobbler is a good proxy for much of my actual consumption of music.

On the topic of playlists, I'll be sharing some shortly so that others can get a sense of my beat matching juxtaposition insanity.

One problem with dealing with statistics from the blog is that for example, even though I'd estimate that 40 percent of my collection is jazz, I write about jazz only occasionally, like when I was Vibing with Abbey Lincoln or reminiscing about A Soul Jazz Thing with a belated appreciation of the late Jimmy Smith, or say alluding to Kamal (of The Roots) and his Ahmad Jamal keyboard stylings. There has only been one "proper posting" on jazz otherwise the allusions are all incidental say comparing Miles Davis's First Quintet to Rokia Traoré's band.

Jazz is less quotable and difficult to write about so perhaps that is why it wouldn't show up more readily in the statistics from the blog. Still the jazz idiom is a big part of my musical taste and outlook on life.

Metrics From My Music Collection

Anyway here is the data for number 3: I give you my music collection for your forensic analyses.

collection.m3u and collection.pls

My digitized music collection stands at 10,346 songs and amounts to 70.8 GB (about 36 days of continuous music). Note that this is larger than the largest iPod (hence my pause in not adopting that ubiquitous platform). I only ripped things that I liked so presumably one should add a good 50 percent filler for a fuller picture. I've been revisiting that notion since the Best Left Unread piece with a newfound appreciation of the bad things in my collection; disk space is cheap and I've just ordered a 250 GB spare hard drive to mitigate future disasters, that disk could be put to work.

Per contra I actually believe human psyches require a little imperfection.

Of the 1,527 artists therein, there were about 15 that were problematic (e.g. I'm sure there's no musician called Hawaii 5.0 or James Bond 007, to take some of the theme songs I occasionally play to spice things up, and similarly Ghana Highlife 1 to 10 are simply proxies for the unknowns I've picked up). I haven't digitized the 4 crates of records that I have lying about. That's a weekend project for this summer like my photo-digitization project of a few months ago (see Cultural Sensitivity in Technology)

Apparently there are 1,540 albums in the collection. I haven't quite diligent about tagging albums so 2,000 songs don't have album information. Handwaving a 10 song per album ratio that's another 200 albums, so lets say 1,750 albums). I'll fix that and update those files in the next few days.

Generating metadata for albums is difficult: what album would 50 Cent's blistering Jay-Z diss that was the talk of mixtapes in 2002 fall in? And there are also things that come from Greatest Hits, or Live albums as opposed to the original album, and I typically have multiple versions of things. I can't remember which version I actually ripped.

I am actually crazy to have spent cash money on most of these albums instead of downloading so if you assume that's $14,000 to $27982.50 (at $8 per album or $15.99 if I were to pay what record companies would prefer). Luckily I got a lot of these for free when I used to DJ and was on record company mailing lists, also I used Columbia House and BMG music zealously. Still that's a lot of money, I bought my first cd in 1992, previously it was all vinyl and tapes. I can't say that I've downloaded much from file sharing networks (I use Gnucleus). At the higher end of the scale of the cost estimates, $30,000 is a mortgage downpayment (well it was before this insane real-estate bubble that even leads to riots). Music companies should love me. I justify it by saying that my only hobbies are music and books; no drink, fast cars (I'm a public transport prole) or other flash.

I've tagged songs by genre but here you really want to be able to specify multiple tags just like in any of the new social software.

I suppose I should switch to iTunes to get an xml representation of these playlists. One of the bad versions of Winamp, the bloated one that which caused a backlash, used to export playlists in an xml format. That would have been better than m3u or pls. Still that is nothing that a capable regexp and Unix pipeline wizardry couldn't fix.

I'm still in disaster recovery mode but once I've got Cygwin set up, I'll be sure to download and install PlaylistManager and slice and dice the data with that Linux tool.

Anyway I hope this helps...

Full Force in the Long Tail

The obligatory musical note... While writing this note, I just rediscovered the wonderfully fresh sounds of Adriana Evans whose 1995 album for some reason didn't blow up (stupid record companies). Adriana Evans is the name that you drop when you want to put someone talking up Ashanti or Brandy in their place. Love is all Around and Seein' is Believing are some of the most laidback soul jams of all time.

Speaking of Seein' is Believin', how about Cheryl "Pepsii" Riley's (erstwhile of Thanks for my Child fame) song of the same name from 1988's Me, Myself and I album, produced by Full Force. That song is soul/funk perfection, much like Jerome Prister's sublime Say You'll Be that I recently found after 18 years.

And speaking of Full Force, what about that Hall of Fame production crew and House Party funkers? In recent years have been souping things up for Britney and Christine (you've got to pay bills). Their roots, like Jam and Lewis, are in pure soul and a hip-hop/funk aesthetic. Guess who's coming to the crib? is my favourite album of theirs although others preferred their eponymous debut album

My picks in the Full Force canon

• Your Love Is So Def
  My dad lost all respect for me when I used to sing this because he heard "deaf" in my loud singing rather than the 80s hip-hop notion of def.
• Love Is For Suckers (Like You And Me)
  Shower crooners at school couldn't stop singing this song for weeks on end. Not to mention the lines we all primed to repeat on cue, their irresistible verbal tics:

  "Come on bust a move."
  "Full Force Get Busy One Time"

• Take Care Of Homework

  "Yo homeboy, your girl looking kinda fly lately, know what I'm saying?"
  "No I don't know what you're saying."
  "What I'm saying is that she be like checking me out 'cos you been neglecting. D'you know what I mean?"
  "No I don't know what you mean!"
  "What I mean is that you better take care of homework or a dude like me will push up and take your place!"

  On the basis of the irreversibly funky break beat from Take Care of Homework, these brothers proved they were funky enough to revamp James Brown in his sadly neglected I'm Real album. The title track is a ferocious call to arms by The Godfather.
  
  
  
• I'd also note that their Smoove album is no spring chicken. It contains such gems as Ain't My Type Of Hype, the Smooved out title track and that phenomenal ballad, Kiss Those Lips. These are cases in point about the versatility of the crew
• They gained fame with Alice, I Want You Just For Me with the famous opening.

  Testing, Testing One, Two
  Testing, One Two
  In The Place To Be...
  Girl I Want To Shower You With Diamonds And Pearls
  And When Were All Alone I'll Take You For A Trip Around The World
  Yes Indeed I Like Your Style
  Ooh You're Worth My While
  Baby, I'm Your Carpenter,
  Please Me Lay Your Tile
  I Don't Want To Share You With No Else
  Alice Be My Girl
  Can't You See?
  I Want You Just For Me?
  Full Force Get Busy One Time

• Their repeated collaborations with Lisa Lisa & Cult Jam are also essential Spanish Fly.
  
  
  
  
  
  I Wonder If I Take You Home will still fill a dancefloor to this day.

  "Would You Still Be In Love, Baby?
  Because I Need You Tonight."

• Their work with UTFO especially on Roxanne, Roxanne also gave rise to the legendary The Complete Story of Roxanne, those 103 responses to UTFO's 1985 novelty hit.
  
  The Story of Roxanne is one of the best stories of of the Long Tail of Music (the other great case in point is All Roads Lead to Apache).
  
  Roxanne, Roxanne brought to prominence The Real Roxanne and more importantly Roxanne Shante, possibly the best female MC of all time, Roxanne's Revenge will go down as the best response record of all time. One of my favourite songs of hers is Live on Stage which is Marley Marl's production at its best.
  
  
  
• Full Force consolidated their fame with collaborations with Kid 'N Play in House Party and its two sequels. The movies were the inspired brainchild of the Hudlin Brothers, a decade before Barbershop.
  
  
  
  
  
  
  
  
  
  Kid 'N Play were no slouches themselves even if they tended towards the pop side of the hip-hop spectrum. They were fun, their beats and lyrics focused on putting a groove in your step. More to the point, they could dance; at a time when Hip Hop was heading into inaccessible navel-gazing, they reminded everyone about the dance in Kool Herc's dancehall that was in the founding mythology of the music. As historians of Hip-Hop would write, break dancing (the B Boying and B Girling aesthetic) is an essential element - some would even term it the most important of the 4 elements of hip-hop, even more than MCing (rapping), DJing, and that maligned "social nuisance" element, Graffiti. Gittin Funky and Rollin' with Kid 'N Play will cause circles to form and Soul Train breakouts on the dancefloor. Funhouse is their best album and that's the spirit. And those flat tops!

Full Force have since cashed out of the soul side of things except to give some "urban" substance to blue-eyed pop, and who can blame them: "black" audiences are so fickle.

File under: technology, design, architecture, software, web, systems, social software, complexity, services, music, soul, funk, jazz, hip-hop, glue, recommendations, algorithms, heuristics, machine-learning, long tail, Full Force, Adriana Evans, Lisa Lisa, Cheryl "Pepsii" Riley, James Brown, toli