Never argue with a rozzer

In all the recent blogging around the ‘Respect Agenda’ (Justin and TP have been particularly good), one point that hasn’t been made is that all this is nothing new. Or rather, it’s nothing new to New Labour. This government has passed huge amounts of law-and-order legislation, much of which has been devoted to taking responsibilities away from the courts and giving them to the police. (And to Community Support Officers, but that’s another story.) Here are some of the highlights.

When this government came in, there was a fairly clear distinctions between charge, arrest and caution. A suspect could be charged, with a view to subsequently issuing a court summons; this was the standard procedure for crimes attracting a penalty of less than five years’ imprisonment at the first offence. In some situations a suspected offender could be arrested pending prosecution: this option was available for more serious crimes and for the prevention of a breach of the peace, as well as for the purpose of enabling a summons to be served (for instance, if a suspect attempted to abscond or refused to supply a valid name and address). Finally, a suspect who was charged with and admitted an offence could be ‘let off with a caution’ in lieu of court proceedings. A simple caution is not a conviction and does not carry any penalty; it does, however, represent an admission of guilt and remains on the offender’s criminal record for five years.

Pretty much all of this has changed. Successive pieces of legislation passed since 1997 have classed a number of less serious crimes as ‘arrestable’, particularly in the area of public order; the 2003 Criminal Justice Act clarifies the increasingly arbitrary boundary between arrestable and non-arrestable offences by the simple expedient of making all offences arrestable.

Since 1998 young offenders are no longer cautioned, but given a ‘reprimand’ at a first offence and a ‘final warning’ at a subsequent offence. A warning will generally be coupled with a referral to the local Youth Offending Team (YOT), who will be charged with developing a programme of activities to address the offender’s behaviour; in some cases a reprimand will also include a YOT referral. While a YOT programme is not a criminal penalty and is not compulsory, the effect is to couple a police caution with an official sanction. This principle is followed by the recent introduction of the ‘conditional caution’ for adult offenders: a caution may be coupled with a programme of restitutive or rehabilitative activity. If the offender does not comply with the programme, a prosecution for the original offence may follow.

The 2001 Criminal Justice and Police Act introduced penalty notices for disorder (PNDs): a type of fixed penalty notice (FPN). An FPN — previously used primarily for traffic offences — is not a penalty for the offence. Rather, the recipient is served notice that he or she may be prosecuted for the offence, but that the liability can be discharged by paying a set fine. PNDs are given primarily for drunk and disorderly behaviour. A 2004 study of a pilot scheme suggested that between half and three-quarters of PND recipients were ‘new business’, i.e. individuals who wouldn’t otherwise have been cautioned, arrested or charged.

The use of ‘vanilla’ FPNs has also been extended. The 2002 Police Reform Act introduced a range of minor offences, generally associated with ‘anti-social behaviour’, for which FPNs can be issued by locally-accredited Community Support Officers as well as by police officers. The range of offences involved has subsequently expanded — under the 2002 Act and by provisions in the 2003 Anti-Social Behaviour Act — from three to 20. Community Support Officers (who go out on the beat after six weeks’ training) have no power of arrest, but can detain a suspect until the police arrive. There has been talk of empowering CSOs to escort truanting children back to school; the power to escort adults to the local nick is probably not far behind.

ASBOs have been around since 1998. An anti-social behaviour order (ASBO) is a court order, which must be obtained from a magistrate (and may be requested by a range of agencies other than police forces – a range which looks set to expand). An ASBO is an injunction to refrain from specified activities, which can be obtained on the grounds that the offender has engaged in these activities as part of a pattern of ‘anti-social behaviour’. The criminal sanctions associated with ASBOs relate to the offence of breaching a court order, rather than to the actions involved. These actions may in themselves be entirely legal; they may not even be ‘anti-social’ if carried out in other circumstances, in other locations or by other individuals. (ASBO provisions have been used to bar individuals from riding bicycles, wearing gloves, etc.)

Then there are measures relating to property. The 2002 Proceeds of Crime Act relates to the confiscation of assets gained through crime or used for criminal purposes; it enables the courts to seize the property of suspects committed for trial in a crown court (not, necessarily, convicted offenders). There is also a provision in the act for the immediate seizure of cash which a police officer believes to be crime-related. The threshold for this type of seizure was set in the act at £10,000; it was subsequently lowered to £5,000 and is about to be brought down again to £1,000. (That’s inflation for you.) The 2003 Anti-Social Behaviour Act, finally, enables the police to obtain a court order closing and sealing premises which they believe to be used for drug offences (the ‘crack house’ provision). There is no requirement to prove that drug offences have taken place at the premises, or been committed by anyone using the premises. This is the provision which the government is planning to extend to cover noisy neighbours.

Some of these interventions relate to behaviour which is not in itself criminal; others broaden the range of criminal offences which are in practice sanctioned, or heighten the sanction applied (as in the case of the extension of powers of arrests). What all these measures have in common is that they erode the distinction between police intervention and penal sanction.

I don’t think there’s any grand plan behind these developments. The police forces of England and Wales are an institution, and like all institutions they would rather have more power than less. What’s extraordinary is the government under which this has happened. Back in the 1980s (hey, you young people…) scary people like Peter Bruinvels would get up at the Tory Conference and talk about giving the police the support and the resources they need – and Thatcher herself would let it be known that she thought James Anderton had it about right – but things never changed all that much. (At any rate, things never changed this much.) Perhaps this was because there was a good solid layer of permanent-government bureaucracy between the Thatcher circle and the places where laws were drafted; perhaps it was because Thatcher faced a Labour opposition. Neither condition obtains now. Or perhaps – saddest thought of all – the reason why Thatcher didn’t give the police lobby everything it wanted was simply that she had ideas of her own.

When I was seventeen

…it wasn’t a very good year. Eighteen wasn’t bad: when I was eighteen I passed my Cambridge entrance exam, got my first editing gig (a church youth newsletter), went abroad for the first time and did my first full-time job; the last of those doesn’t really qualify as good (I was a nursing assistant in a long-term psychiatric hospital) but I’m glad I went through it. (There was also a woman I was quite close to for a while, but I’m not going into details about that.)

Seventeen, though… There was under-age drinking, there were A levels, there was self-importance, frustration and bad poetry, but beyond that I don’t remember much. Apart from the orchestral concert where I played, with no one I knew in the audience and no one to talk to backstage, and where the flute part was doubled (and largely drowned out) by the second B-flat clarinet. And the party a girl called Liz invited me to – well, not the party, but the evening when I stood on her doorstep, holding a bottle of cider and a borrowed copy of Starless and Bible Black, and learned that the party had been the previous night. I have to say the music was good that year – punk had happened while I was 16, and I’d got into it around the time of “Pretty Vacant” (i.e. rather late). I didn’t get to any gigs (100 Club? I never even made it to the Greyhound), but it was bliss in that dawn to listen to John Peel and buy singles from Bonaparte’s, that I can say.

So 1978 wasn’t all bad – particularly since I’d turned 18 by the end of it. Maybe in a few years’ time, with a bit of distance, I’ll be able to be that positive about 2005. Right now it looks like a pretty awful year.

Before I go any further: this post was inspired by an email from a coblogger which ended “May 2006 be better”. When I read those words I had a frisson of alarm – I haven’t talked to him about all that, have I? Well, no, I hadn’t – and we’ve got plenty of shared, public reasons for hoping for a better new year – but the exchange reminded me that I’d wanted for a while to put some more personal content up here. This is one result: 2005, my (first) year as a blogger.

Or rather, my first nine months. It all began in March. I started transcribing Sir Frederick‘s memoirs some time ago, but I’d never seen the need for a blog of my own. In March 2005, though, Need to Know led me to Backing Blair, which in turn led me (from the sublime…) to comments threads on Tom Watson’s site. The arguments being advanced, notably by Tom, against voting for anyone but Labour seemed so faulty, in so many ways, that I immediately felt that a series of posts would be required to answer them. So, over the next couple of months, I wrote a series of posts, pausing only to set up a blog. (The address of this blog confused people for months. What happened was that my browser crashed during the setup process, and by the time I got back in Blogger had flagged ‘’ as taken. Now it can be told.)

But we’re in May already. Let’s get back to March. On Monday the 14th I wrote the first For Tomorrow post (most of which has since been eaten by Blogger, annoyingly enough) and mailed a couple of people about it, hoping to get noticed and generally join the conversation. On Wednesday the 16th I set up my work blog, posted something on it about taxonomies & knowledge representation, and mailed a couple of people about it, hoping to work the same trick in a professional context. All exciting stuff, reminiscent of my early days on Usenet (circa 1996) – will I get followups? will I get followups from the regulars? oh no, I’ve written something really embarrassing, let’s hope nobody notices… hey, I’ve written something really great, let’s hope somebody notices… nonononono, what I thought was really great was actually really embarrassing, let’s hope nobody’s noticed already All exciting stuff, and liable to cause heightened states of anxiety if taken too seriously.

On the evening of Wednesday the 16th I had a migraine – not my first, but the first I’d had in a few years. When the aura had cleared I lay down to wait out the headache, and found I was consumed with anxiety – about the wretched blogs. Was I being read? Was I being linked to? Who was linking to me? Was I not being linked to? Wasn’t that even worse? What could I do about it? Urgh. I eventually got to sleep, resolving to leave the damn blogs alone for a bit.

The following evening, on my way home from the chip shop after my kids’ school disco, I hit a problem with my health. What felt like an innocent fart turned into something copious and very wet. When I got home I made two discoveries. One was that I hadn’t, in fact, crapped myself; what I’d passed was blood. The second was that I urgently needed to go again.

I hung on at home for a while, hoping that it would stop. It didn’t; it gave me about ten minutes and then happened again. And again. Fortunately the nearest hospital was only ten minutes away by taxi. I may have missed one or two, but I think by midnight I’d had a total of ten bleeds. On the tenth I passed out; I remember a horrible moment when I was trying to stay focused, by telling myself who I was and where I lived, but all I could think of was the address of the house where I grew up. The next thing I knew, I was lying flat with an oxygen mask on. (Great stuff, oxygen. I can recommend it.) There weren’t any more bleeds after that, thankfully. I remember a nurse helping me into a bed in a darkened ward, and telling her that I had an important meeting in the morning. “Not now, you haven’t,” she said.

Hospital life was what it always is – tedious, long-drawn-out and hideously uncertain. (On many levels. While I was in hospital we had a new TV delivered; at one stage I was seriously, genuinely worried that I was going to start bleeding again, bleed to death and never see our new TV…) What eventually happened was that I stayed in until I was strong enough to go home (yay, new TV!), then went back in for a colonoscopy. Which was clear; the betting was that the bleed had been triggered by a randomly malformed vein (a.k.a. angiodysplasia coli) and quite probably wouldn’t happen again. Although it might. It hasn’t yet, but I still worry that it’s going to, about two or three times a day.

April was a bit busier on the blogging front – 14 posts (compared to 6 in March), including parts 2-8 of the For Tomorrow series and a sceptical post about Islam and radical politics. The pre-election daily blog roundup which eventually developed into the Sharpener got going this month, as did Tim Worstall’s Britblog roundup; the short pre-election post which Tim picked for 2005: Blogged also dates from this period. Also in April, my wife’s mother (80ish, living alone) had a fall and couldn’t get up. She had to go into hospital, where she became extremely confused and developed a series of infections.

Even more posts (17) in May, including one about Christianity and conservative politics (not so much sceptical as furious). Mostly about the election, but there were a couple each about the EU constitution and the legality of the Iraq war. Plus a wave to an old college acquaintance, a plug for Ellis Sharp’s blog and my first piece at the Sharpener. I had some energy back then…

June: 11 posts. Iraq, terrorism, Blairite triangulation, plus a couple of ‘memes’ and a bit of inconsequential chat (about time too).

Only 9 posts in July, mostly about terror and counter-terror – including this, one of the posts I’m most proud of. I was in a combative mood that month: I disagreed with Norm of that blog, agreed with Oliver Kamm, sniped at Harry and misquoted Nick Cohen, whose honour was defended vociferously by some drink-soaked Trots. (I almost miss them – they brought a lot of traffic with them.) My first post that month was dated 12th July. On the 5th I’d heard that my doctorate had been awarded (six and a half years after I began studying for it and eighteen months after I first submitted the thesis). On the 7th, well, you know. On the 8th, my wife was told that her mother – whose condition had picked up, although she was still quite weak – was much more ill than anyone had thought, and that we only had weeks or months left.

August: 8. Terrorism, de Menezes and a history of the House of Lords. Not very light and frothy. At work my contract ended; fortunately I had a new one lined up. We went on holiday, regretting not having got cancellation insurance. We didn’t have to cancel.

September: 7. Four on Hurricane Katrina, two on Walter Wolfgang and one proposing to fix Wikipedia – whose shortcomings weren’t all over the media at the time, I might add – with a kind of Pledgebank posse comitatus. Maybe this year I’ll actually get something moving on this one. (Note to the Guardian journalist who I mailed about the Wolfgang incident: I’m not a lawyer either, but I know how to google up the text of legislation.) My mother-in-law moved to a nursing home, and started to get a lot more alert. I junked my ten-year-old Windows 98 PC and switched to an iMac.

Only three posts in October and two in November. More de Menezes, plus Bob Dylan, Guy Debord and the destruction of the Labour and Conservative parties by their respective leaders. Five in December: C.S. Lewis, Chomsky, Chavez, poetry and the ‘infantile’. I’ve been posting all over the show, basically, and not very often. What happened? One thing that happened was that I started my third contract of the year (in the same place and very much the same job); this one lasts until January 2007, after which I’m hoping to get something a bit more permanent. It’s four and a half days a week, which is longer hours than I’ve worked since 1998 – a bit of a shock to the system, and cuts down on blogging time. Plus I started playing the flute a bit more seriously around this time (jigs, reels and the odd hornpipe).

But the main thing that happened in October was that my mother (80ish, living alone) had a stroke. We don’t know – and never will – how long it had happened before she was found; my guess is 24 hours. She was taken to hospital, where I saw her soon afterwards; she wasn’t speaking at all, had no use of her right-hand side and seemed very confused. In November she was moved to a specialist rehabilitation unit; she got the use of her right side back and started talking a bit. She clearly wanted to get up and walk, but she was very weak and unsteady, and still seemed very confused.

The rehab specialists seem rapidly to have formed the view that she wasn’t getting anywhere. In December they told me and my sisters that she was going to require permanent nursing care; what this meant was there wasn’t any point their trying to get her up to a level where she’d be able to look after herself, as that was never going to happen. On Christmas Eve I heard that she was moving into a nursing home – which she did, between Christmas and New Year. The council will cover the fees (which are high) for twelve weeks; after that we have to start paying, which essentially means selling the house. It’s not our family home – my parents moved there about twenty years ago, after my father retired, by which time we’d all left home – but it’s got a lot of history and a lot of memories; it’s been part of a continuing story, which suddenly isn’t continuing any more. More than that, the house symbolises the basic level of independence which lets you get by day to day – having things to do, going out and coming home, being surrounded by your stuff. My mother’s now lost that; what’s worse, some time this year my sisters and I are going to have to administer the coup de grace by stripping the house. My wife and her brother had already finished stripping their mother’s house by this time; I said at the time that they’d be in trouble if she made a miracle recovery. (That doesn’t seem likely, although she has hung on for six months so far.)

My mother’s a lot more mobile since going into the home, and a lot more articulate – I’ve actually talked to her on the phone. But there’s still something not there. I think short term memory’s like a tape loop or a cogwheel – we’re never simply in the moment (unless we’re meditating or drunk), we’re always checking back on ourselves, saying “right, I’m here, I’ve been there and next I’m going to do this“. You get on with it for a while, then you stop and check again… and so it goes on. It’s something like how I imagine it must be to skate: your body reacts automatically – reflexively – to bumps and gritty patches and gusts of wind, but your ability to stay balanced enables you to contain those reactions and stay poised. Not to blank out the reflex responses or even override them, necessarily – just to incorporate them into a kind of continuing dance with gravity. At the most basic level, I think it’s that self-correcting – or self-articulating – mechanism that my mother’s lost, probably for good: the stimuli come in and she reacts. And, er, that’s it. (May be updated; I’m seeing her on Friday, for the first time since she went into the home.)

Skating through 2005 has been hard. I hope 2006 will be a bit easier.

Could do better

“Yes, I know, I know. 2005′s over already, we’re more than halfway through the decade… you’re not the only one who’s embarrassed, how do you think I feel? Yes, I know it was supposed to be all m-computing and always-on GPS and ubiquitous wifi jetpacks by now, but it’s been difficult, I mean, look at the state of the economy… OK, OK, I shouldn’t have said ‘jetpacks’, forget I said that, no, I am taking this seriously, really I am… but come on, apart from anything else there’s been this war going on, that hasn’t helped… yes, I know military spending is supposed to be a major driver of high-tech R&D… maybe it just isn’t driving R&D the way we’d like it to, have you thought of that? look at Segways, they were going to be the next big thing at one stage… OK, OK, I’ll stop trying to change the subject… Look, what can I say? We’ll do better this year. I’ll do better this year. Trust me. OK? OK.”

- Web promises to become more pervasive in 2006

Soft enough for you


Is it reasonable to have to learn to ride a bike but expect a computer to be as simple to figure out as a toaster? (Not the perfect analogy I know, but you know what I’m getting at…) Some days I think that user-friendliness was/is a really bad idea, not least because it’s obdurate, so hard to change.

If you have to work at using a technology, in other words, you necessarily end up working with it and through it. You work to adapt it to your needs – and you adapt it. Technologies which offer ease of use, by contrast, make it easy to work in certain pre-defined ways – and resist adaptation by the individual user. (There are, of course, technologies which are both easy to use and flexible – ask any Flickr user. But I think the ‘user-friendliness’ Anne is talking about here is more like the comment a tutor of mine once made on the BBC and ‘open access’ broadcasting: “They say they’ll come and help you, show you how to do it. They don’t, of course – what they do is show you how to do what you do because that’s how you do it.” User-friendliness is very often a matter of HTDWYDBTHYDI.)

But there’s more to it than that. What is this thing called obduracy? Anne again:

[Anique Hommels] argues that one way to emphasise the material aspects [of technologies in society] is to focus on their obduracy or resistance to change. (Imagine what it would *actually* take to replace the infrastructure that currently provides our electricity with something more sustainable.) The notion of obduracy is inextricably connected to embeddedness – a matter of interest to any kind of computing that seeks to become part of something else, be it an event, a habit, a skirt, a chair, a building, a street, a city. As Hommels reminds us, obduracy (or embeddedness) is a relational concept: “Because the elements of a network are closely interrelated, the changing of one element requires the adaptation of other elements. The extent to which an artifact has become embedded determines its resistance to efforts aimed at changing it.”

An embedded technology, then, would be one which has behind it a community of people who do a certain thing in a certain way. Becoming a user entails enrolment in that community. In short, the technology adapts you.

Where does this leave user-friendliness? Perhaps we could think of the embedding of a new technology as a process, which can continue to the point of the collapse of the possible ends and uses inherent in the technology and its reduction to the status of tool: a toaster, not a bicycle. And perhaps a ‘user-friendly’ technology – at least in the HTDWYD sense – is one designed to enlist a tool-using community and collapse its own potential into instrumentality.

(Relatedly, from Dan Hills’ essential critique of digital music: “there is a powerful necessity to think long term; to not take such short cuts which may inadvertently delete possible outcomes; to enable the flexibility and endless modifications seen in previous generations of music devices”. Dan has a lovely quote from William Gibson: “That which is overdesigned, too highly specific, anticipates outcome; the anticipation of outcome guarantees, if not failure, the absence of grace.”)

More broadly, what all this highlights is the value of difficulty, incompatibility, misunderstanding. Dan also led me (indirectly) to this quote from the late Derek Bailey:

There has to be some degree, not just of unfamiliarity, but incompatibility [with a partner]. Otherwise, what are you improvising for? What are you improvising with or around? You’ve got to find somewhere where you can work. If there are no difficulties, it seems to me that there’s pretty much no point in playing. I find that the things that excite me are trying to make something work. And when it does work, it’s the most fantastic thing.

One of the great frustrations in my work with ontologies and e-social science is the recurrent assumption that the concepts used in social science data can be documented cleanly and consistently – or, conversely, that if they can’t be documented cleanly and consistently they’re not worth documenting. The point, surely, is to find ways of recording both the logic of individual classifications and the incompatibilities between them – and the (qualified, partial) correspondences between them. And, of course, to make this documentation changeable over time, without effacing the historical traces which contribute to its meaning. Parenthetically, it’s worth noting here that preservation of historical data has nothing to do with obduracy. History is not obdurate, having no power to resist and (by and large) no enrolled community; the erasure of history can facilitate embeddedness and instrumentality, while the preservation of an artifact’s history may actually preserve resources of flexibility. (That’s enough abstractions – Ed.)

We’re never together

Back here, I wrote:

Social software may start with connecting data, but what it’s really about is connecting people – and connecting them in dialogue, on a basis of equality. If this goal gets lost, joining the dots may do more harm than good.

It’s not about connecting machines, either – and the same caveat applies. Via Thomas, I recently read this item about location-based services (which, I remember, were going to be quite the thing a couple of years ago, although they seem to have faded since people started actually getting their hands on 3G technology). Anyway, here are the quotes:

This project focuses on [location-based technology's] collaborative uses: how group of people benefits from knowing others’ whereabouts when working together on a joint activity … we set up a collaborative mobile environment called CatchBob! in which we will test how a location awareness tool modifies the group interactions and communications, the way they perform a joint task as well as how they rely on this spatial information to coordinate.

And how did that work out?

“We found that players who were automatically aware of their partners’ location did not perform the task better than other participants. In addition, they communicated less and had troubles reminding their partners’ whereabouts (which was surprising). These results can be explained by the messages exchanged. First the amount of messages is more important in the group without the location-awareness tool: players had then more traces to rely on in order to recall the others’ trails. And when we look at the content, we see that players without the location-awareness tool sent more messages about position, direction or strategy. They also wrote more questions.”

Really, we’re back with ‘push’ technology – which was going to be quite the thing round about 1998, as I remember. Give people device for talking to each other: works. Give people device which gives them a constant stream of information: doesn’t work.

The trouble is, we’ve got the technology. The problems with social software are social; see this deeply depressing Register story.

Alongside video on demand TV services from Homechoice, the SDB [Shoreditch Digital Bridge] will offer a “Community Safety Channel” which will allow residents “to monitor estate CCTV cameras from their own living rooms, view a ‘Usual Suspects’ ASBO line up, and receive live community safety alerts.”

Other aspects of the Shoreditch Digital Bridge are less controversial, but likely to be considerably harder to execute. The SDB proposes an education channel, “allowing children and adults to take classes, complete on-line homework assignments and log-on to ‘virtual tutors’”, a “Health Channel” allowing patients to book GP appointments, and providing “virtual Dr/Nurse consultations and on-line health and diagnosis information”, a “Consumer Channel, allowing on-line group buying of common services such as gas, electricity and mobile phone tariffs”, and an “Employment Channel, providing on-line NVQ courses, local jobs website and virtual interview mentoring.”So within that little lot, the educational aspects will require substantial input from, and involvement of, existing schools and colleges, the Health Channel will need a whole new interface to NHS systems that are already struggling to implement their own new electronic booking systems, and the Consumer Channel will merely have to reinvent the co-operative movement electronically.

But CCTV – ah, now, we’ve got CCTV…


Yet again, the technology arrives promising us a vibrant civic and economic future … then beds down as a means of protecting us from each other.

Or rather, as a means of protecting us from Them (caution – sweary link).

If we’re talking about social software or social networks, let’s be clear that we’re talking about connecting people rather than dividing them. Connecting machines doesn’t necessarily help connect people.

Silence and screams

In a recent post at The Sharpener, I mirrored Craig Murray’s telegrams on the British government’s attitude to the situation in Uzbekistan, with particular reference to torture and intelligence obtained through torture. I also put down some thoughts about the Uzbekistan story, which had been bothering me for a while. On one hand, the Foreign Office appeared to attach great importance to Uzbek intelligence, even if it was produced by torture – to the point where the message the British government was sending Uzbekistan was surely that torture was OK, as long as it produced more of the same. On the other, I didn’t (and don’t) see any reason to doubt Murray’s assessment of the ‘intelligence’ produced by Uzbek government thugs as ‘dross’. But if this was the case the British government’s position was both morally dubious and – perhaps more importantly – logically incomprehensible. As I wrote:

“the argument that the torture of Uzbek detainees can sometimes be justified, because it sometimes produces useful information, directly promotes the continuing torture of many other Uzbek detainees – most of whom have nothing ‘useful’ to say to any spook, and all of whom will say anything to make the torture stop.”

Following the argument to its logical conclusion, and feeling ever more like a conspiracy theorist, I added:

“And perhaps that’s what the argument is meant for. Look at it this way: Murray’s argument that the information produced by the Uzbek torturers is ‘dross’ is logically compelling and founded on first- and second-person experience (i.e. he’d talked to torture victims). If MI6 genuinely believe that the information is ‘useful’, either they have better information than Murray (which seems unlikely – ‘MI6 have no operative within a thousand miles of me’) or they’re extraordinarily dim. If we dismiss both possibilities, we must assume that they know as well as Murray that the information’s no good – and that information isn’t what all this is about. Perhaps the fundamental, non-negotiable starting-point isn’t the War on Terror but the Uzbek government itself, and its alliance with the US and Britain.”

Murray has subsequently written a very interesting comment on Brian Barder’s blog. From which:

In two years of seeing this Uzbek intelligence material, I never saw a single piece that even purported to concern a threat to the UK, or indeed to the West. Everything we were given was, without exception, designed to convey the impression that all the Uzbek opposition were Islamic terrorist and linked to al-Qaida (which is very far from the truth).If the material had been about threats in Birmingham, certainly the analysts would have been better placed than I to evaluate it. But as it was about Central Asia, I don’t accept that people who had never even visited Uzbekistan were in a better position than I to evaluate.

What I found particularly chilling were instances where such intelligence was being deliberately accepted or interpreted, in order to justify continuing US support to this odious regime. The US was justifying its presence and policy in Uzbekistan by the common threat faced, and prepared to buy fictions that reinforced that threat as part of the raison d’etre of the War on Terror.

So when I call the intelligence dross, I really mean that. It wasn’t just not useful to the UK, it was positively and deliberately misleading.

Perhaps ‘selling our souls for dross’ is an emotive phrase which didn’t belong in diplomatic correspondence. Perhaps Murray should have acknowledged that there might be broader international considerations which, in the view of the Foreign Office, justified or even mandated conniving in Uzbek torture. But that’s as far as I can go in meeting Murray’s critics halfway. The situation he describes is morally disgusting and of borderline legality; it’s also strategically myopic, answering more to Britain’s current subordinate role within Bush’s neo-conservative alliance than to any longer-term assessment of the prospects for Central Asia. As such, it sets a dangerous precedent and should end.

With no fear of attack

Thanks to Talk Politics, I’ve recently read – or at least glanced at – some remarks made by Hugo Chavez, Constitutional President of the Bolivarian Republic of Venezuela, on Christmas Eve 2005. Here’s the passage which has excited most comment (my translation).

I became a rebel and I dedicated myself to the true Christ – and this is the true Christ, I have no doubt about it. He is not that idiotic image with a stupid face that you can see in some churches, as if he were an idiot. No, Christ was and is one of the greatest revolutionaries in history and the first socialist of our era – the first socialist, and for that they crucified him.

There is enough water in the world for all of us to have water; there are enough lands, enough natural riches in the world to produce food for the whole population of the world; there is enough stone in the world and enough building materials to ensure that nobody is without a home. The world has enough for everyone, but now a few minorities, the descendants of the people who crucified Jesus, the descendants of the people who threw Bolivar out of here and crucified him in his turn, in Santa Marta over in Colombia… a minority has taken charge of the riches of the world, a minority has taken charge of the world’s gold, silver, minerals, water, good land, oil, all its wealth, and it has concentrated that wealth in a few hands. Less than ten per cent of the world’s population has charge of more than half of the wealth of the whole world. More than half of the world’s people are poor and every day there are more poor people in the world. We, here, are resolved to change history, and every day we are joined and will be joined by more heads of state, presidents and leaders. Look at how the Bolivian people… Bolivia, the poorest country in South America, one of the poorest countries in the world, that republic founded by Bolivar and Sucre, which took the name of our own Bolivar – Bolivia is very rich: minerals, gold, silver, tin, oil and gas, fertile land, great mountains. It’s certainly one of the poorest countries on earth, Bolivia, but the poor are waking up and they’ve just elected an Indian as President of Bolivia, for the first time in history. A true Indian – I’m half Indian, but Evo Morales is an Indian and a half.

I don’t think this is as much of an open-and-shut case as Talk Politics suggests; it may not make much historical sense to blame the Jews for crucifying Jesus, but there are certainly those who do. (Russell Hoban riffs on this in Pilgermann, where his narrator visits an alternative reality in which, in 29 CE, a Roman prophet is executed in one province of a Jewish empire. The Jews still get the blame.) That said, Norm has this flat wrong. This isn’t “the socialism of fools” – just socialism.

(But oh, how convenient it would be for some people if Chavez could be labelled as an anti-semite – not only would it divert attention from the substance of his comments, it would delegitimate him for ever after. We may not have heard the last of this.)

A couple of tra-la-las

I quite liked the Lion, the Witch and the Wardrobe, although I have to admit to a certain bias in favour of any film featuring Tilda Swinton – particularly Tilda Swinton riding in a chariot, wearing chain mail and a lion’s mane (spoiler, sorry), brandishing two large swords and glaring, always glaring

Sorry, I seem to have drifted off for a moment there. Anyway, it looks like a fair bet that they’ll plough on with the rest of the series, and I for one am looking forward to the Last Battle. Who could forget that climactic scene in the Narnia beyond Narnia which was also England beyond England, that land beyond all lands which contained all lands and held within it the bright promise of everything that is true and good in human experience…

“One thing yet puzzleth me but a tad,” said Prince Vivien. “In the tales of old we hear of two sons of Adam and two daughters of Eve, and yet here you are and there’s like three of you total? I mean, hell-o? If ye take my drift, good lords and lady.”

Lucy sighed. “Yes, we told Queen Susan that we were going to jump through our magic mirror into the wonderful land of Narnia and have lots of jolly adventures, but she just said something about having a banging headache from last night and could we all go away. Actually, she didn’t say ‘go away’, she said -”

“That’s about the size of it,” Edmund cut in. “Susan doesn’t care about Narnia these days – all she does seem to care about is nylons and lipstick and mascara and eye shadow and foundation and that red powder that you really have to rub in – rouge, that’s right – and there’s this lip gloss she wears sometimes, you know, and she’s got all these different colours of nail polish, there’s one that’s almost clear but when it catches the light it’s got all these sparkly bits… She’s a sight too interested in all that nonsense, if you ask me.”

Peter nodded. “That, and getting bonked silly by that boyfriend of hers – and no, Lu, I don’t think it’s fair to say ‘which one’, she’s told me she only ever has one on the go at a time. Still, the fact remains that she’s foregone the chance to have lots of jolly adventures in the wonderful land of Narnia in exchange for nothing more than the sordid pleasures of the teenage meat market. She always was a sight too keen on growing up, if you ask me.”

Lady Polly frowned. “There’s growing up and growing up – look at me, I haven’t had a good night out in sixty years, but you don’t see me complaining! No, if you ask me Susan’s one of these modern girls who just want to get to the silliest, most irresponsible, most frivolous, most sexually active and most pleasurable stage of life as quickly as possible – and stay there as long as possible. Why, at this very moment poor old Susan’s probably staggering in after a wild night out, she’s probably got roaring drunk and danced till she was ready to drop, and now she’s probably going to summon her last dwindling reserves of energy for a wild session with some young stud. And tomorrow night she’ll probably do it all over again.”

“Poor old Susan,” said Lord Digory. “When you think, she could have been here with us. In this… place.”

“Is this… is this Heaven?” said Lucy in a small voice.

“Well, we are dead,” said Lord Digory, “if that’s what you mean.”

“I thought so,” said Lucy happily.

The silence was broken by a sigh from Prince Vivien.

“Poor old Queen Susan. To think that she’s missing out on all this.”

“Yes,” said Peter. “Poor old Susan.”

Younger than that now

There’s some good stuff from Ross McKibbin in the current LRB:

the two major parties fundamentally share the same ideology. Despite assurances that the political elite is interested only in what works, this is the most intensely ideological period of government we have known in more than a hundred years. The model of market-managerialism has largely destroyed all alternatives, traditional and untraditional. Its most powerful weapon has been its vocabulary. We are familiar with the way this language has carried all before it. We must sit on the cusp, hope to be in a centre of excellence, dislike producer-dominated industries, wish for a multiplicity of providers, grovel to our line managers, even more to the senior management team, deliver outcomes downstream, provide choice. Our students are now clients, our patients and passengers customers. It is a language which was first devised in business schools, then broke into government and now infests all institutions.

But this rings oddly false:

there is still a sense in which the Conservative Party is not of the real world. Its infantile reaction (fully shared by Cameron) to possible reductions in the British EU rebate – like its attitude to Europe generally – is not the behaviour of a party which wants to be taken seriously.

I’m enough of a Marxist to get extremely twitchy when I hear the word ‘infantile’. Even if we could forget Lenin’s infamous use of the term, ‘infantile’ wouldn’t be a term that belongs in serious political discourse. It’s not criticism so much as gatekeeping: you and I, responsible adults, have our legitimate disagreements within the spectrum of legitimate and responsible politics, but as for them… dear oh dear, why don’t they just grow up?

It was (for obvious reasons) several years ago that Roy Jenkins appeared on Desert Island Discs and nominated a ghastly piece of Stalinist choral kitsch as his first choice (“And every propellor is roaring/Defending the USSR!”). It dated back, he explained, to his undergraduate days, when he indulged in “infantile leftism”. Which struck me.

Our Infant
Tell me, Vladimir Ilyich: when you swam
Those bright days, history running fast about you;
When you stood war from the green West, from the frozen sea;
When the paint was flaking, when last month’s posters
Flapped torn in the streets; when time resumed
And progress was stemmed; were you the only adult?
Were they children, who in that dawn saw other days,
Who would unwire your fences, lift the webs
Necessity had placed with your hands:
Were these people children, infants to be corrected? For they died without descendants, these children,
Soon after you died old. The young webs,
The temporary fences lived and flourished
Till a nation’s leader walked in the dark West,
Walked among the nations as an equal.
It was a glorious nation in its new,
Iron adulthood: a land of strength,
A young triumph over the old world;
And that great baby there was singing its song.

How young he was then! How childish to suppose
There was ever a young dawn in this dull world,
How rash to support the new ruler over the old!
Now, old in the old West, he looks back
On a life well-aged, on the drift of time
That has borne him into this maturity:
Now no bright dawn, now no land of glory
Singing in his voice; now in adult tones
He walks in adult passages, swaddled
In the soft belief that nothing could be better,
Drinking the sweet medicine of no change.

In most areas McKibbin has a sharp eye for New Labour’s ‘market-managerialism’, but when it comes to the EU he’s also drunk his medicine. But in a way that’s only to be expected. Hugo Young’s ‘blessed plot’, the entrenchment of unaccountable bureaucratic power at the European level, preceded New Labour – and seems likely to survive it.

When the sweet turns sour

I’ve! just! exported! my! bookmarks! and! deleted! my! account!.
- Sophie, in comments at Burningbird

If this keeps up all the “Web 2.0″ blog nerds will be working at Yahoo! by next month.
- Jake
Yup! I think that’s the plan!
- Tom
(comments at

delicious was not only a community. It was also an experiment. A place for us geeks to meet and discuss. A place where we were changing the Web. Yes, WE were changing the Web through our ideas. And Joshua was good in picking the best ideas. Inviting us to give more. Now do you really think this will continue under Yahoo!’s reign?
- Pietro

Some lessons to learn here:
1. Never trust a startup service to store your important data no matter how the owner seems honest to you.
2. Never trust a corporate entity to continue storing your important data.
3. Never act like a fanboy on services you don’t trust.
- Ronald Johnson, in comments at the blog

Companies offer web services to get free ideas, exploit free R&D, and discover promising talent. They offer the APIs so people can build clever toys, the best of which the company will grab — thank you very much — and develop further on their own. There is no business model for mashups. If Web 2.0 really is just mashups, this is going to be one short revolution.
- Greg

This enthusiasm for big business – as long as it’s a cool big business – strikes me as both dangerous and weird, not to mention being the antithesis of what’s made the Net fun to work with all these years. But it is a logical development of one branch of the ‘Web 2.0′ hype – an increasingly dominant branch, unfortunately.
- me (on Google)

I promise not to be successful if you all give me money.
- Shelley

Update: I’ve switched to Simpy. It’s great.

A herd of independent minds

I read Francis Wheen warily, not knowing from paragraph to paragraph whether I’m going to agree or start swearing. I read young Oliver very warily indeed: most of what he writes is drivel and some of it’s repulsive. And I don’t read Aaro at all if I can possibly help it.

Wheen, Kamm, Aaronovitch: it’s an unpromising troika. They’ve come together to launch an attack on the Guardian over Emma Brockes’ silly and slapdash interview with Noam Chomsky (which has been taken down from the Guardian Web site but can still be read at Chomsky’s own site, apparently). More specifically, the trio object to the Guardian‘s apology for the interview; they argue that the apology goes too far in correcting the misleading impression given by the interview, painting Chomsky – and, incidentally, Diana Johnstone – in an unwarrantedly favourable light. They have argued this case in a letter of around 4,500 words to the Guardian‘s Reader’s Editor, who has – understandably – concluded that it raises issues outside his competence.

I haven’t seen the letter, but I believe I’ve read enough about this somewhat quixotic endeavour – primarily on Kamm’s blog – to form a judgment on it. My judgment is that it’s a really positive initiative, which I support wholeheartedly. Chomsky is a tendentious and untrustworthy polemicist, whose partisans react with outrage (and in numbers) to criticism of his arguments – and whose rhetorical skills make it extraordinarily difficult to construct a cogent critique. (For illustration, wade through this page, recommended recently by a Chomsky partisan.) On both counts, it is very much to the credit of Kamm & co that they are making the effort; it’s a lot more than I’d care to do just now.

A little background from 1995:
Milan Rai, Chomsky’s politics (Verso, £10.95)

Review printed in New Statesman and Society, 18/8/1995

Since 1969 Noam Chomsky has been one of the foremost radical critics of US foreign policy. Chomsky assiduously documents both the promotion of US interests around the world and the biases and omissions in subsequent media coverage. The resultant portrait of power, corruption and lies is presented as a rational deduction from objective study: the implication is that the government’s apologists cannot plead either difference of opinion or ignorance, but stand self-convicted of lying in the service of power. This is a serious matter: the mendacity of the “intelligentsia” entrenches the limitations of US political culture, foreclosing the prospects for any kind of political reform. Chomsky himself, by contrast, shoulders the responsibility of intellectuals, which is “to speak the truth and to expose lies”.

Milan Rai’s presentation of Chomsky’s politics is detailed, comprehensive and uncritical. Rai has even emulated Chomsky’s habitual contemptuous dismissals of his opponents: Auberon Waugh is characterised, not very accurately, as a “brainwashed intellectual”. (A larger problem is Rai’s treatment of French classicist Pierre Vidal-Naquet’s writings on Chomsky, who had – for reasons which remain obscure – written a relatively friendly preface to a work by Holocaust revisionist Robert Faurisson; Rai dismisses Vidal-Naquet’s criticisms undiscussed as “falsehoods”). This book is thus a missed opportunity. Notwithstanding the enormous value of Chomsky’s work in setting the record of US foreign policy straight, his political assumptions deserve a more thorough and more critical examination.

US society, for Chomsky, is dominated by the “elites”: a term which refers variously to the state apparatus, big business, journalists and academics. The relationship of the elites to the US population is that of an occupying power to a subject territory: the choice is between resistance to elite power and collaboration. Similar considerations apply to the US elites’ relationship with the rest of the world. Indeed, Chomsky denies any significance to the internal politics of nations affected by US foreign policy: “It’s just the same things in Washington playing themselves out in different parts of the world”. Prior to 1989 arguments of this sort even led Chomsky to disparage criticism of the Soviet Union: “the moral value of this work is at best very slight”.

Elite rule is sustained by the “propaganda system”, whereby intellectuals abjure their truth-telling responsibility in favour of manufacturing consent to the status quo. A nuanced analysis shows the “propaganda model” to be multi-faceted: conformity is produced by the economic interests of media businesses, government requirements, cultural resistance to unorthodox analyses and reluctance to put in the necessary work, as well as – what is more commonly cited in practice – the moral turpitude of journalists. (A more accurate term than “propaganda” might have been “received ideas within the capitalist media”). Chomsky even acknowledges the existence of journalists who “use whatever leeway they have”, without thereby modifying his judgment on the class as a whole. Given this level of over-determination and defence against counter-examples, Chomsky’s finding that the model is “one of the best-confirmed theories in the social sciences” is to be expected.

Unsurprisingly, Chomsky’s arguments are at their weakest with respect to the question of what is to be done. On one hand, intellectual self-defence against elite lies is easy (it only requires “ordinary common sense”); on the other, “it does require a degree of fanaticism”, which explains why so few have followed Chomsky’s lead. Chomsky approves non-participation in US presidential elections (“people are intelligent enough to understand that … they are voting for Coca-Cola or Pepsi-Cola”); he also advocates voting: “you’ve got to multiply those little differences in policy by the power of the United States.” Tactical considerations are a moral necessity (“if you write, you have a moral responsibility to consider the consequences of what you write”); then again, “you should do what you think is right and not what’s going to be tactically useful”.

This analysis is conducted, despite Chomsky’s stress on objectivity and rationality, in highly polemical terms. Fascism, Stalinism, terrorism are constant reference points: the US intelligentsia inhabits an “intellectual culture dedicated to terrorist values and policies”; “Fascism is deeply rooted in everyone’s mind in the United States”. While assertions like these are invariably backed up by meticulously syllogistic arguments, the terminology seems designed to raise the rhetorical stakes: analysis turns into name-calling.

These paradoxes rest on the two convictions which underpin Chomsky’s politics. There is a quasi-anarchist stress on the primacy of power relations: capitalism, Communism and fascism all hinge on the control of society by a bureaucratic or managerial elite (“Bolshevism and American liberalism are basically manifestations of the same thing”). This is a powerful vision which illuminates many real continuities; however, it needs to be qualified in the light of history if it is not to turn into a theory of the uniform and interchangeable evil of the elites. This kind of qualification can seem to elude Chomsky, who has argued that the Nazis were among the true victors of the Second World War.

Equally significant is the view – stated by Rai as an ethical truism – that “we must take responsibility for what our society does”. This stress on duty explains the persistent tone of outrage in Chomsky’s work: as a responsible US citizen and intellectual, Chomsky weighs the actions of the US government and the intellectual class and finds them wanting. If, as the “elite” model dictates, the US government is quasi-fascist and the intellectual class composed of power-worshippers, this only rouses Chomsky to greater moral indignation. The classical radical analysis of the state – as an illegitimate imposition on society for which nobody is responsible but the bastards themselves – is foreign to him.

The final paradox of Chomsky’s work is that, however ill-founded his convictions may be, his Herculean labours “to speak the truth and to expose lies” are inconceivable without them. Chomsky is perhaps best seen as a figure like Orwell or Ruskin, his virtues inseparable from his faults. Like those predecessors, when Chomsky goes wrong, he goes seriously wrong; but when he’s right he’s unsurpassable.

The conclusion is kinder than I’d be now, obviously.

PS No, I know he didn’t write it as a preface. He wrote a statement solicited by Serge Thion, a (left-wing) associate of Faurisson, and gave it to Thion with instructions to use it as he saw fit. When he heard that Thion planned to use it as a preface to Faurisson’s work he objected, but too late to prevent it appearing; however, he has subsequently repented the objection. In short, he wrote “a relatively friendly [statement which appears as] a preface to a work by Holocaust revisionist Robert Faurisson [with Chomsky's consent, despite initial objections]“.

PPS On Chomsky, Johnstone and Srebrenica, see Lee Bryant’s comments here and this from Attila Hoare. (Personal to JM – I don’t know why Attila’s writing for them either. Because they asked him, probably.)

In any English town

Mark Honigsbaum, the Guardian:

Curators, librarians and archivists across Britain are being asked to scour their collections in search of documents and items relating to the lives of gay people, with a view to establishing a “virtual museum” of lesbian and gay history.Backed by the museums documentation watchdog, MDA, the group Proud Heritage this week began sending out a two-page survey requesting that institutions throughout the country list the gay and lesbian documents and artefacts in their collections. “For the first time ever, we are asking museums, libraries and archives throughout Britain to revisit their holdings and reveal what they have that is queer,” said Proud Heritage’s director Jack Gilbert. “At the moment these are not classified correctly, or held completely out of context and never see the light of day.”
According to Mr Gilbert the aim is to establish a national database first, featuring a few key virtual exhibits. Once the database was up and running, he said, Proud Heritage would look for a site for a permanent museum, possibly in the King’s Cross area of London.

At the moment these are not classified correctly… You don’t have to be Dave Weinberger to have mixed feelings about that statement. Clearly there’s a case for saying that many of these artifacts aren’t classified adequately, inasmuch as historians of gay experience don’t have an obvious point of access to them – and this could be provided by the proposed database. And, clearly, tagging an artifact with ‘gay’ doesn’t preclude tagging it with ‘Wales’ and ‘early nineteenth-century society’ (the Ladies of Llangollen) or with ‘literature’ and ‘penal reform’ (Wilde’s cell at Reading Gaol).

But classifying an artifact only as ‘gay’ would, in almost all imaginable cases, be no more ‘correct’ than classifying it under any other single term. The project of a physical museum of gay history is welcome in terms of visibility, but in taxonomic terms it’s a step back from the purely ‘virtual’ database project. Like any other thematically-organised museum, it would consist – almost by definition – of exhibits which were ‘not classified correctly’ and ‘held completely out of context’.

Museums promote the illusion that the map is the territory: the structure and layout of the galleries, and the arrangement of the exhibits they contain, are designed to reproduce a certain way of structuring knowledge. (The perfect museum would be its own memory palace.) But an illusion is what it is. Objects can only reside in one place, but knowledge can be fluid and multi-dimensional; pressures to collapse those dimensions – whether in the name of group identity or commerce – should be resisted.

Not that group identity and commerce are necessarily that far apart.

Denis Campbell, the Guardian:

Six per cent of the population, or about 3.6 million Britons, are either gay or lesbian, the government’s first attempt to quantify the homosexual population has concluded.
Publication of the figure comes as big name companies such as Barclays bank, Hilton hotels and cosmetics giant L’Oreal join the growing rush to cash in on a gay economy which is worth tens of billions of pounds. Barclays has just received research which showed that gays and lesbians enjoy a combined annual income of £60 billion.
Barclays spokesman Michael O’Toole admitted the bank is very keen to woo Britain’s gays and lesbians by portraying itself as sympathetic to gays’ desire for equality. ‘We want to position ourselves as the bank of choice for Britain’s gay and lesbian community,’ he said. ‘There’s more of a push going on now to enter this market of about 2.5 million adults.’

The key word here is ‘market’. If Barclay’s is planning to make it easier for gay couples to take out mortgages and insurance policies, this is all to the good, but O’Toole’s ambitions clearly go further. The gay ‘market’ is not like, say, the ‘market’ represented by devout Muslims: Barclay’s plan for those 2.5 million adults is not to introduce them to personal banking, but to encourage them, firstly, to identify as gay; secondly, to perceive Barclay’s as a gay-friendly bank; and thirdly, to switch to Barclay’s on that basis. Identifying as English or middle-aged, a Frascati-drinker or a Manchester City supporter, a dog-owner or a Labour voter would just get in the way: if you’re gay, Barclay’s is the bank for you. The benefits for the bank are obvious; the benefits for their prospective customers, less so.

Knowledge – including our knowledge about ourselves – can be fluid and multi-dimensional; pressures to collapse those dimensions should be resisted.

PS I usually let my titular quotes stand with their cover unblown, but in this case I’ll make a partial exception. For me these are probably the best – certainly the most moving – four lines ever written on the subject of exploring knowledge and where it leaves you. What I’m still not sure about is whether it’s a despairing renunciation, a challenge or a celebration. Cue music:

About the university the pages let you down
It helps you find your way around in any English town
About the university the pages are in French
It helps you find your way around in any English town

I wouldn’t pay him any mind


Nobody I know ever buys a volume of poetry.

Me. I bought two volumes of poetry just the other week – Getting the hang of it and Singing the city, both by Colin Watts (who brought them along to a reading I attended). I can recommend both of them, despite the unfortunate fact that the city in question is Liverpool (is there any city in Britain less in need of celebration?)

Modern poets aren’t saying anything anyone wants to read. Pop music has pushed poetry into obscurity. The only people who buy poetry are students, forced to do so for their courses, Eng Lit graduates, and people who themselves write poetry.

Well, all right – Ellis isn’t actually wrong as such. (I’m a folk singer and, in a small way, a performance poet; I’d gone along that night primarily to do one of my own pieces, and was pleasantly surprised that Colin’s work was actually worth listening to and reading.) But isn’t this a bit like saying ‘the only people who read blogs are people who themselves write blogs’? In other words, even if it’s true, is it a problem?

I used to take poetry very seriously indeed; I used to aspire to be another Browning – or Tennyson at a pinch – with my poems appearing in the broadsheets and my collections selling the way celebrity biographies do now. Perhaps needless to say, I used to look on the poets who actually achieved success and gained a wide audience – from Pam Ayres right up to John Cooper Clarke – with the grestest of disdain: performance poetry? what would that be? It came as a disappointment to realise that, by and large, poetry was read by people who read poetry magazines: the conversation that was conducted in poems was no longer happening in the mainstream press, and the sheer brute genius of my poetry wasn’t going to make it happen there. But the point is – the point always is – to find where there’s a conversation happening and see if you can contribute to it. If you want poetic conversations, you can find them; they may not look like you expect them to, though.

My demands, my angels

Peter Campbell on Samuel Palmer:

Palmer and his friends, meeting together in Shoreham, called themselves the Ancients. Like the Pre-Raphaelites who came afterwards and the German Nazarenes in Rome who had gone before, they were a brotherhood of artists – the first of the kind in England – who wished to renew art from medieval sources. The Ancients didn’t live communally, as the Nazarenes had done, and, unlike the Pre-Raphaelites, who were supported by Ruskin, had no critical backer. … Not all the Ancients were painters or engravers; it seems that they were as much a society of like-minded aesthetes as a school with a single visual aesthetic. There was no manifesto. In what is recorded of them, mainly in memoirs written years later – stories of night-time walks in the countryside round Shoreham and recitations from Macbeth and The Mysteries of Udolpho – the impression is of a group getting high on ideas rather than serious art workers. William Vaughan notes … that Palmer’s son ‘darkly remarked of certain early notebooks by his father’ – he later destroyed them – that ‘they showed “a mental condition which, in many respects is uninviting. It is a condition full of danger, neither sufficiently masculine nor sufficiently reticent.”’

In another country, in another century:

The “Hamburg Theses” are assuredly the most mysterious of all the documents produced by the Situationist International. Many such documents had a wide circulation; others were often reserved for a discreet audience.The “Hamburg Theses” were referred to several times in situationist publications, but without a single quotation from them ever being given … They were in fact the conclusions (kept secret by agreement) of a theoretical and strategic discussion bearing on the whole of the activity of the SI. This discussion took place over two or three days early in September 1961, in a randomly-chosen series of bars in Hamburg, beween G. Debord, A. Kotanyi and R. Vaneigem, who were then returning from the fifth Conference of the SI, held at Göteborg from the 28th to the 30th of August. Alexander Trocchi, not present in person at Hamburg, would later contribute to these “Theses”. Quite deliberately, with a view to preventing anything which might have given rise to external remarks or analysis from circulating outside the SI, nothing was ever put on paper regarding this discussion or its conclusions. It was agreed that the most simple summary of these conclusions, rich and complex as they were, could be delivered in a single sentence: The SI must now realise philosophy. Even this sentence was not written down. The conclusion was so well hidden that it has remained secret up to now [1989].

The “Hamburg Theses” had a considerable significance in at least two respects. Firstly, because they mark the main turning point in the entire history of the SI. But equally as a form of experimental practice: from this point of view, they were a striking innovation in the history of artistic avant-gardes, who until then had all given the impression of being only too eager to explain themselves.

The conclusion stated evoked a famous formulation by Marx in 1844 (in his Contribution to a critique of Hegel’s Philosophy of Right). It meant that, from that moment, we must no longer accord the least importance to the ideas of any of the revolutionary groups which still survived, heirs as they were of the old movement for social emancipation which had been crushed in the first half of the century; and that we could therefore count on nobody but the SI itself to begin another era of challenge to power, recommencing from all the starting points of the movement which had been constituted in the 1840s. … the “Hamburg Theses” marked the end of the SI’s first period – the search for a genuinely new artistic terrain (1957-61); they also fixed the point of departure of the operation which led to the movement of May 1968, and what came after.

A condition full of danger, perhaps. A group getting high on ideas rather than serious art workers, definitely – but what a group, and what ideas.

Guy Debord, 28/12/1931-30/11/1994

Bernard, Bernard, disait-il, cette verte jeunesse ne durera pas toujours

PS The birthday game (introduced to me by Chris Dillow), takes me to page 200 of the 1997 Internationale Situationniste anthology (from which the passage translated above was taken), and thence straight back to Debord (the only other possibilities are Joergen Nash, Attila Kotanyi and Helmut Sturm). I can’t find any personal connection with anyone who was born on Debord’s birthday, but I can say that John von Neumann and Linus Torvalds have both had considerable influence on my life. Not as much as Debord, though.

PPSThat Bossuet quote was one of Debord’s favourites, but it’s a bit sentimental – it’s not the outlook of a true revolutionary. Let’s face it:

Les avant-gardes n’ont qu’un temps; et ce qui peut leur arriver de plus heureux, c’est, au plein sens du terme, d’avoir fait leur temps. Après elles, s’engagent des opérations sur un plus vaste theatre. On n’en a que trop vu, de ces troupes d’élite qui, après avoir accompli quelque vaillant exploit, sont encore là pour défiler avec leurs décorations, et puis se retournent contre la cause qu’elles avaient défendue. Il n’y a rien à craindre de semblable de celles dont l’attaque a été menée jusqu’au terme de la dissolution. Je me demande ce que certains avaient espéré de mieux? Le particulier s’use en combattant. Un projet historique ne peut certainement pas prétendre conserver une éternelle jeunesse à l’abri des coups.

Guy Debord, In girum imus nocte et consumimur igni (1978)

We are your friends

I recently attended an “e-Government Question Time” session, organised in connection with this conference. There were some good points made: one speaker stressed the importance of engaging with the narratives which people build rather than assuming that the important facts can be read off from an accumulation of data; one questioner called the whole concept of ‘e-government’ into question, pointing out that the stress seemed to be entirely on using the Web/email/digital TV/texting/etc as a mechanism for delivering services rather than as a medium for democratic exchanges. Much more typical, though, was the spin which the mediator put on this question as he passed it on to the panel:

That’s a very good question – what about democracy? And conversely, if it’s all democracy where does that leave leadership?

The evening was much less about democracy than it was about leadership – or rather, management. This starting-point produced some strikingly fallacious arguments, particularly in the field of privacy. The following statements were all made by one panellist; I won’t single him out, as they were all endorsed by other panellists – and, in some cases, members of the audience. (And no, identifying him as male doesn’t narrow it down a great deal. The people in the hall were 3:1 male to female (approximately), the people on stage 6:1 (precisely).)

I like to protect my own privacy, but I’m in the privileged position of having assets to protect. When you’re looking at people who have got nothing, and in many cases aren’t claiming benefits to which they’re entitled, I don’t think safeguarding their privacy should be our main concern.

At first blush this argument echoes the classic Marxist critique of the bourgeois definition of human rights – if we have the right to privacy, what about the right to a living wage? But instead of going from universalism to a broader (and hence more genuine) universalism, we’ve ended up with the opposite of universalism: you and I can worry about privacy, but it doesn’t apply to them. Superficially radical, or at least populist – you can just hear David Blunkett coming out with something similar – this is actually a deeply reactionary argument: it treats the managed as a different breed from the people who manage them (I like to protect my own privacy, but…). Management Fallacy 1: ‘they’re not like us’.

We’re talking about improving people’s life chances. We need to make personal information more accessible – to put more access to personal information in the hands of the people who can change people’s lives for the better.

Management Fallacy 2: ‘we mean well’. If every intervention by a public servant were motivated by the best interests of the citizens, safeguards against improper intervention would not be required. And if police officers never stepped out of line, there’d be no need for a Police Complaints Commission. In reality, good intentions cannot be assumed: partly because the possibility of a corrupt or malicious individual getting at your data cannot be ruled out; partly because government agencies have other functions as well as safeguarding the citizen’s interests, and those priorities may occasionally come into conflict; and partly because a government agency’s idea of your best interests may not be the same as yours (see Fallacy 3). All of which means that the problem needs to be addressed at the other end, by protecting your data from people who don’t have a specific reason to use it – however well-intentioned those people may be. One questioner spoke wistfully of the Data Protection Act getting in the way of creative, innovative uses of data. It’s true that data mining technology now makes it possible to join the dots in some very creative and innovative ways. But if it’s data about me, I don’t think prior consent is too much to ask – and I don’t think other people are all that different (see Fallacy 1).

I’ve got no objection to surrendering some of my civil liberties, so-called

Have to stop you there. Management Fallacy 3: ‘it looks all right to me’. The speaker was a local government employee: a private individual. His policy for handling his own private data doesn’t concern me. But I would hope that, before he came to apply that policy more generally, he would reflect on how the people who would be affected might feel about surrendering their civil liberties, so-called. (Perhaps he could consult them, even.)

Carry on:

I’ve got no objection to surrendering some of my civil liberties, so-called, if it’s going to prevent another Victoria Climbie case.

Management Fallacy 4: ‘numbers don’t lie’. (Or: ‘Everything is measurable and what can be measured can be managed’.) This specific example is a common error with statistics, which can be illustrated with the example of a hypothetical test for the AIDS virus. Let’s say that you’ve got an HIV test which is 95% accurate – that is, out of every 100 people with HIV it will correctly identify 95 and mis-identify 5, and similarly for people who do not carry the virus. And let’s say that you believe, from other sources, that 1,000 people in a town of 100,000 carry the virus. You administer the test to the entire town. If your initial assumption is correct, how many positive results will you get? And how confident can you be, in percentage terms, that someone who tests positive is actually HIV-positive?

The answers are 5900 and 16.1%. The test would identify 950 of the 1000 people with the virus, but it would also misidentify 4950 people who did not have it: consequently, anyone receiving a positive test result would have a five in six chance of actually being HIV-negative. What this points to is a fundamental problem with any attempt to identify rare phenomena in large volumes of data. If the frequency of the phenomenon you’re looking for is, in effect, lower than the predictable rate of error, any positive result is more likely to be an error than not.

Contra McKinsey, I would argue that not everything can or should be measured, let alone managed on the basis of measurement. (If the data-driven approach to preventing another Climbie case sounds bad, imagine it with the addition of performance targets.) Some phenomena – particularly social phenomena – are not amenable to being captured through the collection of quantitative data, and shouldn’t be treated as if they were.

What all these fallacies have in common is a self-enclosed, almost solipsistic conception of the task of management. With few exceptions, the speakers (and the questioners) talked in terms of meeting people’s needs by delivering a pre-defined service with pre-defined goals, pre-defined techniques, pre-defined identities (me service provider, you service recipient). There were only occasional references to the exploratory, dialogic approach of asking people what their needs were and how they would like them to be met – despite the possibilities for work in this area which new technologies have created. But then, management is not dialogue.

Social software may start with connecting data, but what it’s really about is connecting people – and connecting them in dialogue, on a basis of equality. If this goal gets lost, joining the dots may do more harm than good.

Because he’s a big bloke

[Sorry about the hiatus - life called.]

John Stevens:
I genuinely never thought I’d say this, but I am now convinced that the monster who executed this young woman in cold blood should, in turn, be killed as punishment for his crime.

Logically, I don’t have a problem with the concept of vendetta. I’m a peaceable type – I’ve never really been in a fight, let alone beaten anyone up – but if you were to kill one of mine, I would assuredly want to kill you or one of yours. Which would of course put me in exactly the same position as you – and so it would go on until there was nobody left, basically. The open-endedness of the vendetta is its major design flaw. In practice there is no equivalence between your loss of a son and my loss of a brother: each of them was a unique and irreplaceable person, and both deaths cry out for redress. Consequently each of us has a good and pressing reason for breaking the taboo on unregulated violence within our community – just this once, you understand… Vendetta corrodes community.

There are two basic approaches to managing the vendetta. At one extreme, it can be managed by the imposition of a superior authority – backed by the threat of superior force – on both parties: think of Mob feuds being halted by the intervention of a capo, perhaps with one final, sanctioned hit to balance the books. At the other, it can be replaced from below, by collective conflict-resolution processes which end in the imposition of penalties sanctioned by the community as a whole. The oldest Greek tragedy, the Oresteia, celebrates and re-enacts precisely this recognition of the corrosiveness of vendetta, and its replacement by sanctions imposed by a deliberative community.

These are two extremes; actually existing criminal justice systems have elements of both. The fundamental argument against the death penalty is that – as John Stevens unwittingly reveals – it moves us further from vendetta-replaced-from-below and closer to vendetta-managed-from-above:

the monster who executed this young woman in cold blood should, in turn, be killed

Should, in turn, be executed in cold blood by a monster. But our monster.

“For by art is created that great LEVIATHAN called a COMMONWEALTH or STATE” – ‘by art’, which is to say, by collective human will and intelligence and imagination. It’s a burden, and it’s a responsibility.

Which side of the table?

[Pardon the long silence - life called.]


I think Google Base is a fun experiment, and I’m willing to play a little. It will be interesting to see the directory, especially if the company provides web services that aren’t limited to so many queries a day. But I never forget that Google is in the business to make a profit. If we give it the power, it will become the Wal-Mart of the waves–by default if not by design. Is that what you all want? If it is, just continue getting all misty eyed, because you’ll need blurred vision not to see what should be right in front of you.

I think this is precisely right. I find a lot of the comment on Google Base strange and slightly depressing, in the same way I find a lot of Web 2.0 talk strange and depressing. In the context of social software, when I use a word like ‘enclose’ – or a word like ‘monetise‘ – it means something quite specific and entirely negative: it’s a red-flag word. So it’s weird, to say the least, to see the same words used positively. It’s only a little less strange to see these concerns acknowledged, then batted away as trivial or meaningless (Tom: “Making data available for everyone to use is keeping it in the public sphere.” (I’m Phil #2 in those comments, by the way)).

It seems to me that there’s a fundamental tension between the demands of commerce and the nature of social software, as defined by Tom some time ago:

We believe that for a piece of Social Software to be useful:

  • Every individual should derive value from their contributions
  • Every contribution should provide value to their peers as well
  • The site or organisation that hosts the service should be able to derive value from the aggregate of the data and should be able to expose that value back to individuals

You add content, which has value for you and to other users; the host derives further value from the aggregate of content; the host exposes that added value to all users. Beautiful.

What this suggests is that social software – unlike, say, the e-business of the late 1990s – is all about the content. Specifically, it’s all about freely and collectively contributed content, either held in common or held in trust for the commons. So monetising social software is qualitatively different from making money out of a new piece of stand-alone software, because it’s the contributed content which has the original value – and makes the added value possible. Tangentially, I’m not sure whether Doc Searls gets this or not, and not only because his article’s had a well-deserved slashdotting. On a first reading I got the impression he was saying that both the Net and the Web are valuable common resources which should not be fenced off for the sake of making money, but that part of what makes them valuable is that they’re great environments for fencing things off and making money. (Oh, and we should stop saying ‘common’ because if you put ‘ist’ on the end it sounds kind of like ‘communist’, and when Eric Raymond hears the word ‘Communist’ he reaches for well, you know.) It’s a great article, though, and I look forward to taking a more considered look at it when the tide subsides.

One of Doc’s points is that the Web is, figuratively, all about publishing – a profession which, like many bloggers, I’ve seen close up. Just under ten years ago, I went from Unix sysadmin to magazine editor, and rapidly discovered that commercial publishing looks very different from the inside. Perhaps the biggest single shock was the realisation that content doesn’t matter. Obviously I tried to make it the best magazine I could (and it got better still under my successor), but at a fundamental level editorial content wasn’t what it was about. If the advertising department sold enough space, we made a profit; if they didn’t, we didn’t. (Show me a magazine that relies on the cover price and I’ll show you a magazine with money worries. Show me a publication that gets by on the cover price and I’ll show you an academic journal.) The purpose of the magazine was to put advertisements in front of readers – and the purpose of the editorial was to make readers turn all the pages.

So there’s nothing very new about Google’s business model: Google Base is to the Web what a commercial magazine is to a fanzine – or rather, a whole mass of different fanzines. The only novelty is that we, the fanzine writers, are providing the content: the content whose sole function, from the point of view of Google as a commercial entity, is to attract an audience which will look at ads.

But that’s quite a big novelty.

Bill Burnham:

In my next post I will talk about Google Base’s impact on the “walled garden” listings sites. I’ll give you a hint: it won’t be pretty.

Unless, of course, you like really big gardens with really high walls.

Update: I wrote, I find a lot of the comment on Google Base strange and slightly depressing, in the same way I find a lot of Web 2.0 talk strange and depressing. Cue Cringely:

Google has the reach and the resources … And you know whose strategy this is? Wal-Mart’s. And unless Google comes up with an ecosystem to allow their survival, that means all the other web services companies will be marginalized. … the final result is that Web 2.0 IS Google. Microsoft can’t compete. Yahoo probably can’t compete. Sun and IBM are like remora, along for the ride. And what does it all cost, maybe $1 billion? That’s less than Microsoft spends on legal settlements each year. Game over.

As an aside, I love the idea of International Business Machines as a parasite on the behemoth that is Google; I don’t think we’re quite there yet. But the accuracy or not of Cringely’s prediction concerns me less than his tone, which I think can reasonably be called lip-smacking: “Google’s going to 0wn the Web! Wow!” (Quick test: try reading that sentence out loud with a straight face. Now try substituting ‘Microsoft’ – or, for older readers, ‘IBM’.) This enthusiasm for big business – as long as it’s a cool big business – strikes me as both dangerous and weird, not to mention being the antithesis of what’s made the Net fun to work with all these years. But it is a logical development of one branch of the ‘Web 2.0′ hype – an increasingly dominant branch, unfortunately.

This is the new stuff

Thomas criticises Wikipedia’s entry on folksonomy – a term which was coined just over a year ago by, er, Thomas. As of today’s date, the many hands of Wikipedia say:

Folksonomy is a neologism for a practice of collaborative categorization using freely chosen keywords. More colloquially, this refers to a group of people cooperating spontaneously to organize information into categories, typically using categories or tags on pages, or semantic links with types that evolve without much central control. … In contrast to formal classification methods, this phenomenon typically only arises in non-hierarchical communities, such as public websites, as opposed to multi-level teams and hierarchical organization. An example is the way in which wikis organize information into lists, which tend to evolve in their inclusion and exclusion criteria informally over time.


Today, having seen an new academic endeavor related to folksonomy quoting the Wikipedia entry on folksonomy, I realize the definition of Folksonomy has become completely unglued from anything I recognize (yes, I did create the word to define something that was undefined prior). It is not collaborative, it is not putting things into categories, it is not related to taxonomy (more like the antithesis of a taxonomy), etc. The Wikipedia definition seems to have morphed into something that the people with Web 2.0 tagging tools can claim as something that can describe their tool

I’m resisting the temptation to send Thomas the All-Purpose Wikipedia Snark Letter (“Yeah? Well, if you don’t like the wisdom of the crowds, Mr So-Called Authority…”). In fact, I’m resisting the temptation to say anything about Wikipedia; that’s another discussion. But I do want to say something about the original conception of ‘folksonomy’, and about how it’s drifted.

Firstly, another quote from Thomas’s post from today:

Folksonomy is the result of personal free tagging of information and objects (anything with a URL) for one’s own retrival. The tagging is done in a social environment (shared and open to others). The act of tagging is done by the person consuming the information.

There is tremendous value that can be derived from this personal tagging when viewing it as a collective, when you have the three needed data points in a folksonomy tool: 1) the person tagging; 2) the object being tagged as its own entity; and 3) the tag being used on that object. … [by] keeping the three data elements you can use two of the elements to find a third element, which has value. If you know the object (in it is the web page being tagged) and the tag you can find other individuals who use the same tag on that object, which may lead (if a little more investigation) to somebody who has the same interest and vocabulary as you do. That person can become a filter for items on which they use that tag. You then know an individual and a tag combination to follow.

This is admirably clear and specific; it also fits rather well with the arguments I was making in two posts earlier this year:

[perhaps] the natural state of knowledge is to be ‘cloudy’, because it’s produced within continuing interactions within groups: knowledge is an emergent property of conversation, you could say … [This suggests that] every community has its own knowledge-cloud – that the production and maintenance of a knowledge-cloud is one way that a community defines itself.

If ‘cloudiness’ is a universal condition, and flickr and tag clouds and so forth don’t enable us to do anything new; what they are giving us is a live demonstration of how the social mind works. Which could be interesting, to put it mildly.

Thomas’s original conception of ‘folksonomy’ is quite close to my conception of a ‘knowledge cloud’: they’re both about the emergence of knowledge within a social interaction (a conversation).

The current Wikipedia version of ‘folksonomy’ is both fuzzier and more closely tied to existing technology. What’s happened seems to be a kind of vicious circle of hype and expectations management. It’s not a new phenomenon – anyone who’s been watching IT for any length of time has seen it happen at least once. (Not to worry anyone, but it happened quite a lot around 1999, as I remember…)

  1. There’s Vision: someone sees genuinely exciting new possibilities in some new technology and writes a paper on – oh, I don’t know, noetic telepresence or virtual speleology or network prosody…
  2. Then there’s Development: someone builds something that does, well, a bit of it. Quite significant steps towards supporting network prosody. More coming in the next release.
  3. Phase three is Hype. Hype, hype, hype. Mm-hmm. I just can’t get enough hype, can you?
  4. The penultimate phase is Dissemination: in which everyone’s trying to support network prosody. Or, at least, some of it. That stuff that those other people did with their tool. There we go, fully network prosody enabled – must get someone to do a writeup.
  5. Finally we’re into Hype II, also known as Marketing: ‘network prosody’ is defined less by the original vision than by the tools which have been built to support it. The twist is that it’s still being hyped in exactly the same way – tools which don’t actually do that much are being marketed as if they realised the original Vision. It’s a bit of a pain, this stage. Fortunately it doesn’t last forever. (Stage 6 is the Hangover.)

What’s to be done? As I said back here, personally I don’t use the term ‘folksonomy’; I prefer Peter Merholz’s term ‘ethnoclassification’. Two of my objections to ‘folksonomy’ were that it appears to denote an end result as well as a process, and that it’s become a term of (anti-librarian) advocacy as well as description; Thomas’s criticisms of Wikipedia seem to point in a similar direction. Where I do differ from Thomas is in the emphasis to be placed on online technologies. Ethnoclassification is – at least, as I see it – something that happens everywhere all the time: it’s an aspect of living in a human community, not an aspect of using the Web. If I’m right about where we are in the Great Cycle of Hype, this may soon be another point in its favour.

None of you stand so tall

In the previous post, I showed that the canonical ‘power law’ chart which underlies the Long Tail image does not, in fact, represent a power law. What it represents is a ranked list, which happens to have a similar shape to a power law series: as it stands, the ‘power law’ is an artifact of the way the list has been sorted. In particular, the contrast which is often drawn, in this context, between a power law distribution and a normal distribution is inappropriate and misleading. If you sort a list high to low, it can only ever have the shape of a descending curve.

There are counter-arguments, which I’ll go through in strength order (weakest first).

Counter-argument 1: the Argument from Inconsequentiality.

In the post which started it all, Clay wrote:
the shape of Figure #1, several hundred blogs ranked by number of inbound links, is roughly a power law distribution.

Note weasel wordage: it would be possible to argue that what Clay (and Jason Kottke) identified wasn’t really a power law distribution, it was just some data which could be plotted in a way which looked oddly like a power law curve. Thankfully, Clay cut off this line of retreat, referring explicitly to power law distributions:

power law distributions are ubiquitous. Yahoo Groups mailing lists ranked by subscribers is a power law distribution. LiveJournal users ranked by friends is a power law … we know that power law distributions tend to arise in social systems where many people express their preferences among many options.

And so on. When we say ‘power law’, we mean ‘power law distribution’: we’re all agreed on that.

Except, of course, that what we’re talking about isn’t a power law distribution. Which brings us to…

Counter-argument 2: the Argument from Intuition.

The pages I excerpted in the previous post specifically contrast the power law distribution with the ‘normal’ bell curve.

many web statistics don’t follow a normal distribution (the infamous bell curve), but a power law distribution. A few items have a significant percentage of the total resource (e.g., inbound links, unique visitors, etc.), and many items with a modest percentage of the resources form a long “tail” in a plot of the distribution.

we find a very few highly connected sites, and very many nearly unconnected sites, a power law distribution whose curve is very high to the left of the graph with the highly connected sites, with a long “tail” to the right of the unconnected sites. This is completely different than the bell curve that folks normally assume

The Web, like most networks, has a peculiar behavior: it doesn’t follow standard bell curve distributions … [it] follows a power law distribution where you get one or two sites with a ton of traffic (like MSN or Yahoo!), and then 10 or 20 sites each with one tenth the traffic of those two, and 100 or 200 sites each with 100th of the traffic, etc.

One of my Latin teachers at school had an infuriating habit, for which (in the best school-story tradition) I’m now very grateful. If you read him a translation which didn’t make sense (grammatically, syntactically or literally) he’d give you an anguished look and say, “But how can that be?” It was a rhetorical question, but it was also – infuriatingly – an open question: he genuinely wanted you to look again at what you’d written and realise that, no, actually that noun in the ablative couldn’t be the object of the verb… Good training, and not only for reading Latin.

If you’ve got this far, do me a favour and re-read the excerpts above. Then ask yourself: how can that be?

As long as we’re talking about interval/ratio variables – the only type for which a normal distribution can be plotted – it’s hard to make sense of this stuff. What, to put it bluntly, is being plotted on the X axis? The best I can do is to suppose that the X axis plots number of sites: A few items have a significant percentage of the total resource; a very few highly connected sites; one or two sites with a ton of traffic. There’s your spike on the left: a low X value (a few items) and a high Y (a significant percentage of the total resource).

But this doesn’t really work either. Or rather, it could work, but only if every group of sites with the same number of links had a uniquely different number of members – and if the number of members in each group were in inverse proportion to the number of links (1 site with n links, 2 sites with n/2 links, 3 sites with n/3 links, 4 sites with n/4 links…). This isn’t impossible, in very much the same way that the spontaneous development of a vacuum in this room isn’t impossible; a pattern like that wouldn’t be a power law so much as evidence of Intelligent Design.

This is an elaborate and implausible model; it’s also something of a red herring, as we’ll see in a minute. It’s worth going into in detail, though; as far as I can see, it’s the only way of getting these data into a power law distribution, with high numbers of links on the left, without using ranking. And cue…

Counter-argument 3: the Argument from Ranking.

Over to Clay:

The basic shape is simple – in any system sorted by rank, the value for the Nth position will be 1/N. For whatever is being ranked — income, links, traffic — the value of second place will be half that of first place, and tenth place will be one-tenth of first place. (There are other, more complex formulae that make the slope more or less extreme, but they all relate to this curve.)

“The value for the Nth position will be 1/N” (or proportionate to 1/N, to be more precise); alternatively, you could say that N items have a value of 1/N or greater. (Have a think about this one – we’ll be coming back to it later.) Either way, it’s a power law, right? Well, yes – and no. It’s certainly true to say that a ranked list with these properties confirms to a version of the power law – specifically, Zipf’s law. It’s also true to say that Zipfian rankings are associated with Pareto-like power law distributions: we may yet be able to find a power law in this data. But we’re not there yet – and Clay’s presentation of the data doesn’t help us to get there. (Jason’s has some of the same problems, but Clay’s piece is a worse offender; it’s also much more widely known.)

The first problem is with the recurrent comparison of ranked graphs with bell curves. Adam: “a ranked graph … by definition is *always* decreasing, and can *never* be a bell curve”. If anyone tells you that such and such a phenomenon follows a power law rather than a normal distribution, take a good look at their X axis. If they’ve got ranks there, the statement is meaningless.

Secondly, the graph Clay presented – a classic of the ‘big head, long tail’ genre – isn’t actually a Zipfian series, for the simple reason that it includes tied ranks: it’s not a list of ranks but a list of nominals sorted into rank order.

I’ll clarify. Suppose that we’ve got a series which only loosely conforms to Zipf’s Law, perhaps owing to errors in the real world:

Rank Value
1 1000
2 490
3 340
4 220
5 220
6 180
7 140

Now, what happens on the graph around values 4 and 5? If the X axis represents ranking, it makes no sense to say that the value of 220 corresponds to a rank of 4 and a rank of 5: it’s a rank of 4, followed by no ranking for 5 and a rank of 6 for the value of 180. We can see the point even more clearly if we take the alternative interpretation of a Zipfian list and say that the X axis tracks ‘number of items with value greater than or equal to Y’. Clearly there are 6 items greater than or equal to 180 and 5 greater than or equal to 220 – but it would be nonsensical to say that there are also 4 items greater than or equal to 220. Either way, if you have a ranked list with tied rankings this should be represented by gaps in the graph.

This may seem like a minor nitpick, but it’s actually very important. Back to Adam:

One nice thing about a ranked graph is that the “area” under the curve is equal to the total value associated with the items spanned on the ranked axis

Or, in the words of one of the pieces I quoted in the previous post:

In such a curve the distribution tapers off slowly into the sunset, and is called a tail. What is most intriguing about this long tail is that if you add up all the traffic at the end of it, you get a lot of traffic

What we’re talking about, clearly, is the Long Tail. Looking at some actual figures for inbound linkage (collected from NZ Bear earlier this year), there are few tied ranks in the higher rankings and more as we go further out: 95 unique values in the first 100 ranks and 79 in the next 100. Further down, the curve grows flatter, as we’d expect. The first ten rankings (ranging from 5,389 down to 2,142 links) correspond to ten sites; the last ten (ranging, predictably, from 9 down to zero) correspond to a total of 14,445. As Adam says, if you were to graph these data as a list of nominals ranked in descending order, the ‘area’ covered by the curve would give you a good visual impression of the total number of links accounted for by low-linked sites: the Long Tail, no other. But this graphic does not conform to a power law – not even Zipf’s Law. A list conforming to Zipf’s Law would drop tied ranks – it would exclude duplicates, if that’s any clearer. Instead of a long tail, it would trail off to the right with a series of widely-spaced fenceposts. (“In equal 9126th place, blogs with 9 links; in equal 9593rd place, 8-linkers…”)

Long Tail, power law: choose one.

You can have a Long Tail, but only by graphing a list of nominals ranked in descending order.

You can have a power law series with rankings, but only by replacing the long tail with scattered fenceposts.

Even more importantly, neither of these is a power law distribution. Given the appropriate data values, you can derive a power law distribution from a ranked list – but it doesn’t look like the ‘long tail’ graphic we know so well. I’ll talk about what it does look like in the next post.

Put your head back in the clouds

OK, let’s talk about the Long Tail.

I’ve been promising a series of posts on the Long Tail myth for, um, quite a while. (What’s a month in blog time? A few of those.) The Long Tail posts begin here.

Here’s what we’re talking about, courtesy of our man Shirky:

We are all so used to bell curve distributions that power law distributions can seem odd. The shape of Figure #1, several hundred blogs ranked by number of inbound links, is roughly a power law distribution. Of the 433 listed blogs, the top two sites accounted for fully 5% of the inbound links between them. (They were InstaPundit and Andrew Sullivan, unsurprisingly.) The top dozen (less than 3% of the total) accounted for 20% of the inbound links, and the top 50 blogs (not quite 12%) accounted for 50% of such links.

Figure #1: 433 weblogs arranged in rank order by number of inbound links.

It’s a popular meme, or it would be if there were any such thing as a meme (maybe I’ll tackle that one another time). Here’s one echo:

many web statistics don’t follow a normal distribution (the infamous bell curve), but a power law distribution. A few items have a significant percentage of the total resource (e.g., inbound links, unique visitors, etc.), and many items with a modest percentage of the resources form a long “tail” in a plot of the distribution. For example, a few websites have millions of links, more have hundreds of thousands, even more have hundreds or thousands, and a huge number of sites have just one, two, or a few.


if we measure the connectivity of a sample of 1000 web sites, (i.e. the number of other web sites that point to them), we might find a bell curve distribution, with an “average” of X and a standard deviation of Y. If, however, that sample happened to contain, then things would be off the chart for the “outlier” and normal for every other one.If we back off to see the whole web’s connectivity, we find a very few highly connected sites, and very many nearly unconnected sites, a power law distribution whose curve is very high to the left of the graph with the highly connected sites, with a long “tail” to the right of the unconnected sites. This is completely different than the bell curve that folks normally assume

And another:

The Web, like most networks, has a peculiar behavior: it doesn’t follow standard bell curve distributions where most people’s activities are very similar (for example if you plot out people’s heights you get a bell curve with lots of five- and six-foot people and no 20-foot giants). The Web, on the other hand, follows a power law distribution where you get one or two sites with a ton of traffic (like MSN or Yahoo!), and then 10 or 20 sites each with one tenth the traffic of those two, and 100 or 200 sites each with 100th of the traffic, etc. In such a curve the distribution tapers off slowly into the sunset, and is called a tail. What is most intriguing about this long tail is that if you add up all the traffic at the end of it, you get a lot of traffic

All familiar, intuitive stuff. It’s entered the language, after all – we all know what the ‘long tail’ is. And when, for example, Ross writes about somebody who started blogging about cooking at the end of the tail and is now part of the fat head and has become a pro, we all know what the ‘fat head’ is, too – and we know what (and who) is and isn’t part of it.

Unfortunately, the Long Tail doesn’t exist.

To back up that assertion, I’m going to have to go into basic statistics – and trust me, I do mean ‘basic’. In statistics there are three levels of measurement, which is to say that there are three types of variable. You can measure by dividing the field of measurement into discrete partitions, none of which is inherently ranked higher than any other. This car is blue (could have been red or green); this conference speaker is male (could have been female); this browser is running under OS X (could have been Win XP). These are nominal variables. You can code up nominals like this as numbers – 01=blue, 02=red; 1=male, 2=female – but it won’t help you with the analysis. The numbers can’t be used as numbers: there’s no sense in which red is greater than blue, female is greater than male or OS X is – OK, bad example. Since nominals don’t have numerical value, you can’t calculate a mean or a median with them; the most you can derive is a mode (the most frequent value).

Then there are ordinal variables. You derive ordinal variables by dividing the field of measurement into discrete and ordered partitions: 1st, 2nd, 3rd; very probable, quite probable, not very probable, improbable; large, extra-large, XXL, SuperSize. As this last example suggests, the range covered by values of an ordinal variable doesn’t have to exhaust all the possibilities; all that matters is that the different values are distinct and can be ranked in order. Numeric coding starts to come into its own with ordinals. Give ‘large’ (etc) codes 1, 2, 3 and 4, and a statement that (say) ’50% of size observations are less than 3′ actually makes sense, in a way that it wouldn’t have made sense if we were talking about car colour observations. In slightly more technical language, you can calculate a mode with ordinal variables, but you can also calculate a median: the value which is at the numerical mid-point of the sample, when the entire sample is ordered low to high.

Finally, we have interval/ratio or I/R variables. You derive an I/R variable by measuring against a standard scale, with a zero point and equal units. As the name implies, an I/R variable can be an interval (ten hours, five metres) or a ratio (30 decibels, 30% probability). All that matters is that different values are arithmetically consistent: 3 units minus 2 units is the same as 5 minus 4; there’s a 6:5 ratio between 6 units and 5 units. Statistics starts to take off when you introduce I/R variables. We can still calculate a mode (the most common value) and a median (the midpoint of the distribution), but now we can also calculate a mean: the arithmetic average of all values. (You could calculate a mean for ordinals or even nominals, but the resulting number wouldn’t tell you anything: you can’t take an average of ‘first’, ‘second’ and ‘third’.)

You can visualise the difference between nominals, ordinals and I/R variables by imagining you’re laying out a simple bar chart. It’s very simple: you’ve got two columns, a long one and a short one. We’ll also assume that you’re doing this by hand, with two rectangular pieces of paper that you’ve cut out – perhaps you’re designing a poster, or decorating a float for the Statistical Parade. Now: where are you going to place those two columns? If they’re nominals (‘red cars’ vs ‘blue cars’), it’s entirely up to you: you can put the short one on the left or the right, you can space them out or push them together, you can do what you like. If they’re ordinals (‘second class degree awards’ vs ‘third class’) you don’t have such a free rein: spacing is still up to you, but you will be expected to put the ‘third’ column to the right of the ‘second’. If they’re I/R variables, finally – ’180 cm’, ’190 cm’ – you’ll have no discretion at all: the 180 column needs to go at the 180 point on the X axis, and similarly for the 190.

Almost finished. Now let’s talk curves. The ‘normal distribution’ – the ‘bell curve’ – is a very common distribution of I/R variables: not very many low values on the left, lots of values in the middle, not very many high values on the right. The breadth and steepness of the ‘hump’ varies, but all bell curves are characterised by relatively steep rising and falling curves, contrasting with the relative flatness of the two tails and the central plateau. The ‘power law distribution’ is a less common family of distributions, in which the number of values is inversely proportionate to the value itself or a power of the value. For example, deriving Y values from the inverse of the cube of X:

X value Y formula Y value
1 1000 / (1^3) 1000
2 1000 / (2^3) 125
3 1000 / (3^3) 37.037
4 1000 / (4^3) 15.625
5 1000 / (5^3) 8
6 1000 / (6^3) 4.63

As you can see, a power law curve begins high, declines steeply then ‘levels out’ and declines ever more shallowly (it tends towards zero without ever reaching it, in fact).

Got all that? Right. Quick question: how do you tell a normal distribution from a power-law distribution? It’s simple, really. In one case both low and high values have low numbers of occurrences, while most occurrences are in the central plateau of values around the mean. In the other, the lowest values have the highest numbers of occurrences; most values have low occurrence counts, and high values have the lowest counts of all. In both cases, though, what you’re looking at is the distribution of interval/ratio variables. The peaks and tails of those distribution curves can be located precisely, because they’re determined by the relative counts (Y axis) of different values (X axis) – just as in the case of our imaginary bar chart.

Back to a real bar chart.

Figure #1: 433 weblogs arranged in rank order by number of inbound links.

The shape of Figure #1, several hundred blogs ranked by number of inbound links, is roughly a power law distribution.

As you can see, this actually isn’t a power law distribution – roughly or otherwise. It’s just a list. These aren’t I/R variables; they aren’t even ordinals. What we’ve got here is a graphical representation of a list of nominal variables (look along the X axis), ranked in descending order of occurrences. We can do a lot better than that – but it will mean forgetting all about the idea that low-link-count sites are in a ‘long tail’, while the sites with heavy traffic are in the ‘head’.

[Next post: how we could save the Long Tail, and why we shouldn't try.]


Get every new post delivered to your Inbox.

Join 241 other followers

%d bloggers like this: