Lies, damned lies and Ofcom 

10 August 2009 tbs.pm/1086

Ofcom Communications Market Report results “are essentially meaningless”

In a post today, media correspondents official and unofficial are roundly taken to task by noted social media consultant Joanne Jacobs, regarding their writing about the Ofcom Communications Market Report. Saying that more young people are “staying away from social media”, she says, demonstrates an inability to read statistics.

She puts the record straight. “What the research showed was 5% fewer 15-24 year olds claimed to have a social networking profile when compared with the previous survey, but that the proportion of 15-24 year olds from the AB socio-economic group claiming a profile has risen (up from 29% to 35%) while there was a substantial drop among DEs from 32% to 19%.”

She goes on, “…the only true finding in the research is that there are more 15-24 year olds in poor families who are not setting up a profile on a social networking site. Among financially secure young people, social networking use is still growing.”

She concludes, “Frankly, this is a result which says far more about the digital divide than anything else.”

All good stuff. However, not only are there problems of interpretation of Ofcom’s data: there are problems with the data themselves. In her third paragraph, Jacobs reports that Ofcom claims that “both unique visits and time spent online [are] actually still growing, except for Second Life which has experienced an overall drop in number of hours per month per user”.

Where does that information on the virtual world of Second Life come from? Well, amongst other places, from page 292 of the report.

Here, Ofcom indeed suggests “the biggest change was for virtual world website, Second Life, whose users now spend an average of nine hours per month on the service, down from a recent high of nearly 28 hours”. The figures appear to fall off rapidly and unexplainedly during October 2008. The apparent peak is in September, at 22 hours – it had been rising steadily since June. Then there’s a steep fall to 18 hours and by November it’s allegedly just 4hrs, while thereafter there’s another steady climb.

We’ll leave aside the fact that while Second Life has a web site, Second Life isn’t a web site, any more than Intel Corporation is a microprocessor – so if these figures relate to visitors to the Second Life web site they are already a complete waste of time, as you generally only go there if you have a problem. Instead, let’s look at this data, for “time spent online, per UK user”. It appears significantly out of line with other published figures for users worldwide.

“Residents spent about 41.5 million hours total in Second Life in January [2009], as opposed to the 28.3 million hours users spent in the virtual world the same month in 2008″ (Second Life Finding New Life, Sydney Morning Herald, March 2009)

So in fact, while Ofcom suggest that UK usage fell significantly, for the world overall, user hours rose dramatically, up from 28.3m hours a year ago to 41.5. Ofcom’s figures are indeed rather a surprise: British users are some of the most active in the world after those in the United States. Here’s another report:

“Linden Lab [creators of Second Life] recently delivered its 2008 usage results, which peg user hours up 61% over 2007 for a grand total of nearly 400 million hours total…. Linden Lab lists its total residents at 16,785,351 as of this posting. Roughly half a million of those were logged-in during the last seven days alone. What’s more, peak concurrent users — the number of people playing at the same time — increased 31% over 2007.” (Second Life User Hours, Dollars Spent, Skyrocket in 2008, ABC News, January 2009)

There are plenty of other references which tell a similar story: Second Life usage, after hitting a plateau in 2007, is now rising steadily again. Other indicators, such as in-world commerce, have been rising significantly too, and today you’ll often find almost 100,000 residents online at any one time, a growing number of them from the UK.

The discrepancy between Ofcom’s report and everyone else’s findings on the success of Second Life is rather surprising, as the EU as a whole and the UK in particular are significant user bases for the virtual world outside the US. So what’s going on here? Here’s the Ofcom data, in the form of a graph (Ofcom Communications Market Report, p. 292):

Time spent online, per UK user, on selected member communities. The line that plunges significantly in October 2008 is Ofcom's alleged Second Life usage data.

The answer lies in the small print under the graph, which credits Neilsen Online for the data, and adds: “Note: Due to a change in methodology, use caution for comparisons pre October 2008.” So there we have it: the apparent dramatic fall coincides exactly with a “change in methodology”.

While the graph includes the disclaimer, the report text itself takes these figures as accurate. As Joanne Jacobs put it in a Tweet, “…they say you shouldn’t read stats prior to October, which means their results are essentially meaningless.” Sounds like Ofcom staff need to learn to read statistics, too – in this case, their own.