If you want to skip the background, go to the next subheading

Last week the BBC Education website published a piece about a report into the use of technology by schoolchildren: “Tech addiction ‘harms learning’”:

“Technology addiction among young people is having a disruptive effect on their learning, researchers have warned,” the intro led, before describing the results of the study. No one other than the study authors was quoted.

But GP and Clinial Lecturer AnneMarie Cunningham, hearing of the report on Twitter, felt the headline and content of the article didn’t match up: “The headline suggests a causal relationship which a cross-sectional study could not establish, but the body of the text doesn’t really support any relationship between addiction and learning”, she wrote, and she started digging:

“It … was clear that none of the authors had an education background. The 2 main authors, Nadia and Andrew Kakabadse, have a blog showcasing their many interests but education doesn’t feature amongst them. They descibe themselves as “experts in top team and board consulting, training and development”.”

AnneMarie bought the report for $24.99 – the only way to read it – and started reading. This is what she found:

“I expected the report by university academics to follow a standard format but it doesn’t. It is 24 pages long and contains no references and no appendices. The survey instrument is not included.

“No response rate is given … ‘tech addiction’ … seems to have been a self-assessment … With regards to this addiction harming learning, there is no analysis relating to the perception of being addicted to outcomes in learning. In fact very few of the questions are related in any way to learning.

“It is hard to understand several sections of the report because of lack of access to the questionnaire. For example, with regards to plagiarism the authors state that “A high proportion of students (84.3%) openly admitted that they inserted information from the Internet into their homework or projects on a number of occasions.” The tone of this sentence reflects some of the bias which is found throughout the work. The authors don’t seem to be aware that if referenced it is acceptable to insert information from the internet into work, so the students would have no reason to be ashamed and fear ‘openly admitting’ this.

“… It is also reported that 28.5% of students “feel it acceptable to insert information from the Internet straight into schoolwork without editing or making adjustment, recognising that such behaviour is considered plagiarism.” It would help a lot to see how that question was actually worded in the survey, as in the figure it is simply represented as “Ok to “insert” information from the Internet straight into schoolwork- Yes/no”. That’s not quite the same!

“There is no analysis relating amount of time spent online (or perception of addiction) and likelihood to insert internet contents into work without reading it. It may be that those who spend less time online, have less skills in information literacy and are more likely to plagiarise.

“In summary this report tells us very little about internet addiction or learning. Do you think that someone writing for the BBC website actually read the report?”

Her blog post summarising this was passed around by numerous people on Twitter, including BBC journalists, journalism academics, and ‘bad science’ Twitter users like EvidenceMatters. The blog post itself amassed around 20 comments.

AnneMarie emailed BBC Education, and I emailed the BBC News website education editor, Gary Eason. AnneMarie also emailed Cranfield University (who were also criticised). After a week, we had had no response and the article remained unchanged.

But more importantly, people’s opinions of the ‘bad science’ reporting of the BBC remained unaddressed – and this is what this blog post is really about.

UGC and the news brand

Here’s why this interests me. Last year Alfred Hermida and Neil Thurman published a paper (PDF) on user generated content which included the following finding:

“The potential that UGC has to damage a newspaper’s brand remained a prevailing concern among some editors. The idea of publishing a comment without checking it first was described as “very dangerous” (Avery, 2006), while Bale (2006) said that not to moderate content would be an inappropriate brand risk.”

What the discussion around the BBC Education news piece highlights is the risk to a news brand in not publishing comments (as is the case – for now – on most BBC News reports. Indeed, I would add that not having bylines to all reports or contact emails makes the organisation look even more opaque.)

Today I chased up Gary Eason, and this was his response:

“So far as I know our reporter did obtain a copy of the report. I heard her talking to an author/authors.

“I wouldn’t claim it is the greatest piece of original journalism we’ve ever undertaken nor that the study was a groundbreaking one. It’s clear the report was based on a small survey, which feeds into an ongoing debate. We have done pieces in the past presenting a different perspective, e.g. http://news.bbc.co.uk/1/hi/education/6247853.stm, http://news.bbc.co.uk/1/hi/education/6241517.stm, http://news.bbc.co.uk/1/hi/education/7196836.stm.

“I’ve read the blog. “U may fault methodology, results true” is amusing but cuts both ways: it seems to me the results don’t fit her world view so she sets about rubbishing them. Is she seriously arguing that ‘cut-and-paste plagiarism’ is not a problem?”

Speaking to Gary further, he said that he was aware of some of the criticisms but does not tend to address online discussion unless they were libellous towards his journalists, “otherwise I’d spend all day doing something else”. He said he would re-read the blog post.

Of course having comments on the story would have allowed this discussion to take place in public, from the start, and provide readers of the article with some critical context, turning a single-source ‘He Said’ article into a ‘He Said-She Said’ piece at the very least. That’s a technical issue that is being addressed, but in the meantime the BBC brand suffers.

The phrase “A lie travels round the world, while Truth is putting on her boots” in this case applies both to the study that was being reported, and the belief that the BBC journalist hadn’t actually read it. Having and watching comments allows Truth to get her boots on that much quicker.

PHP Freelancer