If you want to skip the background, go to the next subheading
Last week the BBC Education website published a piece about a report into the use of technology by schoolchildren: “Tech addiction ‘harms learning'”:
“Technology addiction among young people is having a disruptive effect on their learning, researchers have warned,” the intro led, before describing the results of the study. No one other than the study authors was quoted.
But GP and Clinial Lecturer AnneMarie Cunningham, hearing of the report on Twitter, felt the headline and content of the article didn’t match up: “The headline suggests a causal relationship which a cross-sectional study could not establish, but the body of the text doesn’t really support any relationship between addiction and learning”, she wrote, and she started digging:
“It … was clear that none of the authors had an education background. The 2 main authors, Nadia and Andrew Kakabadse, have a blog showcasing their many interests but education doesn’t feature amongst them. They descibe themselves as “experts in top team and board consulting, training and development”.”
AnneMarie bought the report for $24.99 – the only way to read it – and started reading. This is what she found:
“I expected the report by university academics to follow a standard format but it doesn’t. It is 24 pages long and contains no references and no appendices. The survey instrument is not included.
“No response rate is given … ‘tech addiction’ … seems to have been a self-assessment … With regards to this addiction harming learning, there is no analysis relating to the perception of being addicted to outcomes in learning. In fact very few of the questions are related in any way to learning.
“It is hard to understand several sections of the report because of lack of access to the questionnaire. For example, with regards to plagiarism the authors state that “A high proportion of students (84.3%) openly admitted that they inserted information from the Internet into their homework or projects on a number of occasions.” The tone of this sentence reflects some of the bias which is found throughout the work. The authors don’t seem to be aware that if referenced it is acceptable to insert information from the internet into work, so the students would have no reason to be ashamed and fear ‘openly admitting’ this.
“… It is also reported that 28.5% of students “feel it acceptable to insert information from the Internet straight into schoolwork without editing or making adjustment, recognising that such behaviour is considered plagiarism.” It would help a lot to see how that question was actually worded in the survey, as in the figure it is simply represented as “Ok to “insert” information from the Internet straight into schoolwork- Yes/no”. That’s not quite the same!
“There is no analysis relating amount of time spent online (or perception of addiction) and likelihood to insert internet contents into work without reading it. It may be that those who spend less time online, have less skills in information literacy and are more likely to plagiarise.
“In summary this report tells us very little about internet addiction or learning. Do you think that someone writing for the BBC website actually read the report?”
Her blog post summarising this was passed around by numerous people on Twitter, including BBC journalists, journalism academics, and ‘bad science’ Twitter users like EvidenceMatters. The blog post itself amassed around 20 comments.
AnneMarie emailed BBC Education, and I emailed the BBC News website education editor, Gary Eason. AnneMarie also emailed Cranfield University (who were also criticised). After a week, we had had no response and the article remained unchanged.
But more importantly, people’s opinions of the ‘bad science’ reporting of the BBC remained unaddressed – and this is what this blog post is really about.
UGC and the news brand
Here’s why this interests me. Last year Alfred Hermida and Neil Thurman published a paper (PDF) on user generated content which included the following finding:
“The potential that UGC has to damage a newspaper’s brand remained a prevailing concern among some editors. The idea of publishing a comment without checking it first was described as “very dangerous” (Avery, 2006), while Bale (2006) said that not to moderate content would be an inappropriate brand risk.”
What the discussion around the BBC Education news piece highlights is the risk to a news brand in not publishing comments (as is the case – for now – on most BBC News reports. Indeed, I would add that not having bylines to all reports or contact emails makes the organisation look even more opaque.)
Today I chased up Gary Eason, and this was his response:
“So far as I know our reporter did obtain a copy of the report. I heard her talking to an author/authors.
“I wouldn’t claim it is the greatest piece of original journalism we’ve ever undertaken nor that the study was a groundbreaking one. It’s clear the report was based on a small survey, which feeds into an ongoing debate. We have done pieces in the past presenting a different perspective, e.g. http://news.bbc.co.uk/1/hi/education/6247853.stm, http://news.bbc.co.uk/1/hi/education/6241517.stm, http://news.bbc.co.uk/1/hi/education/7196836.stm.
“I’ve read the blog. “U may fault methodology, results true” is amusing but cuts both ways: it seems to me the results don’t fit her world view so she sets about rubbishing them. Is she seriously arguing that ‘cut-and-paste plagiarism’ is not a problem?”
Speaking to Gary further, he said that he was aware of some of the criticisms but does not tend to address online discussion unless they were libellous towards his journalists, “otherwise I’d spend all day doing something else”. He said he would re-read the blog post.
Of course having comments on the story would have allowed this discussion to take place in public, from the start, and provide readers of the article with some critical context, turning a single-source ‘He Said’ article into a ‘He Said-She Said’ piece at the very least. That’s a technical issue that is being addressed, but in the meantime the BBC brand suffers.
The phrase “A lie travels round the world, while Truth is putting on her boots” in this case applies both to the study that was being reported, and the belief that the BBC journalist hadn’t actually read it. Having and watching comments allows Truth to get her boots on that much quicker.
Online Journalism Blog 
Providing “readers of the article with some critical context” by opening it up to comments may involve much more of fundamental issue than just a technical issue, otherwise they would have opened it up already. Clearly this example should prove to the BBC that they can no longer rest on their laurels as a trusted brand. Trust *is* the new black. People don’t trust brands, people trust people.
This is happening with gay news. The Pink Paper stopped printing and went online only but because their online offering is so bad and you have to sign in to comment it attracts none. pinknews.co.uk by contrast gets lots and – I notice – a huge number of RTs showing its better known in social media.
Please, check this story: http://www.universitytorontofraud.com and check the London-based Committeee on Publication Ethics – http://www.universitytorontofraud.com/committee.htm This won’t be ‘He Said-She Said’, it is all documents.
Oh, dearie me. Your response from Gary was considerably longer than mine which read:
“Hi Anne Marie
Thank you for your thoughts.
The author of the article did have the whole report in front of her and interviewed one of the authors. I do not agree that our headline is “sensationalist”.
best wishes
GE”
From his comments to you it sounds like he completely missed the point of the blog.
In any case, through the use of google side-wiki I have been able to comment on the BBC article. http://bit.ly/Z8Mfw
When I get some time I have to write more about this.
Anne Marie
You haven’t established your conclusion here. It’s not lack of comments that is the problem – rather, it’s lack of engagement. You can have engagement without comments, and you can have comments without engagement. The BBC does have a comments system on it’s news site – Have Your Say. And it’s a total waste of time, and filled with the sort of people who you find on street corners proclaiming the end of the world. Rather, what BBC News need is to have someone who will take responsibility for factual inaccuracies and other editorial problems (and I’d say that the example you’ve given is a perfect example of a glaring editorial problem – I mean, the problems that Anna-Marie Cunningham brings up are significant problems with the study that anyone who has read a few scientific papers should be able to spot a mile off – no references, dodgy methodology etc. This isn’t about “worldview” as the BBC guy said, this is about basic competence – the academic equivalent of knowing whether a source you are interviewing for a story is a flaky drunken rent-a-quote or someone who actually knows what’s going on).
Adding comments doesn’t change much. In fact, it makes most news sites less interesting for me: I couldn’t give a monkeys what Ray from Bradford or whoever thinks about some story unless he has some evidence or reasoning that shows that the story is wrong somehow. The Guardian would lose no value at all if they turned Comment is Free off. Same for BBC and HYS. All they do is make pages take longer to load.
For the commercial news sites, comments just mean page views. What’s more important is making sure that when there is a factual error, the feedback loops of the blogosphere and other Internet sources, as well as e-mail and so on, are monitored and responded to appropriately. In most cases, that means ignoring them – except for when they point out some fallacious reasoning or evidence that contradicts the story, and then listen to them and don’t just dismiss them as being driven by their “worldview”.
The same benefits you think a comments area gets would also be served by having a banner at the bottom that says “Have we made a mistake? E-mail news.mistakes@bbc.co.uk and we’ll fix it!” Only it would have the advantage of not being filled with idiots. There’s a whole Internet for people to put their stupidity – it doesn’t need to go on the news websites themselves.
You’re right. I said “Having and *watching* comments” – meaning not necessarily having them on your site but engaging with them wherever they are.
That said, even having comments could provide more context than letting the story sit alone. The quality of those comments depends on how they’re managed, which is a separate blog post entirely, but it’s safe to say how you engage with those is a key part of that. Having seen the inside of The Guardian’s community management operation I’d say they do this very well.
It would have been interesting to see what kind of comments this arose from this story. But I would be happier if the story had not appeared in the first place. My feeling is that the report it was based on was very weak and did not justify the coverage it received.
In any case here is my blog response to this: http://wishfulthinkinginmedicaleducation.blogspot.com/2009/09/bbc-responds.html
I think Tom Morris has hit the nail on the head here. The kind of populist “Have your say” commenting system has no value at all in holding editors to account for their actions. For the editor to come back with comments attempting to discredit the original blog poster (Anne Marie Cunningham) is disingenuous, to say the least. He made no attempt to address her actual criticisms. Anne Marie did not suggest at all that plagiarism as acceptable. What she suggested was that without the methodology and questionnaire backing up the research, the piece was worthless because it was impossible to assess the evidence presented.
On the point that allowing/not allowing comment might damage your brand I’m less sure about. The one thing above all else that will damage any brand is attempting to rubbish (and silence) criticism, as people can see that for what it is: bullying. And nobody likes a bully.
What this actually shows is the shocking weakness of the BBC’s news operations. There is precious little reporting, and a lot of repackaging news releases and PR articles. When you do catch them doing this [as in this case] they get exceptionally defensive: given that, do you really think they’re going to want more comments?
At the senior level, they want more comments. How that plays out comes down to strategy and training. It’s a culture shift.
A reference to Speak Your Branes seems appropriate here.
http://ifyoulikeitsomuchwhydontyougolivethere.com/