Tag Archives: guidelines

If we are using AI in journalism we need better guidelines on reporting uncertainty

Chart: women speak 27% of the time in Game of Thrones

The BBC’s chart mentions a margin of error

There’s a story out this week on the BBC website about dialogue and gender in Game of Thrones. It uses data generated by artificial intelligence (AI) — specifically, machine learning —  and it’s a good example of some of the challenges that journalists are increasingly going to face as they come to deal with more and more algorithmically-generated data.

Information and decisions generated by AI are qualitatively different from the sort of data you might find in an official report, but journalists may fall back on treating data as inherently factual.

Here, then, are some of the ways the article dealt with that — and what else we can do as journalists to adapt.

Margins of error: journalism doesn’t like vagueness

The story draws on data from an external organisation, Ceretai, which “uses machine learning to analyse diversity in popular culture.” The organisation claims to have created an algorithm which “has learned to identify the difference between male and female voices in video and provides the speaking time lengths in seconds and percentages per gender.”

Crucially, the piece notes that:

“Like most automatic systems, it doesn’t make the right decision every time. The accuracy of this algorithm is about 85%, so figures could be slightly higher or lower than reported.”

And this is the first problem. Continue reading

Advertisements

Guest post: Do we need moderation guidelines for dealing with mental health issues?

Last month the Press Complaints Commission made a judgement in a case involving discriminatory comments on a newspaper article. The case highlighted the issue of journalism on mental health and how it is treated by publishers alongside similar considerations such as sexuality, gender, religion and ethnicity. The complaint also led to a change in The Guardian’s moderation rules.

In a guest post for the Online Journalism Blog the person who brought that case, Beatrice Bray, writes about her experiences of comment abuse, and the role she feels publishers should take in dealing both with comments relating to mental health, as well as writers with mental health issues.

Last April I wrote a rallying cry for the Guardian for all who have endured taunts about mental ill health. In my reply article Cartoonists should be careful how they portray mental health (23/4/10) I reclaimed the word “psychotic”. Guardian cartoonist Martin Rowson had used the word to abuse Mrs Thatcher. I put him right.

I am a long-standing reader of the Guardian newspaper but I did not know the website audience. Being a proud campaigner I told Guardian readers that I had bipolar disorder and had experienced psychosis.

I expected a civil hearing. Newspaper readers did oblige but many online readers were foul.

The Guardian’s managing editor Chris Elliott did not warn me about the impending abuse. That was a mistake. I think Mr Elliott knew I would face hostility but I do not think he realised how badly I would be hurt.

Those insults made me physically sick. My head was sore for many weeks. This was all so pointless. If Mr Elliott had given me a chance to discuss the risks involved we both could have taken precautions. Instead there was a row.

Guardian staff gave me an apology but told me to grow a “thick skin”. That jibe spurned me into going to the Press Complaints Commission. It is free. It is also less adversarial and less costly than a disability tribunal.

I was not asking for anything unprecedented. The BBC has guidelines on working with vulnerable people. We need to extend this to new media.

Working with vulnerable people

For example when dealing with discussion sites moderators need to deal swiftly with abuse. They also must facilitate discussions so that they do not turn nasty.

Staff should appreciate the reasons for this action. This is not prima donna treatment. This action is necessary because the writer and many of the readers share a common disability. They all have mental health problems.

Section 2 of the PCC Editors’ code promised fairness to complainants. I thought it only fair to ask for warning of abuse but in my PCC ruling the Guardian and the PCC disagreed with me. The PCC did not say why.

However, I did score other points.

Before the PCC ruling the Guardian at my request did add the word “disability” to its moderation rules.

The PCC and the Guardian and did apologise with regard to the abuse.

Guardian online readers called me, amongst other things, a “nutter” and a “retard”. Unfortunately both the Guardian and PCC refused to accept that this was discrimination as defined by the terms of section 12 of the Editor’s code of the PCC.

This is not just semantics. To me the word “discrimination” is a word with power. It holds the abuser responsible but the PCC fights shy of doing that online.

I now know that you can only complain to the PCC if a staff member makes a discriminatory remark about you. Comments made by non-staff members do not fall within the PCC’s remit. My abusers were not Guardian staff.

It is a shame. By being discrimination deniers both the Guardian and the PCC cut themselves off from a store of knowledge on handling disability and mental health in particular.

BBC new linking guidelines issued – science journals mentioned

The BBC have just emailed new linking guidelines to their staff. They stipulate that linking is “essential” to online journalism and in one slide (it’s a PowerPoint document) titled ‘If you remember nothing else’ highlight how linking will change:

What we used to do…

  • Lists of archive news stories
  • Homepages only on external websites
  • No inline linking in news stories

What we do now – think adding value…

  • Avoid news stories and link to useful stuff – analysis, explainers, Q&As, pic galleries etc
  • On external websites look beyond homepage to pages of specific relevance
  • Inline linking in news stories is OK when it’s to a primary source

Other points of note in the document include the repeated emphasis on useful deep linking, and the importance of the newstracker module (which links to coverage on other news sites). Curiously, when referring to inline links it does say that “different rules can apply” to BBC blogs – “speak to blogs team if in doubt”.

Something I did look for – and find – was a reference to linking to scientific journals. And here it is: “In news stories inline links must go to primary sources only– eg scientific journal article or policy report (1 or 2 per story; avoid intro)”

This is significant given the previous campaigning on this issue.

On the whole it’s a good set of guidance – I’ll refrain from publishing it in hope that the BBC will…

UPDATE: It seems The Guardian followed up the story and embedded the document, so here it is:

BBC guidelines for linking – Sept 2010

The end of objectivity – web 2.0 version

paul bradshaw's facebook network

This week a new nail was driven into the coffin of the notion of journalistic objectivity. The culprit? The Washington Post’s leaked social media policy.

The policy is aimed at preserving the appearance of objectivity rather than its actual existence. It focuses on what journalists are perceived to be, rather than what they actually do.

And in doing so, it hits upon the very reason why their attempt is doomed from the start: Continue reading