There have been rumblings for a while about the establishment of a UK investigation foundation, and now it’s here. They’re not accepting cash at the moment, just pledges of support and help. So go help them.
Here’s their open letter: Continue reading
On the day that Parliament released MPs’ expenses in their ‘official’ form, I was hawking around on Twitter trying to find a good way to crowdsource analysis of the documents (this was before The Guardian’s crowdsourcing tool went live).
Central to the problem was that the expenses were presented in search-unfriendly PDFs. So I was looking for a place people could upload those PDFs and post comments, tag and annotate them.
Scribd was the obvious option: you can comment and tag – but not annotate. After a number of responses on Twitter (in particular Jen Michaels’ suggestions and Marcelo Soares, who had converted Brazilian parliamentary salaries from PDF to Excel with Able2Extract), I had one from Fred Howell of A.nnotate.
A.nnotate was indeed an ideal candidate – however, the website charges for use, which made it redundant for crowdsourcing purposes. But I was feeling cheeky…
“Perhaps you could let users do #mpexpenses for free as a great bit of PR?” I asked.
Fred saw the potential. Within a couple of hours he had twittered back:
“Put a list of all #mpexpenses pdfs for free shared online annotation at : http://a.nnotate.com/beta/mpexpenses/”
So credit to Fred – and the power of Twitter. Had The Guardian not created their tool, we had hacked together our own platform within hours – and there lies lesson #1: the power of the web to enable people to mobilise very quickly. It also brought to mind something I said to a group of people at an event the same day: don’t obsess with the tools – the networks are more important: because through the networks you should be able to find someone who knows the tools, or how to use them.
The Guardian meant Fred’s efforts were – this week – to no avail. But in the longer term, I know who to turn to if I need a bunch of PDFs annotated – as will anyone else who saw those tweets. And anyone reading this blog post will know about A.nnotate too. So there is lesson #2: it wasn’t the PR of ‘delivering a message’ but simply ‘doing a good turn’, which in social media is the best PR there is.
But I still wish there was a free online PDF upload service that did annotation.
So here’s The Guardian’s crowdsourcing tool for MPs’ expenses. If you’ve not already, you should have a play: it’s a dream. There are over 77,000 documents to get through – and in less than 24 hours users have gone through over 50,000 of those. You wonder how long it took The Telegraph to get that far.
Meanwhile, that process is doing much more than just finding ‘stories’. It’s generating data: the date, the amount, the type of expense, the type of document. When this stage is finished, The Guardian will have a database that will allow people to filter, mix and combine the expenses data in different ways.
It’s also about telling a ‘story’ in a different way. There’s an element of game mechanics in the site – that progress bar (shown above) compels you to bring the site to completion (it strangely reminds me of the Twitter game Spymaster). This makes it more engaging than a made-for-print exclusive – as I wrote about Help Me Investigate, this isn’t ‘citizen journalism’: it’s micro-volunteering. And when you volunteer, you tend to engage.
And when you treat news as a platform rather than a destination, then people tend to spend more time on your site, so there’s an advertising win there.
Finally, we may see more stories, we may see interesting mashups, and this will give The Guardian an edge over the newspaper that bought the unredacted data – The Telegraph. When – or if – they release their data online, you can only hope the two sets of data will be easy to merge.
If you go to one webpage today, go to The Guardian’s liveblog of the results of people rummaging through the MPs expenses released today. It’s crowdsourcing in action.
I’ll write about this in more depth when I have a chance…
Last September in ‘Why fantasy football may hold the key to the future of news‘ I wrote that data was one of the few advantages that news organisations have, and they should be doing more with it. A piece in today’s ReadWriteWeb adds a little commercial stardust to that hypothesis:
“MLB.com’s already wildly successful iPhone app [has] a $10 price tag that sports fans are apparently happy to pay, this could provide a great model for other struggling media to find an important new revenue stream – and not just because it charges for content. “… Any media outlet that can leverage statistics and data visualization as a central part of its coverage would be well served to put those visualizations in an iPhone app and sell it. The iPhone and Android platforms are brilliant for scrolling and zooming through layers of data in ways that print, TV and radio could only dream of. Mobile, touchscreen and hand-held beats a web page on the desktop computer too for data visualization.”
Amen to that.
Whenever I do sessions with people in the industry about online business models I show them The Guardian’s Chalkboard. That for me is a prime candidate for a premium mobile app (assuming there are no licensing issues) – not just because of the data, but because it is social. News orgs take note.
Following today’s landmark judgement on one blogger’s right (or not) to anonymity, I thought it might be useful to post the following tips on maintaining anonymity online.
1. Use an anonymous email account to register your blog. Hushmail is one free service that provides encrypted accounts; RiseUp is aimed at activists; MintEmail gives you a 3 hour temporary email address and FilzMail gives you one that expires after 24 hours. You could also use these to post to your blog via email. Posterous is a great blogging service that allows you to do this.
2. Make sure your IP address isn’t logged when you register or post to the blog. You could use something like Anonymizer or Tor or Psiphon. Other services that mask your IP are listed on this forum.
3. Or you could use an anonymous blogging platform. Invisiblog was one but no longer exists. BlogACause claims to be “anonymous” but I’m trying to find out exactly how UPDATE: here’s how, apparently. In the meantime, this post recommends WordPress and something like Tor.
4. Use a pseudonym that you don’t use anywhere else. If you use a pseudonym, don’t use it on other services as well, as this will make it easier to trace you. If you’re struggling, this Random Name Generator will create one for you.
5. If you’re going to register a domain name do so anonymously with a service like The Online Policy Group.
6. Be careful what information you include. Although police blogger NightJack changed or did not include names in cases he was involved in, the details were specific enough for a journalist to track him down.
7. Don’t win awards. Or book deals. It’s safe to say that a major newspaper would not have been interested in the identities of NightJack or Girl With A One Track Mind if both had remained cult underground heroes. So just pretend you’re sub-literate, OK?
For more information, the following guides go into much more detail:
More links and tips welcome. My Delicious bookmarks on anonymity are at http://delicious.com/paulb/anonymity
Widely lauded anonymous police blogger NightJack has had his identity revealed after The Times took the affair to court.
It’s a cloudy affair. The Times’ angle is that media correspondent Patrick Foster wanted to ‘out’ someone he felt “was revealing confidential details about cases, some involving sex offences against children, that could be traced back to genuine prosecutions” as well as offering “advice to people who found themselves the subject of a police investigation.”
NightJack’s case for preventing the publication of his name was that he would be (and indeed has already been) punished by his superiors.
Mr Justice Eady didn’t buy that, saying: “I do not accept that it is part of the court’s function to protect police officers who are, or think they may be, acting in breach of police discipline regulations from coming to the attention of their superiors.”
The Times also reports him as saying “that even if the blogger could have claimed he had a right to anonymity, the judge would have ruled against him on public interest grounds.”
Hugh Tomlinson, QC, for the blogger, had argued that “thousands of regular bloggers who communicate nowadays via the internet under a cloak of anonymity would be horrified to think that the law would do nothing to protect their anonymity of someone carried out the necessary detective work and sought to unmask them”.
The judge said … the blogger needed to show that he had a legally enforceable right to maintain anonymity in the absence of a genuine breach of confidence, by suppressing the fruits of detective work such as that carried out by Mr Foster.
But Mr Justice Eady said that the mere fact that the blogger wanted to remain anonymous did not mean that he had a “reasonable expectation” of doing so; or that The Times was under an enforceable obligation to him to maintain that anonymity.
There are so many elements to this case it’s difficult to pick them apart.
And that last element is the saddest for me.
With the disappearance of NightJack (his blog has already been deleted*), we lose one more ‘voice on the ground’. While The Times focused on the letter of the law that was being broken, the broader public interest of letting public servants voice their…
…has been ignored.
It is difficult enough to get soldiers to blog, for people to get a genuine feel for the experiences of NHS workers, civil servants and teachers.
And it just got harder.
UPDATE: Curiously, The Times appear to have prevented their reporter from speaking about the issue on Radio 5.
UPDATE 2: A couple of Times journalists have gone on the record with their feelings about the affair.
UPDATE 3: NightJack himself has written a piece in The Times on the story behind the case. Anonymong describes it as “reminiscent of a communist show trial where the accused is allowed to publicly confess their sins and misdemeanors.” But the comments tell a very different story of support.
UPDATE 4: I’ve written a guide to anonymity for bloggers.
UPDATE 5: Via Anonymong: “as noted by Anna Raccoon there is now some precedent for investigating and publishing identifying material relating to a serving police office as prohibited by the counter terrorism act 2008.”
UPDATE 6: As you’d expect, someone has dug into Patrick Foster’s past and come up with some dirt of their own.
UPDATE 7: Fellow public service blogger and ambulance driver Tom Reynolds gives his views on the case. Chicken Yoghurt gives his on the media’s use of anonymous sources. David MacLean responds: “Of course journalists rely on anonymous sources, but if a rival national newspaper found out who was tipping off a competitor, they’d more than likely expose them if the resulting story would be of interest to the public.”. Emily Bell highlights the raft of furious comments on The Times’ Crime Central blog. Gary Andrews gives his take. And Journalism.co.uk round up some more besides.
UPDATE 8 [Jan 24 2012] It seems that Nightjack’s email was hacked in order to get that story.
(h/t Girlonetrack) *Thanks to Martin in the comments: if you type “site:nightjack.wordpress.com” into Google, the pages appear to be cached. Don’t know how long that will last though.
Following the tone set so succinctly by Glyn Moody, I thought I would add my own thoughts on what Sir Tim should say to the government when he bends their ear on transparency.
Firstly, I would second everything that Glyn says.
But I’m going to be cynical and strategic, and urge Sir Tim to emphasise the importance of open data on a couple of areas that are close to the government’s hearts.
You could compare a genuinely significant release of public data to an economic stimulus.
Like cutting VAT, only cheaper.
At minimal cost you could have a new raw material that startups and established media organisations alike could create new value out of. Some of those would create commercial implications far exceeding any revenue generated within government (as research recently suggested in relation to the comparably valuable Ordnance Survey data).
Repeat after me: jobs and money, jobs and money.
Samuel Butler’s Erewhon puts it particularly well:
You will sooner gain your end by “appealing to men’s pockets, in which they have generally something of their own, than to their heads, which contain for the most part little but borrowed or stolen property”
Public sector spending is going to drop whichever party is in power. Let’s play to that.
By opening up public data the government will effectively be able to pass on some development costs to willing volunteers who mash up the data in their own ways. The difference is that people will do this to their own agendas and for their own benefit.
But more importantly, the results of this experimentation – if supported and encouraged – should produce work that makes it more efficient to interact with public data and therefore public bodies. If I can use a slider to find out which schools are within 3 miles, that saves 20 minutes of someone answering a phonecall in the local education department. If I can have a Facebook app which tells other users how much money alcohol abuse is costing my local hospital, it might save the NHS a bob or two. You get the picture.
The limited data that’s available in the UK is an embarrassment. Imagine what MySociety could do with what’s available in the US.
Likewise, for all the talk of transparency, the recent announcement that Cabinet Papers and information relating to the Royal Family would be exempt from the Freedom of Information act is a backward step. Heather Brooke’s concerns proved right.
The cynic in me sees the appointment of Berners-Lee as an action intended to generate the illusion of movement – “We’re working on it”. But the Freedom of Information act is possibly the most positive contribution the Labour government has made to this country’s political health since it came to power, and not to follow through on promises made would be an enormous political mistake.
So I will add one request to my advice above: I would stress that any discussion of transparency acknowledges the importance of requiring any organisation using public funds to make their data public too. So much public work is outsourced to the private sector that it is particularly difficult to see whether public money is spent responsibly.
More at Podnosh, BBC, Emma Mulqueeny, Simon Dickson and Amused Cynicism.
You may know about The Guardian’s Datastore: a compilation of “publicly-available data for you to use free” that’s been around for a few months now. You know the sort of thing: university tables; MPs’ expenses; tax paid by the FTSE 100.
It has already produced some great work from what I once described as the “Technician” variant of distributed journalism.
But a column by Charles Arthur recently was the first example I’ve seen of Datastore being used for, well, more ordinary data – the sort of information journalists deal with every week. Here’s how it appeared in print:
“I dug up the figures from the UK music industry: the British record industry’s trade association (the BPI), and the UK games industry (via its trade body, Elspa) as well as the DVD industry (through the UK Film Council and the British Video Association). The results are over on the Guardian Data Store (http://bit.ly/data01), because they are the sort of numbers that should be available to everyone to chew over.
“What did I find? Total spending has grown – but music spending is being squeezed. The games industry – hardware and software – has grown from £1.4bn in 1999 (the year Napster started, and the music business stood rabbit-transfixed) to £4.04bn in 2008. That’s 12% annual compound growth. You’d kill for an endowment like that. Even DVD sales and rental take a £2.5bn bite out of consumers’ available funds, double that of 1999.
“So the music industry’s deadliest enemy isn’t filesharing – it’s the likes of Nintendo, Microsoft and Sony, and a zillion games publishers.”
That link (which frustratingly isn’t active in the online article) takes you to a Datablog post by Arthur which in turn links to a rather simple spreadsheet.
And it’s the simplicity that I think is important.
It’s one thing to link to huge datasets that benefit from lots of eyeballs looking for stories, or perming the data in different ways.
But it’s something else to link to the more everyday figures journalists deal with; to show your sums, in short.
Is this a natural extension of the blogging culture of linking to your sources? I think it is. And the more journalists get used to publishing their work on the likes of Google Spreadsheets, the better journalism we will get.
So why aren’t more journalists doing it? And why aren’t more news organisations providing a place for them to do it? Or are they? I’d love to know of any other individual or organisational examples.
As traditional media outlets close down, the relative importance of non-market players becomes more important.
Governments around the world were quick to see the opportunities for their news agencies. From Xinhua (China) to ITAR-TASS (Russia), from AFP (half of its budget comes from state subscriptions) to Voice of America, governments are trying to shape the world’s public opinion.
The coverage of Gaza by Al Jazeera is a case in point. They produced quality journalism no other outlet could dream of. Now, viewers should keep in mind that money for such newsgathering comes straight from the pocket of the Emir of Qatar. Believe me, I’m sure Al Jazeera’s journalists keep that in mind too.
To help you measure the amount of government-funded journalism, I built this little app, I smell a government rat in my news. Just type in any query and you’ll see the share of articles produced with state funds. Continue reading
Online Journalism Blog 