Matt Burgess, the man behind FOI Directory (and the former editor of Help Me Investigate Education) has launched a new weekly email newsletter providing regular updates on developments in Freedom of Information and transparency. Continue reading
Groundhog image by Alessandro M
In a guest post for OJB, Damian Radcliffe argues that the need for policy makers to support hyperlocal publishers is stronger than ever – and explains just how that support can happen.
When I first started reporting on hyperlocal media in 2009 it was against a daily backdrop predicting the death of newspapers and clarion calls for public intervention to save this vital resource.
Since then, this hysteria has died down, although it’s clear that many of the structural challenges being faced by the local media sector have not gone away.
In January Press Gazette reported that there had been a net reduction of 181 UK local newspapers since 2005, including a further 11 lost this year, whilst a leaked memo from Trinity Mirror shed light on the commercial pressures many newspapers groups face and how this is influencing reporting on the ground.
Despite this, the UK’s industrious hyperlocal media sector continues to beaver away. Continue reading
This is an extract from Finding Stories in Spreadsheets. Get the book here.
The ebook version of this tutorial includes a dataset and exercise to employ these techniques.
Right at the start of my book on Excel for journalists I talk about sorting data to find out which values come top or bottom. However, there is a family of functions which will give you a lot more control in finding out not just who is top or bottom, but the rank of any value in any series of values.
This is particularly useful if you want to compare ranks.
Many stories are based on finding out where your own country or region ranks in the latest data
Ranking isn’t just about statistics – it can be used in consumer stories too
For example, say you had a table showing school performance across the last two years.
Each table shows the percentage of pupils achieving the top grades in that year. You can use RANK to find out what rank each percentage would have placed the school in for each year. Continue reading
Data journalists will always need help. Image by Widhi Rachmanto
When people talk about data journalism the emphasis is almost always on the technicalities of the role: visualisation tools and spreadsheet formulae; scraping and cleaning; coding and mashing.
But data journalism isn’t just a technical skill – it is a cultural skill too.
Let me explain what I mean. If you were to list the technologies involved in data journalism you might start with Excel or a similar spreadsheet tool. Then add Open Refine for cleaning. Some scraping tools. Mapping tools. Some tools for charts, and infographics. Some understanding of HTML and CSS will help. Also XPath, SQL, regular expressions. JavaScript, Python or Ruby or PHP. R probably too… I could go on.
If those technologies sound like too much for one person to master all at once, you’d be right. They are.
So how do data journalists get the job done? They collaborate.
They use sites like CodePen, Stack Exchange and GitHub, where others can build on your work – and you can build on the work of others. They contribute to mailing lists; they share resources; and they work with a range of other individuals and groups.
It is an open approach to reporting that borrows more from the culture of programming than journalism’s own culture of guarding information jealously.
And understanding that culture is, for me, one of the first steps to becoming a successful data journalist.
For example, notice my choice of words in the sentence two lines earlier: “contribute to”; “share”; “work with”. Sometimes journalists can make demands of communities of web developers that betray an exaggerated sense of their own importance, and an ignorance of the environments that developers often work in.
Those journalists are often given short shrift as a result of their clumsiness and lack of empathy.
If journalists were the gatekeepers of the 20th century, programmers are the gatekeepers of the 21st.
We no longer need journalists to get information to an audience; but we do need programmers to connect different parts of the networks we operate in.
Recognising this is so important that I’ve codified the requirement for understanding in my data journalism teaching at Birmingham City University and City University London.
Students at BCU on the MA in Online Journalism, for example, are required to engage with – and contribute to – wider communities of practice.
That means sharing what they learn, curating useful discussions in the community, interviewing key individuals and researching problems and questions that are important to that community.
The intention is twofold: firstly to embed good habits as a member of that community. And secondly to position them so that they are able to continue to learn not just while they are on the course, but after they graduate, as technologies and practices continue to develop.
A final difference is also important to highlight: journalists and programmers have different learning cultures.
One of the questions I am asked most often by aspiring data journalists is “What should I learn first?” My response is: “What you need to for the story you’re doing right now. And if that’s too much, then pick a simpler story then work up from there.”
If you think you learn to be a data journalist by doing Codecademy or reading a book on Python, you are likely to end up frustrated. It can be helpful – but it’s neither effective nor efficient.
The learning culture of the programmer is much more piecemeal, strategic, and reliant on others.
So I would never advise a journalist to learn a particular programming language for the sake of it. Instead learn some basic concepts in programming, such as variables, data types, loops and if/else tests, and then search the web for code that solves the problem you’re trying to solve, whether that’s “making a chart in JavaScript” or “scraping a spreadsheet in Python” or “Excel function to extract a year from a date”.
Often the next step will be a case of copying and pasting someone else’s code, and changing it slightly to see what works.
That might feel like plagiarism to a journalist, but to a programmer it is simply standing on the shoulders of giants.
Equally, if you’re trying things out in programming they often don’t work first time.
Again, that can feel like failure if you come from a humanities background. But look at it more like science: experimentation, trial and error are part of the process. In fact, programming is essentially a process of working with failure: diagnosing it, looking for solutions, and trying them with a vague expectation that it might not work.
I realised that I had learned this culture when some code of mine worked first time – and I was not only surprised – I was also vaguely disappointed. “Oh. It works. What do I do now?”
Journalists often straddle two cultures: the sports reporter has to connect with fans, players and management; the health reporter with both doctor and patient.
In data journalism we have to draw on the same skill: only it’s not just our audience we’re connecting with, it’s the people who make those connections work.
By Agustin Palacio
Quotes lie at the heart of what journalists do. It is often what makes news ‘new’; they are the ingredients of the ‘national conversation’ that journalism seeks to host. Now one site is seeking to provide an overview of that conversation.
Launched in March as a beta, Trooclick tracks quotes from news sites and social media and classifies them by topic, showing on one screen who is saying what and whether their comments are ‘positive’ or ‘negative’.
Its creators believe that it provides a more neutral account of current events due to a greater variety of sources than anything else currently available. Community manager Paul Nolan said:
“Our aim is to provide an alternative to article-based journalism that can be used by the general public.
“Why read a handful of articles when you can get a cross section of every opinion on a breaking news story at a glance? We think that telling stories through articles needs to be challenged and we are providing an alternative.”
David Higgerson writes about some depressing recent developments – and equally depressing wider trends – around the lack of transparency in public office and public spending. It’s worth reading:
“The reason this is so important now is because we are on the cusp of another wave of political restructuring. Devolution is on its way to Greater Manchester, and to other major city regions too. Whether you believe this is a good thing or not, there is hopefully no denying that with such major power moves there has to also be a cast-iron guarantee that those making decisions will be accountable.”
And here’s the background:
Magazines should be investing more effort in developing their online communities because they represent an increasingly important “strategic resource,” according to new research.
The research, “Audience Community as a Strategic Resource in Media Work” by Nando Malmelin and Mikko Villi, identifies two main advantages that publishers gain when working actively with their audience community:
The researchers looked in particular at two successful magazines in Finland: Demi (aimed at 12-19 year old girls) and Lily (aimed at women aged 18-39).
The magazines’ two websites reach 75% and 45% of their respective target audiences, and benefit from highly active communities that actively feed into the editorial process.
As one interviewee puts it:
“If we didn’t have a community producing contents and subjects that they themselves are interested in 24/7, we wouldn’t be able to keep up to speed on what’s important to our target audience.”
But collaborating with audience communities requires new kinds of journalistic practices, and the researchers identify 4 different roles that journalists adopt in this respect:
The observer monitors the audience community’s discussions to identify interests, needs and concerns.
The developer helps improve the online platform(s) and service so that users are more likely to contribute.
The facilitator helps start and maintain online discussions and feed those back to the editorial team. The emphasis is on stimulating, rather than driving, as one journalist points out:
“We can’t interfere too much. The biggest mistake we could make would be to decide amongst ourselves what we like at the moment and what other professionals respect and what’s in vogue for our industry, that’s perhaps the pitfall we fall into every now and then.”
Finally, the curator might highlight the best work by members of the community, both online and in print.
Magazines are typically in a better position than newspapers when it comes to developing audience communities, and readers of speciality publications (for example hobby magazines) tend to have a stronger attachment to the brand than generic magazines – but that doesn’t mean that all magazines have a community:
“When audience members have more regular communication amongst themselves, they can be said to form an actual community; otherwise atomized media consumers simply form a crowd.”
The key question for publishers, then, may be to what extent that communication between users exists – and where.
Community management specialists such as Richard Millington tend to warn against the tendency to launch technically impressive platforms without consideration of the cultures of communities themselves. Sometimes it makes more sense to participate in existing communities than try to recreate them (another option, taken by some publishers, is to buy blog networks).Meanwhile Malmelin and Villi settle for recommending that:
“Media organisations should invest increasing effort in creating and managing audience communities … the successful operation of the media industry should in many cases be in fact as much about content production as it is about facilitating the maintenance of social relations among and with its audience.”
A month ago I blogged an extended version of a chapter I was invited to write for an edited collection by the Political Studies Association.
That collection is now out. It features over 70 contributions on everything from the role of social media in the election (including specific focuses on gender and UKIP) and media influence to analysis of reporting and, of course, those polls.
Genie in a bottle image by Herval
Steve Yelvington has a great phrase for the oft-repeated claim that newspapers ‘sowed the seeds of their own demise’ by putting their content online for free many years ago:
He calls it the ‘original sin’ myth:
“The most charitable thing I can say about it is: This is bullshit.”
I’ll let you read his post to get the full background, but as someone else who was there at the time I can only say: He’s right. What choice did publishers have? Let AOL or MSN steal an emerging fast-growing market as theirs declined?
The thing about this myth is that it relies upon some sort of ‘genie in the bottle’: the idea that news organisations had something special that they ‘let out’. That’s a nice story, but it’s only half the story. Continue reading
So Yahoo! Pipes will be ‘retired’ in a couple of months. It was a seminal tool for its time, helping people like me demonstrate and explore the potential of RSS, APIs and automation without having to become programmers first.
But times move on. I’d stopped using Pipes years ago: I’d caught the programming bug and wanted to do more; and while Pipes fell into disrepair dozens of other tools were springing up showing how similar things could be done.
And top of the crop has been IFTTT.
IFTTT (If This Then That) was Pipes without the pipes. And although it lacked some of the functionality and control of Pipes (searching multiple sites all at once was a particular favourite), it more than made up for that with simplicity of interface and an ever-growing list of services it supported.
The proposition was a clear one: if you’re having to anything more than once, get IFTTT to do it for you. Cross-posting from WordPress to Facebook? Automate it. Saving all the tweets using a particular hashtag? Automate it.
When I first started using IFTTT there were around a dozen ‘channels’ you could connect: mainly social media and blogging services like Twitter, Facebook and WordPress – and of course, any RSS feed.
Now there are dozens and dozens of channels: not just newer social media platforms including Instagram and Pinterest, but email and phonecall triggers, iOS reminders and notifications, wearable tech like Nike+, Android Wear and Fitbit, connected home devices like WeMo light switches and Honeywell thermostats, connected car devices like Dash and Mojio, and triggers from your mobile phone including its location and text messages.
As the world has become more connected, so IFTTT has grown in potential. It is also one of the clearest demonstrations of how ‘visible’ we are when we connect to the web: sharing everything from the temperature of our house and the speed of our car to the position of our phone, and our partner’s phone.
Like Pipes you can set all sorts of filters, and you can browse, search and adapt ‘recipes’ by other users, including ‘notify me of a zombie apocalypse‘.
Of course, it’s a gateway drug to programming. By demonstrating one of the most basic concepts in programming – the ‘if’ – and how much time you can save with just that, it makes you want to do more. And it opens you up to the world of possibilities that comes with working with developers.
You may never have used Pipes, but it’s probably a good time to play with IFTTT.
Tony Hirst has written about the announcement and how that reflects changes in the culture of web companies. Adam Tinworth has pointed out that many organisations relying on pipes created by departed staff may find that things stop working.Ghacks Technology News outlines alternatives ClickScript, Quadrigram, and Superpipes.
Online Journalism Blog 