I was recently invited to speak to students at Tampere University in Finland, and had the opportunity — with the help of Esa Sirkkunen — to translate the ‘Inverted pyramid of data journalism‘ into Finnish. I’m sharing it here for anyone else who might find it useful.
What is dirty data and how do I clean it? A great big guide for data journalists
If you’re working with data as a journalist it won’t be long before you come across the phrases “dirty data” or “cleaning data“. The phrases cover a wide range of problems, and a variety of techniques for tackling them, so in this post I’m going to break down exactly what it is that makes data “dirty”, and the different cleaning strategies that a journalist might adopt in tackling them.
Four categories of dirty data problem
Look around for definitions of dirty data and the same three words will crop up: inaccurate, incomplete, or inconsistent.
Inaccurate data includes duplicate or misentered information, or data which is stored as the wrong data type.
Incomplete data might only cover particular periods of time, specific areas, or categories — or be lacking categorisation entirely.
Inconsistent data might name the same entities in different ways or mix different types of data together.
To those three common terms I would also add a fourth: data that is simply incompatible with the questions or visualisation that we want to perform with it. One of the most common cleaning tasks in data journalism, for example, is ‘reshaping‘ data from long to wide, or vice versa, so that we can aggregate or filter along particular dimensions. (More on this later).Continue reading
Angles for data stories — in Finnish (yleistä näkökulmaa datatarinoihin)
I recently had the opportunity — thanks to Esa Sirkkunen of Tampere University — to translate the diagram from ‘8 angles that journalists use most often to tell data stories‘ into Finnish. I’m sharing it here for anyone else who might find it useful.
FOI, diversity, and imposter syndrome — an interview with Jenna Corderoy
On Tuesday I will be hosting the award-winning investigative journalist and FOI campaigner Jenna Corderoy at the Lyra McKee Memorial Lecture. Ahead of the event, I asked Jenna about her tips on investigations, FOI, confidence, and the challenges facing the industry.
What’s the story you have learned the most from?
The story that I learned the most from was definitely our Clearing House investigation. Back in November 2020, we revealed the existence of a unit within the heart of government, which screened Freedom of Information (FOI) requests and instructed government departments on how to respond to requests. The unit circulated the names of requesters across Whitehall, notably the names of journalists.Continue reading
VIDEO PLAYLIST: An introduction to Python for data journalism and scraping
Python is an extremely powerful language for journalists who want to scrape information from online sources. This series of videos, made for students on the MA in Data Journalism at Birmingham City University, explains some core concepts to get started in Python, how to use Colab notebooks within Google Drive, and introduces some code to get started with scraping.Continue reading
Availability bias: a guide for journalists
I’ve written previously about the role that cognitive biases play in journalism, how to avoid confirmation bias, and anticipate criticism based on fallacies — but one cognitive bias I haven’t written about yet is the availability heuristic — or availability bias.
Availability bias is the tendency to reach for the most available reason, event, or tool, when confronted with a problem or decision.Continue reading
Here’s how the ‘8 data story angles’ can help you get stories from company accounts
A couple of years ago I mapped out eight common angles for identifying stories in data. It turns out that the same framework is useful for finding stories in company accounts, too — but not only that: the angles also map neatly onto three broad techniques.
In this post I’ll go through each of the three techniques — looking at cash flow statements; compiling data from multiple accounts; and tracing people and connections — and explain how they can be used to get stories, with examples of articles that have used those techniques successfully.
We start, naturally, with the money…Continue reading
9 способов найти историю в финансовых отчётах компаний
This is a masterclass in writing a story about company directors’ pay — so I reverse-engineered it
Company directors’ pay regularly provides material for stories — and this front page story by The Guardian’s Robert Booth was such a masterclass in the genre (as well as other open source intelligence techniques) that I decided to reverse-engineer it for a Twitter thread.
I’ve embedded the thread below, or you can read it on Threadreader here.
Using company accounts in journalism
You can find other posts about using company accounts at the following links:
- VIDEO PLAYLIST: Finding stories in company accounts
- Here’s a story about a celebrity fashion charity which provides some useful tips and tricks for journalists using company accounts
- Here’s how one journalist used all sorts of company account techniques to tell a story about a social media platform for sex workers
- Teaching journalists how to find stories in company accounts: the story treasure hunt
- Here are 9 ways to find stories in company accounts (and only three of them involve numbers)
How ‘triangulating’ can help you identify more sources
In this edited extract from the forthcoming third edition of the Online Journalism Handbook I look at how a ‘triangulation’ approach to sourcing can help broaden story research and improve reporting.
Two centuries ago journalists were called reporters because they drew their information from official reports — documents.
Then in the late 19th century a new source became part of journalistic practice: people, as interviews and eyewitness accounts were added to news articles.
The late 20th century saw reporting undergo another expansion in sourcing, as digital data was added to the journalist’s toolkit.
Although reports had included tables and other sources of data, the properties of digital data — filterable, sortable and searchable — have been significant, and make data a qualitatively different type of source.
How documents, people and data all lead to each other
Considering sourcing along those three dimensions — people, documents, and data — can be particularly useful when planning sourcing.Continue reading