Spreadsheet analysis is part of the research phase of a story
Generative AI tools like ChatGPT and Gemini can be a big help when dealing with data in spreadsheets. In this third of a series of posts from a workshop at the Centre for Investigative Journalism Summer School (the first part covered idea generation; the second research), I outline tips and techniques for using those tools to help with spreadsheet formulae and reshaping data.
Whether you come across data as part of story research, or compile data yourself, chances are that at some point you will need to write a formula to ask questions of that data, or make it possible to ask questions (such as creating a column which extracts data from another).
If you find yourself coming up against the limits of your spreadsheet knowledge, then genAI tools can be useful both in breaking through those — while expanding your knowledge of functions and formula writing.
Writing spreadsheet formulae with ChatGPT or other genAI tools
Generative AI can be used at all points in the journalism process: this post focuses on the research stage
In the second of a series of posts from a workshop at the Centre for Investigative Journalism Summer School (read the first part on idea generation here), I look at using generative AI tools such as ChatGPT and Google Gemini to improve sourcing and story research.
Research is arguably the second-highest risk area (after content generation) for using generative AI within journalism. The most obvious reason for this is AI’s ability to make things up (“hallucinate“) — but there are other reasons too.
Data journalism during the election lacked case studies and quotes from those outside of the Westminster bubble, according to an analysis of over 50 pieces published during the election campaign.
But the rare exceptions offer examples of how we can do better election data journalism which sets the agenda and gives voters a louder voice.
Generative AI can be used at all points in the journalism process: this post focuses on pre-production
Last week I delivered a session at the Centre for Investigative Journalism Summer School about using generative AI tools such as ChatGPT and Google Gemini for investigations. In the first of a series of posts from the talk, here are my tips on using those tools for idea generation.
Generative AI tools may not be entirely reliable, but that doesn’t mean that they’re not useful. Journalism, after all, is about more than just gathering information: reporters also need to generate story ideas, identify and approach potential sources, plan ahead, write and edit stories and solve a range of technical challenges. All of these are areas where genAI can help.
Machine learning and Natural Language Processing (NLP) are two forms of artificial intelligence that have been used for years within journalism. In this video, part of a series of video posts made for students on the MA in Data Journalism at Birmingham City University, I explain how both technologies have been used in journalism, the challenges that journalists face in using them, and the various concepts and jargon you will come across in the field.
SRF Data example (note: this uses a Random Forest algorithm, which employs a collection of ‘decision trees’, and is not a decision tree as stated in the video)
Having outlined the range of ways in which artificial intelligence has been applied to journalistic investigations in a previous post, some clear challenges emerge. In this second part of a forthcoming book chapter, I look at those challenges and other themes: from accuracy and bias to resources and explainability.
Investigative journalists have been among the earliest adopters of artificial intelligence in the newsroom, and pioneered some of its most compelling — and award-winning — applications. In this first part of a draft book chapter, I look at the different branches of AI and how they’ve been used in a range of investigations.
Planning an investigation, or any larger editorial project, raises its own particular challenges — but if you know where to look, you can find resources that are especially useful in anticipating and tackling those.
The most basic change to the Inverted Pyramid of Data Journalism is the recognition of a stage that precedes all others — idea generation — labelled ‘Conceive’ in the diagram above.
This is often a major stumbling block to people starting out with data journalism, and I’ve written a lot about it in recent years (see below for a full list).
The second major change is to make questioning more explicit as a process that (should) take place through all stages — not just in data analysis but in the way we question our sources, our ideas, and the reliability of the data itself.
A third change is to remove the ‘socialise‘ option from the communication pyramid: in conversation with Alexandra Stark I realised that this is covered sufficiently by the ‘utilise’ stage (i.e. making something useful socially).
Alongside the updated pyramid I’ve been using for the past few years I also wanted to round up links to a number of resources that relate to each stage. Here they are…
One of the most common challenges in a data-driven story is combining two sets of data — such as events and populations — to put a story into context. In an extract from the ebook Finding Stories in Spreadsheets, I explain how to use lookup functions to combine two tables.The longer ebook version of this tutorial includes a dataset and exercise to employ these techniques.
Combining data is often a great way of telling new stories about spreadsheets. For example: you may have one table showing pass rates for each school in an area, and another table showing their addresses. Combining these would allow you to identify geographical patterns, or to place them on a map.
You could also combine the addresses with poverty rates for different locations, or unemployment to see if there’s a possible relationship (remembering that correlation does not equal causation), or to identify the schools performing particularly well despite local conditions. In the video below, for example, I walk through an example of combining data on different sports teams’ attendances with data on their rankings, allowing you to see who’s attracting large crowds despite their poor performance.
The VLOOKUP function is one of the most widely-used tools in combining data in this way. It stands for Vertical lookup, and means that the spreadsheet will look up and down a column (i.e. vertically) for whatever you ask it. In more recent versions of Excel the XLOOKUP function has been introduced to make the process easier — but the process is similar for both.