Journalism, AI and satellite imagery: how to get started

Satellite image of the Amazon. Tocantins, Brazil. Source: Copernicus Sentinel data [2022] processed by Sentinel Hub, using Highlight Optimized Natural Color.

In the first of two guest posts for OJB, first published on ML Satellites, MA Data Journalism student Federico Acosta Rainis explains how to get started with satellite journalism — and avoid common pitfalls.

Working with satellite imagery and AI models takes time and patience. There is no general rule: you have to find the right model for each case, in a process of trial and error, while crunching large amounts of data.

That is why the advice of Anatoly Bondarenko, data editor of Texty, is crucial:

“Find a task or story where you will not be sorry to put many resources in.”

The combination of satellite imagery and AI is useful for providing general context where no data exists, discovering patterns over large territories or over time, counting scattered objects or finding needles in a haystack.

“AI makes sense when you want to do things at a larger scale or repeatedly” says Edward Boyda from Earthrise, who worked on the Amazon Mining Watch project. “Either because you’re working on such a big area that it’s impractical to do it as a human or because you want to do it again and again over time.”

Before you go any further, ask yourself: is this the best tool to tell the story?

Perhaps there is an easier or more efficient way to get the information you are looking for. There may already be a dataset available, or you may be able to get it through a public information request. Or you may manually count objects in a satellite image, without the need for AI.

Know the territory and the basic tools

The Amazon Mining Watch project website
The Amazon Mining Watch project website

If you decide to go ahead, the next step is to familiarise yourself with your region of interest (ROI) and the basic tools you will use.

One of the advantages of satellite data is that it allows you to investigate from afar: you can observe any corner of the globe from your computer. But you should still familiarise yourself with the ROI in order to interpret your findings correctly.

If you can visit the area of interest, so much the better; if not, gather as much information as you can about it.

Find out what the problems or hot topics are, get non-satellite images, contact local people or experts that work there. This will help you to minimise errors during the analysis.

Boyda says that satellite imagery is useful for investigating areas where it is “hard for journalists to operate”, but that it should always be used “in a combination with traditional on the ground reporting”, as satellites are “useful for illustrating or quantifying, but don’t bring that human story element in”.

As for the tools, take some time to experiment and choose the ones you are most comfortable with or can best adapt to your case.

Start with the familiar: if you already know R or Python, for example, look for libraries such as RSToolBox or Earthpy.

You don’t always need to write code: there is free software with AI algorithms for classifying images or detecting objects. Two very useful ones are QGIS (with the SCP and dzetsaka plugins) and SNAP (ideal for European Space Agency satellite data).

For an initial analysis, you don’t need anything too fancy:

  1. Use Google Earth Pro to quickly look at an ROI over time.
  2. If you need more information, Sentinel Hub’s EO Browser is good for a first analytical approach: you can use different satellites, test band combinations and filter cloud cover.
  3. If this is still not enough, then it’s time to create your own framework by combining different tools.

Ask for help from others

Most journalistic pieces that use AI and satellite imagery are collaborative projects and rely on a data expert.

You can look for help in specialised forums or contact other journalists who have used such techniques.

Diagram showing satellites orbiting the earth at different distances and categories: science, Copernicus and Meteorology
Earth observation missions: image created by the European Space Agency (ESA), (CC BY-SA 3.0 IGO)

But also look beyond journalism: Earth Observation (EO) and remote sensing are used in many fields such as agriculture, ecology, biology and disaster management.

If you seek advice from others before starting the technical work, you will avoid some of the dead ends that others have already gone down.

It is not just about solving technical problems. An NGO working on environmental issues may be able to supply high-resolution images, give you hard-to-get information about your ROI or put you in touch with local experts.

“This knowledge is not something that is intrinsic in the news industry,” says Mathias Felipe de Lima Santos, project manager of data-driven projects at InfoAmazonia and researcher in the Digital Media and Observatory at the Federal University of São Paulo (Unifesp). “So news organisations need to go where they can find it, looking at civil society organisations or startups focused on using the satellite imagery.”

This way, he says, organisations can “produce investigations that go beyond their own limits”.

Start small and be organised

Always start small. Even if your ultimate ROI is a much larger area, start by limiting your analysis to an area that is representative of the phenomenon you want to investigate, but as small as possible.

Using AI on a single satellite image takes a fair amount of intellectual effort, computational power and processing time.

As you add more images, the difficulty grows exponentially: the information to be processed increases and new problems arise, such as how to stitch the images together, work out the overlaps, and compare images from different dates.

Do all the tests you need in a small area and only when you are satisfied with the results extend the model to your entire ROI.

You may be using a lot of information from different sources. Satellite imagery in raw format often has complicated names, combinations of letters and numbers that reference the coordinates, date and type of processing used.

It is good to be consistent: using a spreadsheet as a master file can help you to organise all this data.

As when writing code, it is better to use extra-explicit nomenclature rather than generic names that will make it difficult to remember what they refer to.

Working with AI: models and computational power

Semantic segmentation involves identifying different elements in an image. Image by B.Palac

AI comes in different flavours but you will probably use one (or a combination) of the following tools:

If you plan to use a supervised model for object detection, find out if there are already image datasets available to train it. Creating datasets is a time-consuming task.

Try different models – and different parameters – to see which one suits you best. Some of the best known are Random Forest, Supporting Vector Machine (SVM) and Nearest Neighbours.

Multi-layered Deep Learning (DL) models tend to give better results, but are much more complex to build.

If you have to process a lot of data, consider cloud computing. There are free tools such as Google Colab or Google Earth Engine (GEE) that can crunch huge amounts of data and get results in a short time.

GEE also provides easy access to datasets from Landsat, Sentinel, and MODIS missions.

Amazon Mining Watch, for example, uses Descartes Labs, a paid cloud computing service to process large amounts of satellite data.

“It’s built around in a way that you can break the images into tiles for parallel processing and that can be ingested directly by the model,” says Boyda. “So we don’t have to think about that part of the pipeline.”

If you work on your personal computer, it’s essential to have a solid-state disk (SDD), at least 8 or 16 GB of RAM and a good amount of available storage: a single raw satellite image often weighs several gigabytes.

A powerful processor, a dedicated video card and a good-sized monitor will also make your work easier.

Verify the results and share your methodology

Scepticism is a powerful weapon in journalism. Treat AI predictions like any other data: don’t take them for granted.

Like all technology, AI does not work magic or replace traditional journalism methods. Rather, it complements them.

“It’s always this golden rule: 80% of the work was data,” says Anatoly about Texty‘s investigation into illegal amber mining in Ukraine. “Also, a correct interpretation of data, because we had to find images in big resolution, but no less important was to find some first examples of the specific patterns of mining.”

A good part of your job will be to clean up false positives or understand why there are so many false negatives to improve your model.

If you are working on a story about change, for example, also pay attention to what didn’t change and ask yourself why.

It is important to get some ground truth to check the results of your model: field data obtained directly in the area you are analysing: is what looks like a crop field in the satellite image really a crop field?

That doesn’t mean you have to travel: you can contact local experts to help you evaluate your findings.

When you publish, share the methodology you have used, either in the piece itself or on an external site such as GitHub. It’s a good transparency practice, aids in understanding your analysis and helps others interested in working on the same topic.

But also think about the audience, who don’t necessarily know how AI works, and explain its scope and limitations.

“Journalists need to be transparent and explain not only about the metadata, but also explain what AI is, what computer vision is, and how we approach that,” says Mathias. “And make clear the limitations [of the model].”

Always make explicit the gaps in your research, the things that are unverified and the levels of uncertainty in your model. Putting on record what you couldn’t solve doesn’t diminish your work; on the contrary, it makes it more reliable.

Federico is a Google News Initiative Fellow at The Guardian. You can find him on Twitter @facostarainis


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.