This interactive chart is generated from some data you can grab
Increasingly you might come across an interesting set of interactive charts from a public body, or an interactive map, and you want to grab the data behind it in order to ask further questions. In many cases you don’t need to do any scraping — you just need to know where to look. In this post I explain how to work out where the data is being fetched from… Continue reading
A cartogram represents geography diagrammatically. Note above for example that London takes up more space than parts of Wales because it has much denser population
The New York Times didn’t use a cartogram for its EU referendum map, Gregor Aisch explained last week, because they are “too confusing for readers unfamiliar with the geography”.
The Guardian, on the other hand, did use a cartogram. In a series of tweets their visuals editor Xaquin Gonzalez explained why
Original image by GotCredit/Flickr
If you are working with map data that uses the shapes of regions or countries, chances are you’ll need to work with KML. In this guest post (first published on her blog) Carla Pedret explains how you can use the data cleaning tool Open Refine to ‘read’ KML files in order to convert them into other formats (for example to grab the names of places contained in the file).
KML (Keyhole Markup Language) is the default format used by Google’s mapping tool Fusion Tables (Google bought the company which created it in 2004), but it is also used by other mapping tools like CartoDB.
The open source data cleaning tool Open Refine can help you to open, process and convert KML files into other formats in order to, for example, match two datasets (VLOOKUP) or create a new map with the information of the KML file.
In this post, you will learn how to convert a KML file into XML and download it as aCSV file.
XML – Extensible Markup Language – is a language designed to describe data and it is used in RSS systems.
XML uses tags like HTML, but there is a big difference between both languages. XML defines the structure of the information, whereas HTML focuses on other elements too, including their meaning and arrangement (even when it is not supposed to focus on appearance), and the importing of other code and media.
KML – Keyhole Markup Language – documents are XML files specific for geographical annotations. KML files contain the parameters to add shapes to maps or three-dimensional Earth browsers like Google Earth.
The big advantage of KML files is the users can customize the maps according to their data and without knowing how to code.
You can find KML files using Google Tables search (make sure you have ‘Fusion Tables’ secleted on the left).
Type what you are searching for and add the word geometry or KML.
Open the fusion table and check that it has shapes by looking for a ‘map’ view (normally this has its own tab).
You should be able to download the KML when looking at that map view by selecting File > Download.
Once downloaded, to convert the file, upload your KML in Open Refine (download Open Refine here) and click Next.
In the blue box under your data, select XML files.
Now in the preview you can see the XML file with the structure of the information.
If you want to create a map with your own data and the shapes in the KML file, you need to match the KML with your data.
The example I have used contains the shapes of local authorities in the UK. I want to match the shapes in one dataset (the KML file) with information in another dataset on which party runs each council.
The element both datasets have in common (and therefore the element which will be used to combine them) is the name of the councils. But you need to check that those elements are the same: in other words, are the councils named in exactly the same way in both datasets, including the use of ampersands and other characters?
Have a look at the XML preview and try to find the tags that contain the information you need: in this case, authority names. In the example the tags containing the authority name are <name></name>.
Hover over that element so that you get a dotted box like the one shown below. Click on that rectangle and wait until the process has finished.
You should then see a column or columns as the picture shows.
On the right hand side of the page, change the name of your file and click on Create a new project.
Once created, you now only need to export it. Click on Export and select the format you prefer.
What originally was a KML file is now a filtered list with data ready to check and match against your other dataset.
Do you use Open Refine? Leave a comment with your tips and techniques or send it to me at @Carlapedret..
Here’s the 7th lesson from my year 2 undergraduate module in Online Journalism, where I look at a range of examples of mapping in journalism, as well as the broader idea of image maps (the rest of the series can be found here):
Online Journalism Blog 