I come upon examples of bad practice in publishing government data on a regular basis, but the Universal Jobmatch tool is an example so bad I just had to write about it. In fact, it’s worse than the old-fashioned data service that preceded it.

That older service was the Office for National Statistics’ labour market service NOMIS, which published data on Jobcentre vacancies and claimants until late 2012, when Jobcentre Plus was given responsibility for publishing the data using their Universal Jobmatch tool.

Despite a number of concerns, more than a year on, Universal Jobmatch‘s reports section has ignored at least half of the public data principles first drafted by the Government’s Public Sector Transparency Board in 2010, and published in 2012.

This is why those principles are important for anyone doing research on issues of public policy (in this case, benefit claimants and the jobs open to them).

“Public data will be timely and fine grained”

On Universal Jobmatch reports you can have data down to one particular district or ward – but you can’t have more than one at the same time, unless you want aggregate figures for the areas overall.

Instead, you will have to download them separately, manually, then add the name of the region, and repeat this however many hundreds of times you need to get all the areas you need.

And then repeat again for each month – there’s no way of requesting more than one month’s data in any form.

If you wanted a national picture broken down by ward for just one month this would involve 9,523 manual downloads. If you wanted to look at change over 12 months, you’re looking at 114,276 repetitive actions – before you get to the cleaning and merging.

Which is where we come to the next principle:

“Public data will be published in re-usable, machine-readable form”

Well you can certainly download the data as CSV or XML – but again, only manually after you’ve selected your criteria.

When people talk about ‘machine-readable’ they sometimes overlook the importance of machine-readable URLs.

Just because a machine can read one CSV that a human downloads, doesn’t mean you should be expected to download over 100,000 of them yourself. It’s for this reason that the next principle is included:

“Public data will be published using open standards, and following relevant recommendations of the World Wide Web Consortium (W3C)”

The World Wide Web Consortium’s guidelines are very clear on permanent URLs as a basic step in publishing government data:

• “use permanent patterned and/or discoverable “Cool URIs“”

When you look at the Universal Jobmatch reports, the URL ends “Reports/Reports.aspx” whether you are on the reports index, the search page for one report, or the results page.

In other words, there’s no way to link to the results of any search – or even the search page itself.

A URL is computer-readable: ideally, it should be possible to generate a list which a computer might cycle through, grabbing the data.

“Public data will be available and easy to find through a single easy to use online access point (data.gov.uk)”

15 months since taking over responsibility for the data, the DWP have yet to publish data on Jobcentre Plus on central data hub Data.gov.uk.

The holding page for the data says it “Will be reviewed in the light of demand and prioritisation”. When NOMIS published the data, more was available, and it’s clear that the demand is there, as the consultation responses show.

“Public data policy and practice will be clearly driven by the public and businesses who want and use the data, including what data is released when and in what form”

The consultation on moving the data from NOMIS to DWP received a long list of concerns – including criticisms of the consultation process itself.

Despite having over a year to do so, few of these concerns appear to have been addressed in the Universal Jobmatch reports, despite a promise that “All points raised will be considered as part of phased planned improvements to the Universal Jobmatch MI [management information] Tool.” to be “available in Spring 2013”.

There’s also one final problem with the report generation service: you cannot select any location starting with a letter in the second half of the alphabet. When it comes to wards, the drop-down menu only goes halfway through the ‘A’s before it is cut off.