This was first published on the BBC College of Journalism website:

I have just spent 10 months publishing an ebook. Not ‘writing’, or ‘producing’, but 10 months publishing. Just as the internet helped flatten the news industry – making reporters into publishers and distributors – it has done the same to the book industry. The question I wanted to ask was: how does that change the book?

Having written books for traditional publishers before, my plunge into self-publishing was prompted when I decided I wanted to write a book for journalists about scraping: the technique of grabbing and combining information from online documents.

There was a time when self-publishing was for those who couldn’t get themselves printed. Increasingly, however, it’s for those who cannot wait to. This was just such a case, with classic symptoms: a timely subject that is prone to change; a small market (or so I thought) and a dispersed and knowledgeable audience.

To carry it through I turned to the self-publishing website Leanpub, having seen what my Birmingham City University colleague Andrew Dubber had been doing with the service. Most ebook services offer the timeliness of ebook publishing, but Leanpub had something else: agility.

‘Agile development’ is a popular concept in technology development: it is the idea that, rather than launching a ‘finished’ product upon the world, you should instead launch something part-finished and develop it in response to user feedback.

In other words, it is better to see how people actually use something and respond to that, than to assume you know what they will use it for. My ebook was designed to be used – but would people use it how I imagined?

So, in July 2012 I put up a page announcing the imminent publication of the book. Users could suggest how much they might be prepared to pay. Immediately, I had some indication of suitable pricing. Free market research.

When the first two chapters were published, I started with a cheap price: readers were, after all, taking a gamble on the content that followed. You might also argue that these ‘early adopters’ of the book would be key to its continued success. Why discount a book that has grown old, when you can discount one that isn’t even finished yet?

I published a new chapter every week for the first few months. People who had bought the book would receive an email alerting them to the new content to download. An accompanying Facebook page, and my own Twitter account, helped provide other platforms for announcements, but also reader feedback.

One reader told me about idiosyncrasies in how tools worked in different countries: I added additional notes in the books. Others told me how they used links: I changed the way that I formatted them. Readers suggested alternative solutions to problems outlined in one chapter – and I added those at the end of that chapter.

The book evolved out of that call-and-response, including usage data: which formats were most popular; how pricing affected buying behaviour; what languages might be best for future translations. It has combined the best elements of blogging (readers as editors; iterative writing; analytics) with the best of books (comprehensiveness; structure).

When I set out to write it, I thought there might be barely 100 people in the world who would want to buy it. As I began that final chapter, it had sold five times that – the rate of a mildly successful textbook. This has genuinely shocked me. No publisher would have guessed that market existed. Even if they wanted to bet on it, they couldn’t have distributed the books effectively enough.

There is an exchange that sometimes takes place, perfectly described by Beth Ashton, between those who use technology, and those who don’t. It goes like this:

Prospective data journalist: ‘I’d really like to learn how to do data journalism but I can’t do statistics!’

Data journalist: ‘Don’t let that put you off, I don’t know anything about numbers either, I’m a journalist, not a mathematician!’

Prospective data journalist: ‘But I can’t code, and it all looks so codey and complicated’

Data journalist: That’s fine, NONE OF US can code. None of us. Open angle bracket back slash End close angle bracket.

“These people are coding deniers,” argues Beth.

I think she’s on to something. Flash back to a week before Beth published that post: I was talking to Caroline Beavon about the realisation of just how hard-baked ‘coding’ was into my workflow:

• A basic understanding of RSS lies behind my ability to get regular updates from hundreds of sources
• I look at repetitiveness in my work and seek to automate it where I can
• I look at structure in information and use that to save time in accessing it

These are all logical responses to an environment with more information than a journalist can reasonably deal with, and I have developed many of them almost without realising.

They are responses as logical as deciding to use a pen to record information when human memory cannot store it reliably alone. Or deciding to learn shorthand when longhand writing cannot record reliably alone. Or deciding to use an audio recorder when that technology became available.

One of the things that makes us uniquely human is that we reach for technological supports – tools – to do our jobs better. The alphabet, of course, is a technology too.

But we do not argue that shorthand comes easy, or that audio recorders can be time consuming, or that learning to use a pen takes time.

So: ‘coding’ – whether you call it RSS, or automation, or pattern recognition – needs to be learned. It might seem invisible to those of us who’ve built our work patterns around it – just as the alphabet seems invisible once you’ve learned it. But, like the alphabet, it is a technology all the same.

But secondly – and more importantly – for this to happen as a profession we need to acknowledge that ‘coding’ is a skill that has become as central to working effectively in journalism as using shorthand, the pen, or the alphabet.

I don’t say ‘will be central’ but ‘has become‘. There is too much information, moving too fast, to continue to work with the old tools alone. From social networks to the quantified self; from RSS-enabled blogs to the open data movement; from facial recognition to verification, our old tools won’t do.

So I’m not going to be a coding denier. Coding is to digital information what shorthand was to spoken information. There, I’ve said it. Now, how can we do it better?

A new survey for journalists, bloggers and online publishers, which can be found at this link, aims to collect information about their experiences of and views on libel and privacy law

A system of arbitration is at the heart of Lord Justice Leveson’s recommendations, and different versions are included in the the government’s draft Royal Charter and the industry’s own proposals [PDF].

The suggestion is that an arbitration service could deal with libel and privacy complaints that would otherwise go to court.

Last minute amendments to the Crime and Courts bill (now Act) would allow for bloggers to opt into the regulatory arbitration system and receive costs benefits.

Additionally and separately, recommendations have also been made for Mediation and Early Resolution in defamation disputes.

However, there is very little solid data about the nature and quantity of legal claims made against the media, including small bloggers. Because the majority of libel claims, for example, are believed to be resolved out of court, there is no complete record of disputes.

In short, little is known about bloggers’ and journalists’ actual legal experiences and opinions.

In an effort to build a better picture and to help inform the development of new alternative dispute resolution mechanisms, I am launching a survey as the final part of my doctoral project at the Centre for Law, Justice and Journalism (CLJJ), City University London.

This questionnaire is open to all types of journalists and online writers who expect their readership to be predominantly based in England and/or Wales.

Please take part and share your experiences and encourage your colleagues and friends to participate as well.

All data will be collected anonymously with no identification of organisations or individuals.

The questionnaire can be found here:

• https://www.surveymonkey.com/s/medialawsurvey

Many thanks for your help! If you have any questions you can email me: (judith.townend.1@city.ac.uk) or tweet  (@jtownend).

About the project

This survey is part of Judith Townend’s doctoral project at the Centre for Law, Justice and Journalism (CLJJ), City University London. The research project, which has been given ethical approval by the CLJJ, explores how journalists and online writers are affected by libel and privacy law, as well as other social and legal factors. It will draw attention to the issues faced by online writers and journalists, and help inform the development of resources in this area.

About this questionnaire

• The questionnaire is open to all types of journalists and online writers who expect their readership to be predominantly based in England and/or Wales.
• It should take between 10 and 30 minutes to complete, depending on your experiences and views. Some questions require an answer so you can be taken to the next relevant question.
• All data will be collected anonymously with no identification of organisations or individuals.
• The information you have submitted will included in a final report to be published in 2013/14, which may be used for future online and print publications.
• Please contact Judith Townend with any questions, or to obtain the final results.

Contact details:

Judith Townend, c/o Peter Aggar, Centre for Law, Justice and Journalism, City University London, Northampton Square, London EC1V 0HB, Tel: +44 (0)20 7040 8167

E-mail: judith.townend.1@city.ac.uk

Here’s another collection of questions answered in public as part of the FAQ section – this time concerning citizen journalism:

What effect do you think citizen journalism will have on the future for professional journalists?

It’s already had an impact on their awareness of scrutiny: I’ve heard journalists and editors saying they feel they have to produce better quality journalism and get it right, because they know if they get the facts wrong people will point it out.

It’s also already had an effect on how they work: journalism is becoming more collaborative as journalists realise that many ‘citizen journalists’ either have more expertise on a subject, or first hand experience of an event. But also that ‘citizen journalists’ actually have little interest in being journalists – they’re not in competition.

You could probably also add its effect on the news agenda: if a particular story or aspect of a story is generating a lot of heat online then it is harder for journalists to ignore it – for example the remarks by Trent Lott that initially went largely unreported; or Trafigura.

As the saying goes: the future is already here, it’s just unevenly distributed. These changes are already here and will probably continue to impact journalism’s future more broadly.

Do you think the rise of blogs and social media might endanger aspiring journalists’ employability in the future?

Not any more than the rise of pubs and music venues might endanger aspiring musicians. Blogs and social media are platforms to practise your journalism. In that sense it provides a space to showcase your work and build your contacts and reputation – particularly for people who might otherwise not be able to access jobs in the media because they cannot afford to put themselves through university, do unpaid internships in an expensive city, etc. For those people, it actually improves employability.

There is the danger that some bloggers are exploited by publishers who use their content without payment, but ultimately if they want reliable, high quality content they will have to employ someone. Ultimately, if someone is producing content without payment or employment, it’s generally because they get some reward other than payment or employment – and that means writing what you want, not what you’re paid to.

Do you think people being able to write whatever they want on Twitter and similar sites without rules and regulations is necessarily a good thing?

They cannot write ‘whatever they want’ ‘without rules and regulations’. The same laws that apply to what you do offline apply online – in some ways, more so.

There are numerous people who have been arrested, charged, and even sent to jail for things they have said on Twitter, Facebook, and elsewhere. If they’d said those same things in the pub, they likely would not have faced the same consequences.

What do you think the future holds for citizen journalism?

It’s too broad a question to really answer. I don’t think ‘citizen journalism’ is even a phrase that’s widely used any more – so perhaps the future is that we will stop talking about it altogether. We will just talk about journalism, and more often than not it will involve contributions from both professional journalists and non-journalists finding, publishing and distributing information online.

The idea of ‘citizen journalism’ was, in retrospect I think, something of an ego trip by journalists: it came from an assumption that all citizens aspired to be journalists; that their attempts to publish came from a desire to ‘be like us’.

My own experience with Help Me Investigate has been that they don’t: what they want is answers, or simply to communicate with other human beings. ‘Citizens’ show little desire to tell news stories about their discoveries in the same way that journalists do. We’re quite odd that way, really.

Videographer Franzi Baehrle has published an ebook documenting lessons in delivering video training to non-journalists.

The ebook was part of her final project for the MA Online Journalism at Birmingham City University, and based on her experiences of working with communities online and offline in Birmingham, with the Guardian Media Group’s n0tice project, the Birmingham Mail’s digital team, and independently.

I forgot to blog about it at the time it was published last Autumn, but better late than never: it’s an excellent piece of work, and well worth reading.

Last night I published the final chapter of my first ebook: Scraping for Journalists. Since I started publishing it in July, over 40 ‘versions’ of the book have been uploaded to Leanpub, a platform that allows users to receive updates as a book develops – but more importantly, to input into its development.

I’ve been amazed at the consistent interest in the book – last week it passed 500 readers: 400 more than I ever expected to download it. Their comments have directly shaped, and in some cases been reproduced in, the book – something I expect to continue (I plan to continue to update it).

As a result I’ve become a huge fan of this form of ebook publishing, and plan to do a lot more with it (some hints here and here). The format combines the best qualities of traditional book publishing with those of blogging and social media (there’s a Facebook page too).

Meanwhile, there’s still more to do with Scraping for Journalists: publishing to other platforms and in other languages for starters… If you’re interested in translating the book into another language, please get in touch.

“Imagine if you could search for any calculations and then just use them directly without ever having to work it out yourself from scratch.”

This is the vision of developer Daniel Maxwell, the creator of livesheets.com, whose dream it is for no one in the world to perform the same calculation twice again.

Livesheets allows users to input calculations and formulas and to save and share them with the world. Perhaps the easiest way to describe Livesheets is the way that it describes itself, as ‘numerical lego’. So someone sitting in NASA makes and saves a formula and you log on and can use and manipulate it, add it to other formulas and experiment.

Maxwell believes that the benefits of this are twofold:

“Technophobes can use stored formulae over and over again and people right at the top can pull other people’s expertise off the shelf. If you want to forge new boundaries in science you can always be doing something new, you never have to open a blank sheet and figure stuff out from scratch.”

Although it may sound like a glorified place to save Excel formulae the possibilities range far beyond that. Livesheets can calculate your income tax, solve quadratic equations or just work out your porn star name.

Livesheets is – and Maxwell promises it will remain – a “completely open source, not-for-profit platform,” and he believes that it will offer new capabilities to those with inquisitive minds without the need to understand the science behind calculations.

“I want this to make every child into a rocket scientist and we can really do that. Kids could go on there and search for calculations that have been done by NASA or the Astronomy department at Cambridge University and just start snapping them together and see what they can do.

“They don’t necessarily have to be able to understand everything that’s going on, they just have to know what that calculation is supposed to do.”

What Livesheets needs now is input from those people with calculations to build a database and, if that can be built up successfully, there may one day be no real need for anyone to ever perform the same equation twice.

I’ve pulled together a collection of research, posts, book chapters, interviews, case studies and other material documenting the process of running Help Me Investigate for anyone writing a research paper or wishing to ask questions about the site. I hope it’s useful.

There’s an interview with me in both Spanish and English at Mas Investigacion covering a number of questions about data journalism, Help Me Investigate, MOOCs, and teaching.

Although the introduction is in Spanish, scroll down and click on ‘Read the interview in English’ if you want that version.

The European Journalism Centre and Open Knowledge Foundation are organising panel discussions and workshops as part of a School of Data Journalism at this year’s journalism festival in Perugia. More details here; registration here.