Tag Archives: text mining

Those Android Trump tweets: David Robinson on using text data to get an election scoop

1 Reply

Washington Post story tweet

Data scientist David Robinson was behind one of the most striking data stories of this US election season, when his analysis of Donald Trump tweets appeared to confirm that Trump was posting the angriest comments on that account (jointly managed by his campaign staff). Barbara Maseda spoke to Robinson about the story behind that text analysis and what comes next. 

It was August 9 when David Robinson published his analysis of Trump tweets on his blog. Robinson had used a series of libraries in the programming language R to collect, clean, process and visualise the data. The process took just 12 hours, from Saturday night through Tuesday morning.

In the following days, the piece would be re-posted and cited by multiple websites, including The Washington Post and Mashable. The original piece alone had hundreds of thousands of views in just a few days.

The result wasn’t just one election story, but one of the biggest indications yet of the potential of text analysis for journalists, with three takeaways in particular: Continue reading →

AUDIO: Text mining tips from Andy Lehren and Sarah Cohen

1 Reply
Searches made of the Sarah Palin emails

Searches made of the Sarah Palin emails - from a presentation by the New York Times's Andy Lehren

One of the highlights of last week’s Global Investigative Journalism Conference was the session on text mining, where the New York Times’s Andy Lehren talked about his experiences of working with data from Wikileaks and elsewhere, and former Washington Post database editor Sarah Cohen gave her insights into various tools and techniques in text mining.

Andy Lehren’s audio is embedded below. The story mentioned on North Korean missile deals can be found here. Other relevant links: Infomine and NICAR Net Tour.

And here’s Sarah’s talk which covers extracting information from large sets of documents. Many of the tools mentioned are bookmarked ‘textmining’ on my Delicious account.