Neil Smith is an investigative researcher and open source intelligence trainer. In this interview, originally posted on the Help Me Investigate blog, he gives his tips for using online sources to follow people and verify information.

PHP Freelancer

Answers to another set of questions around ethics and online journalism, posed by a UK student, and reproduced here as part of the FAQ series:

Do you believe online journalism presents new ethical dilemmas and should have standards of its own?

Yes, I think any changing situation – whether technological or cultural – presents new ethical dilemmas.

But should ‘online journalism’ have a separate code? I don’t see how it can. Where would you draw the line when most journalists work online? Ethical standards are relatively platform-agnostic, but journalists do have to revisit those when they’re working in new environments.

How far do you agree with the notion that immediacy is now being prioritised over accuracy?

Whether I agree or disagree doesn’t matter – that notion can only be proved or disproved based on evidence, not opinion. You could make arguments on both sides: the internet allows for faster news (immediacy), and also for more fact-checked news (interactivity), but ultimately it comes down to evidence (and remembering that correlation is not causation – even if you discover a decrease in fact-checking that might be down to institutional and commercial factors rather than technological ones)

Do you agree that the increase in competition in online news has the potential to glorify rumour and hearsay?

Again, whether I agree or disagree doesn’t matter – especially when there are terms here that need further definition, such as “glorify”. News consumers have always been interested in rumour and hearsay, regardless of the technology. The question is, are news media providing more of that, and if so is it because of technology, commercial pressures, or other factors?

You’d need quantitative research to get the numbers and qualitative research (e.g. ethnography) to get the motivations.

Do you think it is now harder for the reader to recognise news from a reliable source?

No. I think people are more critical news consumers. Partly because of the spread of media education, partly because more people have become media producers in their own right, and partly because new media allows people to seek out the sources of news and/or competing versions of events.

But again, you need research to prove this, not just my opinion.

How is online news affecting traditional values of objectivity?

The factors that gave rise to objectivity in news (a relatively modern idea) are to some extent challenged by new media: there is no limitation on ‘channels’, so no need to control who has access to those to ensure equal voice. The need for a mass market and to appeal to advertisers is reduced, so publishers can be less ‘neutral’.

There’s also cross-cultural and market competition influence here: UK publications (less objective) entering the US market (where objectivity and neutrality is a strong value).

There’s a lot of literature on the weaknesses and limitations of objectivity as a news value – it’s worth reading that if you haven’t already.

How is online journalism affecting the notion of transparency?

I’m not sure how the notion of transparency is affected. Certainly it is being used more widely, not just in journalism but in politics too. Essentially internet technologies make it possible to be more transparent, and gives less reason not to be.

Do you think that online journalism has threatened the role of ‘gate-keeping’?

See the answer on objectivity above – there are still gatekeepers, but these have multiplied to such an extent that the term is almost meaningless and it is more useful to talk of those without access to publishing and distribution technology, or of unequal access/literacy.

Journalists always have to respond to the information environment their audience (now users) live in, in all sorts of ways from the language and jargon that they can use, to the assumptions they can make about prior knowledge and understanding.

They are still gatekeepers in the sense that they must make editorial judgements on what to report, but they are now more likely to assume that their users have access to various other pieces of information, that the story has already broken elsewhere, etc.

There are debates concerning whether some content available online is entirely ethical; e.g. the execution video of Saddam Hussein.  Do you think there is a need for some form of gate-keeping?

Firstly, we need to remember that ethics are culturally dependent: what appears offensive to some cultures will be acceptable in others, including some images that UK users might find quite upsetting.

This become problematic when we move to a global pubishing environment in two ways: firstly, we have access to information from cultures with different ethical frameworks and tastes; and secondly, we are open to accusations of censorship from members of those cultures if we refuse to publish footage which they are aware of.

With that established, you then have to weigh up the advantages and disadvantages of establishing some sort of gatekeeping structure on the internet to somehow ‘protect’ people from information that may be offensive.

This gatekeeping already exists – for example, nazi memorabilia online in France, or Holocaust denial sites in Germany. But any level of gatekeeping is open to abuse and that should be recognised: proposals to allow sites to be shut down based on accusations of copyright abuse, for example, may sidestep due process and have potentially damaging implications for free speech (imagine a shop being closed down because ‘someone’ says it is illegal, or because the shop next door is illegal, i.e. shares hosting).

Your own example is a good one: to find that video, you have to seek it out. Therefore, you are taking on responsibility for that. If a media organisation shows you it, then they take on some responsibility.

But should they decide whether you are allowed to seek it out at all? And who decides who ‘they’ are?

How would you define a professional journalist in an age where anyone is able to publish online? Would you class a blogger as an online journalist?

A blogger is someone who uses a blog to publish content. The term is based on platform, not the content itself, so you can’t say a blogger is or isn’t a journalist. As I’ve written before, it’s like asking “Is ice cream strawberry?”

A journalist is someone who practises journalism – it’s as simple as that. Being employed by a media organisation is not enough alone (otherwise ad sales, marketing, distribution and other staff would also be ‘journalists’).

So you then look at definitions like Stuart Adams’s. I think it’s pretty broad, but also you have to ask: why does it matter what we call someone? Is it ego?

Do you believe that bloggers and other citizen journalists should be expected to work under the same codes of practice as professional journalists?

No, for the simple reason that professional journalists don’t all work under the same codes of practice.

A journalist chooses to work under a code of practice in two ways: through joining the NUJ or similar professional body, and by doing so signing up to their code of conduct; and through becoming an employee of a publisher who has signed up to a code of conduct (that might be the PCC, Ofcom, or neither) and may have their own internal one too.

Bloggers and CJs have the same choice. As publishers themselves, they can write their own code of conduct. They can join the NUJ or another body which has a code. Or they can abide by a personal code of conduct which is implicit in their work. But that’s their choice, just as it is the choice of journalists and publishers.

PHP Freelancer

A number of news outlets – including the BBC, Guardian and Channel 4 News – mentioned yesterday in their coverage of the closure of the News Of The World that TheSunOnSunday.com had been registered just two days ago. (It was also mentioned by Hugh Grant on last night’s Question Time.)

It’s a convenient piece of information for a conspiracy theory – but a little bit of digging suggests it’s unlikely to have been registered by News International as part of some grand plan.

When I tweeted the claim yesterday two people immediately pointed out key bits of contextual information from the WHOIS records:

Firstly, it is unlikely that News International would use 123-reg to register a domain name. @bigdaddymerk noted, News International “use http://bit.ly/cWSHia for their .coms and have their own IPS tag for .co.uk”

Murray Dick added that it would “be odd for big corporation to withhold info on whois record”

And – not that this is a big issue given recent events – according to @bigdaddymerk “in the case of the .co.uk registering as a UK individual would be whois abuse.” (UPDATE: The specific abuse is detailed here)

You might argue that the above might be explained by News International covering their tracks, but if were covering their tracks it’s unlikely they’d do it like this.

UPDATE: From Malc. in the comments: more digging has been done at Loutish – note the comments as well.

UPDATE 2: It seems there are other web addresses registered by other companies, too. This post points out, however, potential trademark issues (none has been registered) and conflict with Trinity Mirror.

UPDATE 3: Those other addresses are now registered to News International – but not the .com domains.

UPDATE 4: I think News Corp missed an opportunity with FoxNewsUK.com…

The timeline

Anyway, digging further into the timeline of the ‘Sunday Sun’ casts further doubt on any conspiracy connected to News Of The World.

For example, it was reported over a week ago that The Sun was moving to 7-day production (thanks to Roo Reynolds, again on Twitter).

Between that announcement and the registration of TheSunOnSunday.com, anyone with a habit of domain squatting could have grabbed the domain in the hope that it would become valuable in the future.

Either way, even if it has been registered by someone at News International, the timings just don’t add up to a News Of The World-related conspiracy. Certainly it will have been a factor in deciding to close the NOTW, and plans to launch a Sun On Sunday are now likely to be accelerated (I’m amazed that they hadn’t registered the domains before, at least as a defensive move) – but it’s pretty clear that those plans pre-date the closure of NOTW.

So, as I wrote yesterday, a ‘Sunday Sun’ is not a rebranding of News Of The World. They have just closed the country’s biggest selling newspaper – its most profitable tabloid – and made 200 people redundant.

Note: this post was udpated to correct an error: the NOTW is not the highest selling English language newspaper in the world (that is probably The Times of India). Thanks to Paul Carvill in the comments for highlighting.

PHP Freelancer

I’m holding a one-off training day in August on verifying information online and finding sources, in London at the Royal Society of Medicine.

In the context of various straight men pretending to be gay women, it’s quite timely.

PHP Freelancer

Google have launched a ‘Search by Image’ service which allows you to find images by uploading, dragging over, or pasting the URL of an existing image.

The service should be particularly useful to journalists seeking to verify or debunk images they’re not sure about.

(For examples where it may have been useful, look no further than this week’s Gay Syrian Blogger story, as well as the ‘dead’ Osama Bin Laden images that so many news outlets fell for)/

TinEye, a website and Firefox plugin, does the same thing – but it will be interesting to see if Google’s service is more or less powerful (let me know how you get on with it) Find it here. Video here.

PHP Freelancer

Both the Daily Mail and the Daily Mirror today – among with several others in the US (including the New York Post, which credits the image to AP) and other countries – published an image purporting to be that of the dead Osama Bin Laden.

It clearly wasn’t.

Any journalist with a drop of cynicism would have questioned the source of the images – even if they did appear on Pakistan television.

It certainly passed the ‘Too good to be true’ test.

Instead, it was users of Reddit and Twitter who first highlighted the dodgy provenance of the image, and the image it was probably based on. Knight News and MSNBC’s Photo blog‘s followed soon after.

It took me all of 10 seconds to verify that it is a fake – by using TinEye to find other instances of the image, I found this example from last April.

But instead of owning up that their image was a fake, both The Daily Mail and Mirror appear to have simply removed the image from their site, leaving that image to circulate amongst their users. Ego, pure and simple.

PS: More on verifying images and other hoax material here.

PHP Freelancer

Various parts of the media were hoaxed this week by Belfast student Jonny Campbell’s claim to have won a Twitter internship with Charlie Sheen. The hoax was well planned, and to be fair to the journalists, they did chase up documentation to confirm it. Where they made mistakes provides a good lesson in online verification.

Where did the journalist go wrong? They asked for the emails confirming the internship, but accepted a screengrab. This turned out to be photoshopped.

They then asked for further emails from earlier in the process, and he sent those (which were genuine) on.

They should have asked the source to forward the original email.

Of course, he could have faked that pretty easily as well (I’m not going to say how here), so you would need to check the IP address of the email against that of the company it was supposed to be from.

An IP address is basically the location of a computer (server). This may be owned by the ISP you are using, or the company which employs you and provides your computer and internet access.

This post explains how to find IP addresses in an email using email clients including Gmail, Yahoo! Mail and Outlook – and then how to track the IP address to a particular location.

This website will find out the IP address for a particular website – the IP address for Internships.com is 204.74.99.100, for example. So you’re looking for a match (assuming the same server is used for mail). You could also check other emails from that company to other people, or ideally to yourself (Watch out for fake websites as well, of course).

And of course, finally, it’s always worth looking at the content the hoaxer has provided and clues that they may have left in it – as Jonny did (see image, left).

For more on verifying online information see Content, context and code: verifying information online, which I’ll continue to update with examples.

PHP Freelancer

When the telephone first entered the newsroom journalists were sceptical. “How can we be sure that the person at the other end is who they say they are?” The question seems odd now, because we have become so used to phone technology that we barely think of it as technology at all – and there are a range of techniques we use, almost unconsciously, to verify what the person on the other end of the phone is saying, from their tone of voice, to the number they are ringing from, and the information they are providing.

Dealing with online sources is no different. How do you know the source is telling the truth? You’re a journalist, for god’s sake: it’s your job to find out.

In many ways the internet gives us extra tools to verify information – certainly more than the phone ever did. The apparent ‘facelessness’ of the medium is misleading: every piece of information, and every person, leaves a trail of data that you can use to build a picture of its reliability.

The following is a three-level approach to verification: starting with the content itself, moving on to the context surrounding it; and finishing with the technical information underlying it. Most of the techniques outlined take very little time at all but the key thing is to look for warning signs and follow those up.

Level 1: Content

At its most basic level, alarm bells should ring if the information you’re looking at is simply too good to be true. The disgruntled sacked employee who makes lights up the exterior of Harrods with a farewell message fits this category. Ask yourself: would this really happen? And if so, who else would have known about it?

If the information is coming through social media you have to ask: is this bait? Jan Moir’s Twitter ‘apology’ is one good example – lending itself to easy retweeting. Peter Serafinowicz’s ‘deleted’ offensive joke is another. So are various Facebook rumours, such as paedophiles who want you to change your profile picture, or party gatecrashers, and the occasional protesting Facebook group. And forum rumours (sometimes placed intentionally to expose journalists who plagiarise without giving their source). And press releases.

Embarrassing emails that go viral can turn out to be PR tricks. Video diaries can be revealed as new forms of narrative. Spectacular video footage can turn out to be more PR (by the way, read through that thread to see how it is infiltrated by a PR person but their identity is challenged). Check the facts, and see what other people have uncovered. And click on all of these links: the more hoaxes you are familiar with, the more likely alarm bells are going to ring at the right time.

The frequency and recency of information will give you a clue as to its veracity: the more recent the information, the more up to date it is likely to be (although it may be based on out of date information – trace it back to its source). And the more frequently a source is updated (over a long period of time), the less likely it is to come from an opportunistic hoaxer. You can get browser bookmarklets that tell you when a webpage was last updated (as well as many other pieces of information).

Finally does the style and personality of the information match the supposed source? Do they write in the same tone? Do they make spelling mistakes?

For images look for cloning and airbrushing. Cloning is the replication and repetition of small areas of a photograph to, for instance, make a crowd look bigger by duplicating faces; make an air attack look more dramatic by adding extra plumes of smoke, or to make an operations room look more active by filling blank screens.

Airbrushing is the removal of details – the Harrods image mentioned above was most likely created in this way, by removing lights so that those remaining spelled out the message. Also worth watching for are composite or staged images, such as the various Google Street View hoaxes.

This article suggests that inconsistent lighting, eye shapes and light reflections within eyes are all good clues to look for as well. And this related infographic allows you to explore how one image has been retouched. This article by Judith Townend goes into more detail about spotting manipulted images.

Level 2: Context

Social media lends itself particularly well to verification because, in our activity in social networks, we effectively verify each other. If your information comes from a social network account, ask yourself some of these questions:

How long has the account existed? If it’s only existed since a relevant story broke (e.g. Jan Moir’s column; an earthquake where someone claims to be a witness) then it’s likely to be opportunistic.

Who did the person first ‘follow’ or ‘friend’? These should be personal contacts, or fit the type of person you’re dealing with. If their first follow is ReadWriteWeb, then it may be that you’re not actually dealing with a Daily Mail columnist.

Who first followed them? Likewise, it should be their friends and colleagues.

Who has spoken to them online? Ditto.

Who has spoken about them? Here you may find friends and colleagues, but also people who have rumbled them. But don’t take anyone else’s word for their existence unless you can verify them too.

Can you correlate this account with others? The Firefox extension Identify is a useful tool here: it suggests related social network accounts which you can then try to cross-reference. For companies the Chrome extension Polaris Insights does something similar for companies.

For Twitter you might also try other tools including PeerIndex and Klout, both of which use algorithms to give extra information on the ‘human-ness’ and content of particular accounts. On Facebook there is the social commenting plugin which attempts to give a credibility score to commenters.

Finally, of course, you should try to speak to the person. Phone their office or their employer and confirm whether they do indeed have the account in question.

For websites the checks are broadly similar. On Google you can use the advanced search facility to look for other pages that link to the one you’re checking. These might include other website that have rumbled the hoax before you – or are bragging about it.

Similarly look what links the webpage contains to other sites: does this fit what you would expect? The browser bookmarklets mentioned above will collate these for you. At this point we’re starting to move onto the third level…

Level 3: Code

First, look at the website address. If it is purporting to be a governmental website it should end in .gov, .gov.uk etc. Health websites may end in .nhs, police in .police, defence in .mod and so on. Academic websites should end in .ac.uk or .edu but this is no guarantee: less reputable ‘establishments’ have managed to obtain web addresses with these extensions. And of course .com addresses offer no guarantees.

Murray Dick gives more advice on the other elements of a web address, and recommends using an open directory to check your searches, as these are maintained by people, not computers, are less likely to contain hoax websites.

Use a Whois service to find out who the web address is registered to. This isn’t immune to fakery but the hoaxer may not have thought about it, and if the details are hidden you may wonder why. Try variations of the domain – when the viral ‘Labservative’ campaign first began it was not clear who was behind it, and I started by looking at Whois details – the company had kept their details private for the .com address, but they had forgotten to do so for the .co.uk variation. I then called up the company and tried to call their bluff by asking who was managing the campaign.

If you are asking for emails verifying a story, make sure you are forwarded the original email, and not a screengrab, and follow this process to check the IP address of the email against who it’s supposed to be from.

Archives and caches can be useful to compare the latest version of a webpage with older versions. Conducting a relevant Google search and clicking on ‘cache’ next to the relevant result can show up recent changes. The Internet Archive‘s Wayback Machine (recently revamped) can give you snapshots going further back. On Wikipedia and other wiki-based sources, look for ‘history’ and ‘discussion’ links where you can see what changes have been made and the discussions about those.

For images you can check out the EXIF data – this is information about when the image was taken, on which camera, and with what settings. This online tool (there’s a Chrome extension too) allows you to quickly see the EXIF data on any web-based image. This information is best used when speaking to the photographer – ask them when to give you the details that you can verify against the EXIF data. This isn’t a foolproof method but it will screen out most hoaxers.

Some news organisations – such as the BBC, in its UGC hub – have systems that look for Photoshop modification (not necessarily a sign of hoax – a user could simply have cropped or lightened an image). You can also see this yourself by looking under “details” > “origin” > “program name”. JpegSnoop will provide more details on images. Error Level Analysis is another useful tool to detect possible alteration, although it’s not perfect.

Finally, right-click on the page and view the source code. Occasionally hoaxers intentionally leave clues here, but you can also find other clues such as the author, date, location, and technologies used.

Any other techniques?

Those are just the techniques and tools that I can call to mind but I’m sure there are others I’m not aware of. Any you can suggest?

UPDATE: The BBC College of Journalism’s post on verifying content adds some other useful tips on cross-verification with maps, weather reports and other details.

PHP Freelancer

The first study (PDF) of magazines and their various approaches to websites, undertaken by Columbia Journalism Review, found publishers are still trying to work out how best to utilise the online medium.

There is no general standard or guidelines for magazine websites and little discussion between industry leaders as to how they should most effectively be approached.

Following the responses to the multiple choice questionnaire and the following open-ended questions -

• What do you consider to be the mission of your website, does this differ from the mission of your print magazine?
• What do you consider to be the best feature of aspect of your website?
• What feature of your website do you think most needs improvement or is not living up to its potential?

- the researchers called for a collective, informed and contemporary approach to magazine websites with professional body support.

The findings were separated into the following 6 categories:

Staff Structure and Decision Making

The researchers found decision making on the website to be the single most important factor in how its website functions.

Most websites were staffed by people who primarily worked on the print editions, and less than a quarter of staff were hired with web experience (29 per cent).

Independent web editors were the only decision makers in the most profitable websites, and the higher a magazine’s circulation and monthly web traffic, the more likely it was to have an independent web editor making budget and content decisions.

Standards and Practices

The researchers found the approach to fact-checking and sub-editing for online content website standards were in general much less rigorous than for printed editions; 51 per cent of original content that appears on web sites is either not copy-edited at all, or is copy-edited less rigorously than in print.

Just under half (43 per cent) of respondents reported either a lower standard for fact-checking online (35 per cent), or no fact-checking at all (8 per cent).

Strangely, they found that websites are more likely to have lower standards in these areas as web traffic rises and when content decisions are made by independent web editors.

Many website editors correct errors without acknowledging the mistake; they are often more likely to be corrected than print, but less likely to publicise the correction – particularly when an independent web editor is involved.

The most common reason for material to appear online is because it ran in the print edition, often because it is breaking news, multi-media content or to maintain freshness and, sometimes, because the quality is not high enough to run in the print edition.

Business Model

For 68 per cent of surveyed publishers, advertising is the largest revenue source – just over half of the magazines (52 per cent) offer all their print material online for free, and profitable sites offer all of their content online for free more often than non-profitable ones.

Only about a third of magazine web sites make a profit, and magazines that publish more frequently, and those that have a higher web traffic, tend to have more profitable web sites.

However, they found magazine circulation generally has little bearing on web site profitability.

62 per cent of the web sites with between 1.5 million and 2 million unique monthly visitors were profitable, compared with 21 per cent of those with less than 50,000 unique monthly visitors.

Social media and community building

Unsurprisingly, most web sites, (47 per cent), have adopted social media tools and techniques, and do so more when independent web editors are in decision-making roles.

However, editorial standards tend to slip even more in this environment. Blogs are rarely copy-edited or fact-checked and comments are moderated at editors’ discretion.

Most magazines have blogs on their Web sites (64 per cent), and those are mostly maintained by staff members (87 per cent); 39 per cent use freelancers or contract-writers for blogs.

Web sites are more likely to have blogs when independent web editors are in charge of the budget. Most magazines allow comments on blogs or other online content (73 per cent).

Technology

The researchers found most magazines are not keeping pace with mobile display and interactivity technology.

Less than one in five are designed for smartphones and very few are formatted for e-book readers (4 per cent).

Again, web sites are more likely to have multiple display options when independent web editors are in charge of budget or content decisions.

Roughly half of magazines surveyed use metrics to guide content decisions (47 per cent), but only 8 per cent closely monitor and rely on them.

Less than half use traffic statistics (43 per cent), and those that do so regularly for content decisions are significantly more likely to be profitable.

Web sites that receive more traffic are more likely to use traffic statistics in content decisions.

Most magazines name Google Analytics as the online metric that is most helpful to their web sites.

Content management systems vary, with custom-designs proving most popular.

Mission

Most editors said their website and their print magazine shared a common mission.

16 per cent of respondents said their Web site’s mission involved community-building with readers.

Interestingly, only 5 per cent mentioned new or unique content as integral to the site’s mission, with 96 per cent reporting the primary use of content from the print magazine online.

In conclusion, the researchers call for a “Habermassian convention” to continue the discussion of issues raised by the study.

They suggest an inclusive and wide-ranging approach to help foster the democratic ideal of the public sphere in online publishing, to address the challenges for the future of journalism and of online business models.

Specifically they call for the following questions to be addressed:

• What is keeping web experience out of magazines and why?
• Why doesn’t the industry create codes of conduct and guidelines on matters such as online fact-checking, copy-editing, and error-correction?
• Is it true, as one respondent said, “if it’s fact-checked, it’s not a blog,” and is this an existential or a definitional question?
• Subject for discussion: Why have earlier attempts at standardizing the world of blogs and social media notoriously failed? Is it, at long last, possible to identify best practices for using the tools and techniques of digital journalism?

PHP Freelancer

This article frames the problem of news dissemination as a problem of market lemons, analogous to the issue raised by George Akerlof in 1970. Framing news as a mechanism of vetting common knowledge as opposed to entertainment allows one to see that instant common knowledge in the byzantine and uncertain way in which humans communicate and live in is unattainable. Given this frame of the problem a potential solution is posited which allows traditional newspaper companies to serve and focus on the role of validating news rather than simply creating or capturing it. The most value added service that traditional news organizations can provide is validation of truth and quality assurance.

“It is hard to get the news from poems, but everyday, men die miserably for lack of what can be found there.” (William C. Williams)

Introduction

Gauging quality of entertainment is fairly simple and self-evident. Consumers know instantly whether a product is entertaining and consumers continue to pay attention if they find the material to be entertaining.

News providers tend to serve both an individual’s desire for entertainment and information in one product bundle. Although it is very easy for consumers to test the quality of the entertainment component of news it is much more difficult to gauge the information quality of news.

Consumers face the intangible dilemma of assessing whether news is accurate or true, which poses a problem of asymmetric information for consumers.

Despite the availability of virtually infinite potential news sources and automated search engines, the search costs of getting the truth are too high. Human beings are bombarded with information throughout the day and despite the ease of search engine technology only 28% of the internet is actually available for search (Barabasi, 2002). The internet is growing in content exponentially and current computing cannot search the majority of the internet.

The threat of news becoming a market for lemons is an important issue worth exploring as news serves to provide a gatekeeping and watchdog function in democracies.

Although it might appear that the advent of increased competition for news via independent and unbiased bloggers on the internet would improve news quality this may not be true in practice. Without a way to assess the accuracy and quality of the information the market of news on the internet tends toward a market for news lemons.

Shleifer’s research on the market for news shows that competition is not enough to ensure accurate news and that, ironically, competition results in “lower prices, but common slanting toward reader biases” (Shleifer and Mullainathan, 2005).

Shleifer posits that ‘a reader with access to all news sources could get an unbiased perspective’ and that ‘reader heterogeneity is more important for accuracy in media’ (2005).

That said, the issue of search costs of consumers has not been explored as in practical terms as no reader has time to read all news sources to form a perfect model of unbiased information.

The problem of assessing the validity of news quality is in essence the ‘market for lemons’ problem raised by Akerlof (Akerlof, 1970). The market for lemons phenomenon relates to ‘quality and uncertainty’ and news is clearly a business in which “‘trust’ is important” and, as Akerlof points out, “Informal unwritten guarantees are preconditions for trade and production” and “where these guarantees are indefinite, business will suffer” (1970).

The aim of this paper is raise the issue of the market for internet news lemons as the quality of free information served piping hot on the internet is ‘indefinite’. When the quality of a good is unknown consumers are willing to pay for it, assuming it is not reliable, and thus this drives sellers with a good product out of the market as the consumer is unable to determine high quality from low quality goods.

Akerlof showed the detrimental effect of markets for lemons using the case of used cars in the 1970s where people with good used cars could not obtain the price their car was worth and would not sell their cars, thus leaving the market full of lemons in a self-fulfilling prophecy of sorts. Similarly, any market for good where the quality of the product is uncertain tends to a market for lemons.

This phenomenon has been at play in the mortgage securities market and is no different for news as a product.

Towards a definition of news and newspaper quality

News as a system for humans provides the following affordances to humans:

• connects people with information,
• provides branding of perceived truth,
• helps support democracy and its ideals, and
• fulfills an entertainment component via narrative integrity.

The narrative integrity itself has recently been criticized by Taleb as it encourages readers to build unrealistic assessment of risk in financial and other aspects of daily life (Taleb, 2005). Newspapers in general tend to either exaggerate or under-represent risks faced by individuals and are not sound guardians from a risk management point of view.

Quality for a news product is a perception of validity and truth amongst peer groups that consumers communicate with. Most consumers of news want to know what is going on. What is big? News thus functions to provide roles of gatekeeping, watchdog, anti-corruption, and in general a sharing of true facts of interest to human communities in relation to purported values and themes.

The existence of a strong free press has been associated with lowered corruption across nations (Brunettia & Wederb, 2003). In a study of government ownership of the news media, which is the case in 97% of countries, it was found that per ‘public choice theory…government ownership undermines political and economic freedom” (Schleifer, Djankov, Mcliesh & Menova, 2003).

Scoping News

For the scope of this work the emphasis will be on the non-entertainment quality aspects of news as a product. This is consistent with Shleifer’s definition that the ‘quality of [news] information is its accuracy. The more accurate the news, the more valuable is its source to the consumer. Pressure from audiences and rivals force news outlets to seek and deliver more accurate information, just as market forces motivate auto-makers to produce better cars’ (Shleifer, 2005).

Hamilton’s book on the economics of news highlights the fact that news is meant for rapid commoditization, it is information good and is a product of network effects (Hamilton, 2003). Per Hamilton’s point, speed of delivery, accuracy, and relevancy seem to be desirable characteristics of news as a product (2003).

If we step back and look at this, news is really a mechanism of generating ‘common knowledge’ within a byzantine environment where quality and truth are uncertain.

Taking this perspective one can see that the work in artificial intelligence and philosophy conducted by Halpern and Moses is relevant in this context (Halpern etal, 1984). Halpern and other students of common knowledge find that in practice it is impossible to guarantee reliable and true common knowledge in real time. The closest one can get to is almost common knowledge (Halpern etal, 1994).

Given the complex nature of the problem of common knowledge in a distributed uncertain environment Halpern et al point out that the modeling of time is critical in achieving eventual common knowledge. One way to look at this is, given that a consumer wants common knowledge, they should wait a sufficient time until a news story can be vetted. The expectation of instant and true knowledge is a pipe dream, as Eugene O’Neil would say.

One side effect of the current market equilibrium for news is the segmentation of the market for news into the following groups of people:

1. people who don’t read the news,
2. people who the read news to interpret facts to suit agendas i.e. politicians, lobbyists etc, and
3. people who read what they want to believe and are aware of it.

I believe this segmentation exists due to high search costs for the truth.

I personally don’t read the news much at all. If I am interested in a topic I research the field, get input from experts, and make my own inferences. I of course do not engage much in casual conversation. For the majority of citizens who do, news is an invaluable source to relate with others and share experiences of ‘true events’ and common knowledge.

Noted anthropologist Roy Wagner has pointed out the pervasive problem of information which humans grapple with:

“Persuasion, from the days of Aristotle onwards, never works as it is intended to and has its greatest effect on the persuaders … To the extent that the vast, worldwide communications industry, the media, the internet or Web, the ubiquitous ‘sensory’ modes and guidance-circuitries use ‘information’ or ‘communication’ as code words for what is really going on, we live in a world that is actually created by a failure of persuasion.

“This means that we live in a world of information-stealth – the half truths of our lies and the lies of half truth  - or what the CIA, or at least its critics, would call disinformation. I wouldn’t be kidding you, now would I? Disinformation has a far more ambiguous or ambivalent effect than persuasion ever could have and is both more informative and communicative than its buzz-word surrogates. It works on a ‘leakage’ principle, partial truths leaked out in the telling of deliberate lies and deliberate lies leaked in the telling of partial truths. It is motivated by goals and objectives that have nothing directly to do with either belief or conviction on one hand or doubt and cynicism on the other; it offers deniability with both hands. ‘It is either half true,’ as the Viennese aphorist Karl Krauss said of the aphorism or ‘one and a half times true’.

“We are unconvinced (e.g. apathetic) on one hand, and overconvinced on the other, and the middle ground is the most contested of all … Disinformation rules the world, and it does so through ‘deniability’. We know for a fact that every single trade, occupation, and especially profession has its secrets, known to its initiates and unknown to others.” (Wagner, 2000)

The last piece applies to journalists as well.

Potential solutions: a new business model for news

To date innovation in news has been focused on either transforming traditional media into high tech companies, which is unlikely, and the adoption of the market niche strategy of hyperlocal news.

The model of niche and differentiation/specialization has potential but is perplexed with the issue of changing interest and taste. How does one know which hyperlocal news is of interest? With limited time and highly contested attention spans hyperlocal news is a difficult to maintain proposition. That said, given non-profit and community support it can work as a niche solution.

The solution we propose here is targeted to larger well established news players and is a novel approach to the problem.

Traditional print sources like the Washington post etc. have a platform and reputation for checking and ensuring high quality information. The expertise that existing print media companies have can be used to focus on validation and authentication of breaking news stories, as on the internet there is no authority for the validity of news.

One innovative solution to the market of news lemons problem might be for traditional news papers to create reputation-based blogging spaces where stories are tested and validated before publication. This is consistent with the work of Yamagishi who studied the market-for-lemons problem in online trading and found an online reputation system to be a useful solution to the problem (Yamagishi, 2002).

Yamagishi noted that online trading results in “information asymmetry” which “drives the … market into a lemons market” (2002). This is analogous to the problem of news consumption.  Yamagishi’s analysis segments reputation into 2 forms: positive and negative reputation. Yamagishi finds the openness of internet trading precludes negative reputation and  “promotes positive reputation as an effective means for curtailing the lemons problem” (2002).

An important aspect of understanding why negative reputation is not effective on the internet is that it is too easy to switch and create new identities. Thus methods of “inclusion” which validate positive reputation are critical to combating the lemons problem (2002).

Per Yamagishi’s suggestion, existing newspapers with positive brand reputations have value as providers of positive reputation in an open market of internet news.

An enterprise devoted to assuring quality of the news could be a new hybrid form of existence for traditional newspapers in which the goals of the news system is preserved.

The price differential paid to the news companies would be based on their quality of checking and not on slant of the news or sensational quality of it.

Under this model papers would specialize in news domains with expertise and offer objective validation stories. For true objectivity the influence of advertising profit would need to be removed. Perhaps the advertising revenue would accrue to content providers who provided the stories along with advertisements which underwrite the authors. The news intermediaries who select the stories based on quality and validation would be paid only for quality assurance.

A successful example of dealing with ‘cyber lemons’ was that of an ‘online intermediary’ used by China’s largest online consumer-to-consumer trading site, which built a “credit evaluation system to serve as a quality-intermediary and reputation” (Pan, 2005).

In short, several eBays for news, specializing in different news domains, would serve to mitigate the lemons problem.

The newspaper industry must face the disintermediation of its power to dictate the news agenda. The notion that a few, supported by commercial advertising, would decide what was newsworthy was paternalistic and with the disintermediation of this component the responsibility of what to pay attention to falls on society. This issue itself is best tackled through education and the fostering of civic and democratic ideals in youth.

Dhruv Sharma is an independent scholar in the fields of organization behavior, risk management, artificial intelligence, and systems engineering. A graduate of the McIntire School at the University of Virginie, he holds a Masters in Systems Engineering and a Masters in Organizational Development from Marymount University.

This article is dedicated to Emma Brown, a greater writer and journalist and George Akerlof, the great economist.

Citations:

• Akerlof, GA. (1970) The Market for “Lemons”: Quality Uncertainty and the Market Mechanism. The Quarterly Journal of Economics, Vol. 84, No. 3. (Aug., 1970), pp. 488-500
• Barabási, A.L. (2002) Linked: The New Science of Networks, Perseus, Cambridge
• Brunettia, Aymo & Wederb, Beatrice (2003) A free press is bad news for corruption. Journal of Public Economics 87 (2003) 1801–1824
• Hamilton, James T., (2003), All the News That’s Fit to Sell. Princeton, NJ: Princeton University Press
• Halpern,J.Y. and Moses,Y. (1984). Knowledge and common knowledge in a distributed environment. Journal of the ACM, 37(3):549–587, 1990. A preliminary version appeared in Proc. 3rd ACM Symposium on Principles of Distributed Computing
• Halpern,J.; Fagin, R; Moses,Y. and Vardi,MY (1994). Common knowledge revisited. Theoretical aspects of rationality. Proceedings 6th Conference. Retrieved from http://www.cs.cornell.edu/home/halpern/papers/ck_revisited.pdf
• Schleifer, A. Djankov, S., Mcliesh, C. Menova, T. (2003) WHO OWNS THE MEDIA? Journal of Law and Economics. vol. XLVI
• Shleifer, Andrei & Mullainathan, S. (2005) The Market for News. The American Economic Review
• Taleb, N.N. (2005) “THE OPIATES OF THE MIDDLE CLASSES” Retrieved from http://www.edge.org/3rd_culture/taleb05/taleb05_index.html
• Yamagishi, T. Masafumi, Matsusa. (2002) Improving the Lemons Market with a Reputation System: An Experimental Study of Internet Auctioning. Retrieved from http://joi.ito.com/archives/papers/Yamagishi_ASQ1.pdf Hokkaido University
• Wagner, Roy (2000) “Our Very Own Cargo Cult”. Oceana

PHP Freelancer