This week the UK government released a report into social mobility. While mainstream reporting focused mainly on the broad picture, I wanted to read the original government report itself. Which publishers linked to it?

• The Telegraph: fail. Not one of the 4 articles I could find linked to the report.
• The Times: fail. Alan Milburn’s own piece about the report fails to link to it. These articles don’t either.
• The Independent: fail, despite having more articles on the issue than other websites.
• The BBC: links very clearly to both a summary (PDF) and the whole report (PDF). Curiously, however, both are hosted on the BBC’s own site.
• Sky: fail. Oh, and an appalling search facility – top result for a search on ‘Milburn Report’? From 2002.
• ITN: fail.
• Reuters: fail.
• Channel 4 News: no link on the video report, but there is a link below a line at the end of this story. You have to scroll to see it. Although it’s labelled as an ‘external link’ the PDFs are hosted on C4′s own site.
• The Guardian: mixed. This article didn’t and nor did this; but this one did – albeit in par 5, three pars after the report is first mentioned. Notably, 2 pieces on their blogging platform Comment is Free did - both in the first paragraph, no less, and to the Cabinet Office version.

I’ve written and spoken extensively on the importance of linking, but it comes down to 2 core reasons:

Firstly, Google will rank a page more highly if it includes more outgoing links.

Secondly, people will return to your site more often if they know they can expect useful links.

So, get your act together, please what are news organisations doing to address this?

PHP Freelancer

Searching is the most popular activity online after email. It is the prism through which we experience a significant proportion of the world’s information – from news and information about our community, through to health information, commerce, and just about anything that has a presence online.

Search Engine Society takes a critical look at search engines, how they work, the techniques used to manipulate them – from gaining better rankings to censorship, and the implications for privacy and democracy.

Chapter one looks at the development and workings of search engines, from the once-essential directories of Yahoo! and the citation-based algorithms of Google that now dominate the search landscape, through to lesser-known players such as social bookmarking service Delicious which relies on user-generated ‘folksonomies’ to organise material, and specialised regional and ‘vertical’ search engines like the French language Voila or the genetic materials search engine The Bioinformatic Harvester. This is situated within a wider discussion of information retrieval histories from the Library of Babylon onwards – and touches on recent moves into geospatial, mobile, social and semantic search.

Balancing that focus on technology, the following chapter focuses on users, looking at how people search. Search behaviours vary widely between users and between searches – Halavais discusses research that showed how many users simply add ‘.com’ to a word as the start of their search, while others use a ‘shopping mall’ approach of going direct to the likes of Wikipedia and the Internet Movie Database (which also contain search facilities). Using a search engine, Halavais argues, is only one method of search, and search is “not only an iterative process, but one that is rarely linear and requires seeking out the concepts that surround a problem or question. In other words, the query and search strategy is likely to change as more information becomes available.”

Search as ‘re-finding’

Halavais also emphasises the importance of ‘re-finding’ – “not as a sub-set of finding, but the other way around” – indeed, this is the basis of social bookmarking services like Delicious and Digg that allow the user to store and label (‘tag’) webpages for later retrieval, as well as searching for webpages that have been given similar tags by other users.

Power law distribution patterns famously recur throughout the web and in the third chapter Halavais looks at how this affects search results. With Google’s rankings relying so strongly on how many links point to a particular page, it is important to look at how those links are distributed. The fact that highly linked pages are likely to attract ever more links – what Huberman calls “preferential attachment” – leads to the “chunky” nature of the web – in concrete terms the dominance of websites like those of the BBC and Guardian; a quality which, Halavais argues, Google’s PageRank technology ‘calcifies’.

But when Google tweaks its search engine algorithms to attempt to improve results, it can have enormous consequences for organisations dependent on their rankings in search results. Halavais uses the example of Skyfacet.com and Answers.com which saw sales and visits drop by 17% and 28% respectively when they dropped off the first page of related Google searches. It is as if someone moved your shop from the main high street to an industrial estate. In this context it is not surprising that search engine advertising accounts for the majority of online advertising spend.

Digital divides

Following up on those issues, the fourth chapter looks at implications for democracy on two sides: firstly, the division between winners and losers in the contest for public attention; and secondly, the division between skilled and unskilled users of search engines. Halavais is keen to highlight that division is nothing new:

“Current search engines, like communication technologies before them, contain both centralizing and diversifying potentials. These potentials affect the stories we tell ourselves as a society; and the way we produce knowledge and wisdom.”

In practice, these potentials are heavily weighted towards US sites:

“In the language of PageRank, US sites simply have more authority: more links leading to them … sites have existed longer in the United States, where much of the early growth of the internet occurred… Add to this the idea that early winners have a continuing advantage in attracting new links and traffic, and US dominance of search seems a foregone conclusion … the search engines do not merely reflect this authority, they help to reproduce it.”

Indeed, ranking systems that reinforce authority, says Halavais, are conservative in nature and comprise what Lewis Mumford, writing 40 years ago, called “authoritarian technics”.  But because of the unlimited size and reach of the internet compared to previous media technologies, it is not so simple:

“The current structure is a complex combination of a high degree of centralization at the macro-level, with a broad set of diverse divisions at the micro-level.”

Blogger as ‘search intellectual’

Interestingly, at this point Halavais introduces the blogger as a “search intellectual”, upsetting existing structures of authority on the web and acting as “a counterweight to the hegemonic culture of the search engines” in bringing otherwise overlooked material into the “circle of reputation and links that search engines tend to enforece”. The recent rise of Twitter in performing a similar role would be worth adding to that list.

Chapter 5 takes a broad look at censorship – “just another word for filtering” – while Chapter 6 looks at privacy – search engines as “databases of intentions” where even anonymised logs of what individuals are searching for can lead to people being identified. Chapter 7 revisits the rise of “sociable search” tools and folksonomy – where classification is created by a mass of users’ ‘tags’ rather than any centralised scheme, and ‘finding’ is a social act closely related to ‘sharing’.

The book closes with a roundup of the possibilities of future search and the factors that will influence that, from increasing digitisation of material to improved mapping and the possibilities of RFID tags (which makes objects a part of the web too). Semantic search – technology that understands the meaning of what you are searching for, or of relationships between objects – is the promise that lies forever ‘just over the horizon’, while sociable search offers a more likely immediate move.

As is natural, there are areas which have developed since this book was written and so are not tackled in depth – most notably real-time search. The rise of Twitter and the ability to search through what people are talking about ‘right now’ represents such serious competition to Google that it introduced the first major new features to its homepage in years. Wolfram Alpha – the “computational knowledge engine” that made newspaper front pages this year – is not even mentioned.

But those are incidental issues in what is an important book. Halavais manages to acknowledge the dominance of Google without being distracted by it, and gives due attention to non-Western tools and services not commonly seen as search tools. He avoids the pitfalls of technological determinism and manages to distinguish between top-down domination and bottom-up diversity. What emerges is a sophisticated picture of power in flux. “Search engines are interesting to the person who wants to understand the exercise of power in the information society,” Halavais writes in the his conclusion. “In an era in which knowledge is the only bankable commodity, search engines own the exchange floor.” The more readers understand this exchange floor, the better we can exchange and interrogate what information we possess.

A shorter version of this review will appear in Journalism

PHP Freelancer

That’s the question bumping around my mind after reading this post at SEObook.com:

“if you are not an AdWords advertiser, are not in universal search verticals (like news and video), and are not wikipedia, then you don’t have many organic search results that you can rank for on the first page.”

The image makes it clearer:

In some ways, blogs are better placed than ordinary websites, as Google may be indexing your blog as part of its news search. But that isn’t particularly comforting. The wider move towards mainstream results that keep you within Google doesn’t look particularly healthy. 

Here’s what SEObook suggests:

• If your site is fairly close to what it takes to be considered in some of Google’s verticals – like Google news, then consider upping your game a bit and submitting an inclusion request.
• Try to make some video content. Not good for everyone, but most sites could use some, and the competitive bar with video is much lower than it is with text – though I wouldn’t expect it to stay that way for more than a couple years.
• If you have some top rankings that are bouncing around consider focusing on promoting that content again – when stratification occurs you are going to be better off focusing on owning a few ideas rather than being average to slightly above average at many. Top ranked sites also benefit from self-reinforcing rankings. Read up on cumulative advantage if you have not yet done so.
• Usage data (and/or brand searches) may become a big part of future algorithms. Get ready for that by reading about BrowseRank then invest in advertising, branding, and user experience.

The only upside? Google may be making itself less relevant, and more open to competition.

PHP Freelancer

In the same week that the BBC’s head of editorial development for multimedia journalism was quoted as saying they must do better at linking to external sites, it’s been revealed that the corporation is using a convoluted linking mechanism which means those sites will be denied any benefit in their Google ranking.

Pete Clifton is quoted as saying “It’s not about people slavishly coming back to the BBC. This is a real change in our view that we feel much more part of the web.

“We want to say [to the audience] ‘go and look at some alternative views’ and to reflect a wider media community on the website.”

But in a separate story search engine optimisation blog Blogstorm is reporting that those external links, once direct, “are now passing through two redirect scripts using a 302 redirect which is highly unlikely to pass any PageRank.”

Google’s PageRank system ranks webpages based on a number of factors, but most significant is the number of other webpages linking to it – and their PageRank.

If the website linking to your webpage has a high PageRank of its own, that makes the link even more valuable in boosting your webpage’s PageRank.

The BBC’s News website has a PageRank of 9.

Blogstorm lives up to its name by calling the act “an outrageous act of selfishness and greed”. It is more likely to be an outrageous act of stupidity and/or laziness: a clumsy way of monitoring how many links are clicked. It certainly doesn’t send out a particularly positive statement about how it perceives its relationship to the rest of the web, however.

To the BBC’s credit, the tech editor of BBC News, Darren Waters, responds in the comments promising to pass it on.

UPDATE: Malcolm Coles has a good post on other major sites that hold back on Google juice when they link, including YouTube, MySpace, Wikipedia, Spock and most newspaper websites.

More? See my Delicious bookmarks on ‘linking’

PHP Freelancer

Matt Wardman has an interesting post on the Economist having its PageRank cut by Google (translation: Google punishes Economist for unknown transgression by giving its website less importance and therefore, probably, lower ranking).

Here’s what he says:

It seems that they have made a basic mistake of selling text links on the home page, which bear no real relation to the content of the page – and then by leaving simple “links” in place which will enhance the position of those client sites in Google.

Both practices are resolutely opposed by Google (summary), and have been for some time.

Here is a screen shot of the bottom of the Economist home page this lunchtime:

The list of advertisers is areas which are often found trying to gain advantage in the search engines by link buying.

… multiple links to the same site, a mass of keywords, and they are simple links that pass pagerank.

… There seem to be several howlersmistakes:

• Text links being sold to advertising Clients not particularly relevant to the page. Bearing in mind that the practice has been under fire from Google for some considerable time, that was asking for trouble.
• Allowing those links to be visible to search engines. The recommendation is to add a “nofollow” attribute to the links, use Javascript, or to use a redirect.
• To do this with Credit Checking Services and “Cheap Loan” Finance companies seems – on top of the above – to be like running a red-flag up the flagpole for the sake of it.

… The moral? Take great care when you implement adverts and text links: you are in bed with an elephant.

PHP Freelancer

If you read the final part of my model for the 21st century newsroom concerning new media business models, I strongly recommend ‘Free! Why $0.00 Is the Future of Business‘, an article by Chris Anderson, author of The Long Tail. Have you not clicked yet? Here are some quotes to persuade you:

“To follow the money, you have to shift from a basic view of a market as a matching of two parties — buyers and sellers — to a broader sense of an ecosystem with many parties, only some of which exchange cash.

“… There are dozens of ways that media companies make money around free content, from selling information about consumers to brand licensing, “value-added” subscriptions, and direct ecommerce (see wired.com/extras for a complete list). Now an entire ecosystem of Web companies is growing up around the same set of models.”

Anderson maps out a ‘Taxonomy of free’ including

• ‘Freemium’ (what I called the velvet rope model);
• Advertising;
• Cross-subsidies (buy something get something free);
• Zero marginal cost (e.g. giving away music because it costs nothing to produce or distribute)
• Labour exchange
• Gift economy

And he saves his best till last: 

“Two of the main scarcity functions of traditional economics — the marginal costs of manufacturing and distribution — are rushing headlong to zip. It’s as if the restaurant suddenly didn’t have to pay any food or labor costs for that lunch.

“Surely economics has something to say about that?

“It does. The word is externalities, a concept that holds that money is not the only scarcity in the world. Chief among the others are your time and respect, two factors that we’ve always known about but have only recently been able to measure properly. The “attention economy” and “reputation economy” are too fuzzy to merit an academic department, but there’s something real at the heart of both. Thanks to Google, we now have a handy way to convert from reputation (PageRank) to attention (traffic) to money (ads). Anything you can consistently convert to cash is a form of currency itself, and Google plays the role of central banker for these new economies.”

The whole thing is a generous taster for his forthcoming book of the same name coming next year (if Long Tail is any guide, expect copious examples to illustrate the same points).

Meanwhile, there’s a wiki on ‘Make Money Around Free Content’ linked from his article which is worth keeping an eye on.

PHP Freelancer