While it’s one thing to understand interactive storytelling, community management, or the history of online journalism, the changes that are affecting journalism are wider than the industry itself. So although I’ve written previously on essential books about online journalism, I wanted to also compile a list of books which I think are essential for those wanting to gain an understanding of wider dynamics affecting the media industries and, by extension, journalism.

These are books that provide historical context to the hysteria surrounding technologies; that give an insight into the cultural movements changing society; that explore key philosophical issues such as privacy; or that explore the commercial dynamics driving change.

But they’re just my choices – please add your own.

1. The Master Switch – Tim Wu

The best mainstream history of media technologies I’ve certainly read (although Winston’s ‘Media, Technology and Society‘ is very good too, if a more academic read). Wu tells the story of how radio, film, television and other media technologies went through a consistent path from ‘democratised technologies’ to media monopolies.

It’s a salutary tale for those who think the internet is different. If it is, then it will need to avoid the mistakes made by regulators, legislators and inventors. And those who don’t learn from history…

2. The Information – James Gleick

An astonishing masterwork that begins with why African talking drums were so wordy (it’s all about redundancy), takes in genetics, code-breaking and quantum physics, and in the process draw some very useful lessons about the changing nature of communication and information that help you take a step back from our own assumptions.

3. The Pirate’s Dilemma – Matt Mason

This covers the histories that lie behind the rise of mashups, guerilla marketing, and other cultural movements. A valuable lesson on where to look for change, and how that movements themselves change as different groups adopt their ideas. The book is available as a free download at http://thepiratesdilemma.com/, as is Lawrence Lessig’s book exploring similar themes, Free Culture.

4. The Wealth of Networks – Yochai Benkler

Widely recognised as the most comprehensive book on network dynamics. Given that these are so integral to everything that takes place online, that makes this a pretty vital book. And this is not just about online networks: the book draws on research into real world networks and communities and where they succeed and fail – vital foundations for any online project.

5. The Spy in the Coffee Machine – O’Hara & Shadbolt

A compact exploration of privacy in the networked age, and how digital technologies are impacting on that. Particularly useful are the passages that explore different cultures’ attitudes to privacy, and the case studies that help the reader explore the ethical issues raised by recent developments and technological possibilities.

6. Search Engine Society – Alexander Halavais

Another compact book, this explores research around how people use search engines, including some types of behaviour that you would not otherwise think about, such as the importance of re-finding, and different types of search literacy. Useful in understanding how people navigate the virtual world.

7. Creative Disruption – Simon Waldman

Although there are many books exploring the successes of new digital businesses, Simon Waldman’s book attempts something much more difficult: looking at how established businesses have tried to adapt to survive in the midst of great change. The book is very well written and does a particularly good job of explaining the various elements that form the basis of any business’s competitive advantage; how the internet changes those; and methods that have been used to respond. It’s a welcome reminder that, like any business, publishing is not just about content, but advertising, distribution, manufacturing and numerous other factors too.

A good book on the legal or political history would be particularly welcome to add to the list – or just something very good that I’ve never heard of. What books would you add?

Searching is the most popular activity online after email. It is the prism through which we experience a significant proportion of the world’s information – from news and information about our community, through to health information, commerce, and just about anything that has a presence online.

Search Engine Society takes a critical look at search engines, how they work, the techniques used to manipulate them – from gaining better rankings to censorship, and the implications for privacy and democracy.

Chapter one looks at the development and workings of search engines, from the once-essential directories of Yahoo! and the citation-based algorithms of Google that now dominate the search landscape, through to lesser-known players such as social bookmarking service Delicious which relies on user-generated ‘folksonomies’ to organise material, and specialised regional and ‘vertical’ search engines like the French language Voila or the genetic materials search engine The Bioinformatic Harvester. This is situated within a wider discussion of information retrieval histories from the Library of Babylon onwards – and touches on recent moves into geospatial, mobile, social and semantic search.

Balancing that focus on technology, the following chapter focuses on users, looking at how people search. Search behaviours vary widely between users and between searches – Halavais discusses research that showed how many users simply add ‘.com’ to a word as the start of their search, while others use a ‘shopping mall’ approach of going direct to the likes of Wikipedia and the Internet Movie Database (which also contain search facilities). Using a search engine, Halavais argues, is only one method of search, and search is “not only an iterative process, but one that is rarely linear and requires seeking out the concepts that surround a problem or question. In other words, the query and search strategy is likely to change as more information becomes available.”

Search as ‘re-finding’

Halavais also emphasises the importance of ‘re-finding’ – “not as a sub-set of finding, but the other way around” – indeed, this is the basis of social bookmarking services like Delicious and Digg that allow the user to store and label (‘tag’) webpages for later retrieval, as well as searching for webpages that have been given similar tags by other users.

Power law distribution patterns famously recur throughout the web and in the third chapter Halavais looks at how this affects search results. With Google’s rankings relying so strongly on how many links point to a particular page, it is important to look at how those links are distributed. The fact that highly linked pages are likely to attract ever more links – what Huberman calls “preferential attachment” – leads to the “chunky” nature of the web – in concrete terms the dominance of websites like those of the BBC and Guardian; a quality which, Halavais argues, Google’s PageRank technology ‘calcifies’.

But when Google tweaks its search engine algorithms to attempt to improve results, it can have enormous consequences for organisations dependent on their rankings in search results. Halavais uses the example of Skyfacet.com and Answers.com which saw sales and visits drop by 17% and 28% respectively when they dropped off the first page of related Google searches. It is as if someone moved your shop from the main high street to an industrial estate. In this context it is not surprising that search engine advertising accounts for the majority of online advertising spend.

Digital divides

Following up on those issues, the fourth chapter looks at implications for democracy on two sides: firstly, the division between winners and losers in the contest for public attention; and secondly, the division between skilled and unskilled users of search engines. Halavais is keen to highlight that division is nothing new:

“Current search engines, like communication technologies before them, contain both centralizing and diversifying potentials. These potentials affect the stories we tell ourselves as a society; and the way we produce knowledge and wisdom.”

In practice, these potentials are heavily weighted towards US sites:

“In the language of PageRank, US sites simply have more authority: more links leading to them … sites have existed longer in the United States, where much of the early growth of the internet occurred… Add to this the idea that early winners have a continuing advantage in attracting new links and traffic, and US dominance of search seems a foregone conclusion … the search engines do not merely reflect this authority, they help to reproduce it.”

Indeed, ranking systems that reinforce authority, says Halavais, are conservative in nature and comprise what Lewis Mumford, writing 40 years ago, called “authoritarian technics”.  But because of the unlimited size and reach of the internet compared to previous media technologies, it is not so simple:

“The current structure is a complex combination of a high degree of centralization at the macro-level, with a broad set of diverse divisions at the micro-level.”

Blogger as ‘search intellectual’

Interestingly, at this point Halavais introduces the blogger as a “search intellectual”, upsetting existing structures of authority on the web and acting as “a counterweight to the hegemonic culture of the search engines” in bringing otherwise overlooked material into the “circle of reputation and links that search engines tend to enforece”. The recent rise of Twitter in performing a similar role would be worth adding to that list.

Chapter 5 takes a broad look at censorship – “just another word for filtering” – while Chapter 6 looks at privacy – search engines as “databases of intentions” where even anonymised logs of what individuals are searching for can lead to people being identified. Chapter 7 revisits the rise of “sociable search” tools and folksonomy – where classification is created by a mass of users’ ‘tags’ rather than any centralised scheme, and ‘finding’ is a social act closely related to ‘sharing’.

The book closes with a roundup of the possibilities of future search and the factors that will influence that, from increasing digitisation of material to improved mapping and the possibilities of RFID tags (which makes objects a part of the web too). Semantic search – technology that understands the meaning of what you are searching for, or of relationships between objects – is the promise that lies forever ‘just over the horizon’, while sociable search offers a more likely immediate move.

As is natural, there are areas which have developed since this book was written and so are not tackled in depth – most notably real-time search. The rise of Twitter and the ability to search through what people are talking about ‘right now’ represents such serious competition to Google that it introduced the first major new features to its homepage in years. Wolfram Alpha – the “computational knowledge engine” that made newspaper front pages this year – is not even mentioned.

But those are incidental issues in what is an important book. Halavais manages to acknowledge the dominance of Google without being distracted by it, and gives due attention to non-Western tools and services not commonly seen as search tools. He avoids the pitfalls of technological determinism and manages to distinguish between top-down domination and bottom-up diversity. What emerges is a sophisticated picture of power in flux. “Search engines are interesting to the person who wants to understand the exercise of power in the information society,” Halavais writes in the his conclusion. “In an era in which knowledge is the only bankable commodity, search engines own the exchange floor.” The more readers understand this exchange floor, the better we can exchange and interrogate what information we possess.

A shorter version of this review will appear in Journalism