Abstract:

The first free news search engines appeared in the mid 1990s. Among today’s more important news search engines, Yahoo! began developing its news website in 1996. Google News, Topix and Bing News have all made their appearance since 11 September 2001. This chapter highlights the free news search tools available on the internet today. The major free search engines offer advanced search features, a range of display and presentation options and clustering of stories from different sources. Free news searching is also available from a range of news sources, among them the major newspapers and their archives, news broadcasters, news agencies, and press and scientific newswires. Historical material is accessible through newspaper digitization projects.

Key words

news search

news search engines

news sources

Introduction

It’s now nearly 15 years since the first freely available news search engines appeared on the web. This chapter starts by tracing the history and development of these news search tools. Several of the currently available free news search engines enable you to search a variety of news sources offering free content. The majority of these sources make their stories available for only a limited time, while some others have a longer back archive on the web. There are also news databases and search services that will let you search for free but charge for viewing the full text. In a sense, these are free news search engines too. In this chapter we describe some of the most important tools currently available, such as Google News, Yahoo! News, Topix and Bing News. We also look at the different types of news-related sources, including newspapers with web archives, broadcast news, news agencies, press release wires and newspaper digitization projects.

A short history of free news search

The first free news search tools appeared on the internet in 1996 (Figure 2.1). Already in June 1995, InfoSeek had started providing a service called a personal newspaper for its subscribers. At that time you had to pay to use InfoSeek, which was then a combined web directory and database of articles from computer science periodicals. However, its Net Search, which was a search engine indexing the web pages that its crawler program found on the internet, was free of charge. Its News Search was a part of the services that were later offered for free and became an early favourite among the free news search services.

The personal newswires that InfoSeek introduced in 1995 were created by allowing users to automatically run search queries at certain intervals on indexes consisting of web pages, Usenet newsgroup messages and commercial sources. The later free InfoSeek News Search searched the contents of news wires from news agencies such as Reuters and press release wires like Business Wire and PR Newswire. In addition, you could also search national news sources on the web, such as USA Today, CNN and ABC News. InfoSeek was acquired by Disney in 1999 and was subsequently terminated in 2001.

In 1996 at least two interesting news search sites were launched that didn’t belong to any major company: Newsindex.com, in April, and Totalnews.com, in October. Newsindex.com boldly described the motivation for providing its service as a mission to let users find the truth. It argued that no single media outlet could do that, and that only by being able to access several sources could the reader get behind the headlines and find out what was really happening. It also stated that it wanted to do more than merely search the wires and the big media outlets and added many other websites with quality news content to its service.

Newsindex.com provided a current awareness tool as well as a facility to search its database directly. However, searching was limited to stories that were up to a maximum of one week old. Totalnews.com, on the other hand, provided an archive that sometimes enabled you to retrieve articles that were as much as a year old. The Times (of London) accused Newsindex.com of copyright infringement – something that would be repeated many times in the future with other news search engines and other news companies and organizations. In some countries news search engines have been very controversial and have come under attack from writers’ organizations and copyright holders, while in others it is felt that the news search engines drive traffic to the source sites, and this is seen as being very good for the source sites with the original content.

The most popular website for searching the web at this time was Yahoo! and it continued to be so for many years. Starting from 1996, it added several other services to complement its immensely popular web directory. Its Yahoo! Daily News section initially offered content mainly from Reuters. During succeeding years it added more sources, including wires like AP, Business Wire and PR Newswire, and also some news websites. In 1998 it began to include content from international wires like AFP and Deutsche Presse-Agentur (DPA). It remained a limited wire-oriented search engine, but this changed when, through a chain of company acquisitions, Yahoo! gained possession of the powerful Norwegian search engine AlltheWeb, created by FAST Search & Transfer.

Another popular early search engine was Excite, which was created by students at Stanford University, just as Yahoo! had been. The burrito-loving youths who started Excite launched their NewsTracker in February 1997 (Figure 2.2), in their press release calling it the first free news clipping service on the web. They initially indexed sources such as the New York Times, Fortune, Sports Illustrated and other news media sites. News wires were added later to NewsTracker and it was possible to search both types of sources simultaneously, even if the results were displayed separately. Excite didn’t manage to survive the turbulent years and, like InfoSeek, it was sold and its search technology abandoned in 2001. The Excite@Home portal, however, continued to exist with changed content.

Many users came to embrace the newer Northern Light search engine, launched in 1997. This was AltaVista’s (see below) main competitor, in having the largest web search engine index at that time, and introduced many new advanced search features. It created a set of specialized search engines called Special Editions that were much appreciated, and the concept was a novelty at the time. Its Current News Search was one such topical search engine, and became available in September 1998. It claimed that its index of news articles was updated every 15 minutes. It covered news wires only, but a broad selection of these could be searched simultaneously. Northern Light was bought by a company called Divine, which subsequently went bankrupt, and it disappeared from the free web in 2003. Northern Light Current News Search was one of the few news search engines available in September 2001.

One of the most important news search companies to emerge during the late 1990s was San Francisco-based Moreover. It provided a free news search engine and later also started licensing its database to companies like AltaVista and Inktomi. The latter delivered search technology for Microsoft’s MSN portal and had its own search engine, available at HotBot. The HotBot News Channel didn’t index wires, but only a selection of the most popular news media sites, and it was known for having the most current index of this kind of news content. In March 2001 Moreover became the supplier of news search for AltaVista. Because of this AltaVista was the most heavily used news search engine in September 2001 (Figure 2.3), while the leading search engine, Google, had nothing to offer in this way. Moreover has continued to provide news search for many major clients and today offers a broad selection of topical RSS news feeds for free.

One of the very best English-language news search engines outside of the US was Rocketnews, based in Ottawa, Canada. Although it became available late in 2000 it was formally launched in October 2001, shortly after the terrorist attacks in the US. Rocketnews became known early on for its excellent news search, and most particularly for having a very up-to-date index. In fact it offered a search engine for articles that were five days old at the most. It introduced a free RSS reader in March 2004, later the same year became one of the first to support search results delivered as RSS feeds, and shortly after that started offering blog search as well. It also added another feature – tracking your previous searches and building on these to enhance search results. In this busy year it became the first to offer a news search API and the following year, in 2005, it introduced its powerful desktop news search tool. In 2006 it expanded its search to audio and video news, and even podcasts.

In the aftermath of the events of September 2001, both Google and...