Search for Advantage

Looking back at the evolution of the Internet, its development looks a lot like the history of New Yorks Manhattan Island. Both started as systems of addresses: a grid of streets and avenues in one case, a branching tree of numbered IP net and subnet octets in the other. Each of those open address systems then got an overlay of named locations: subway stops in the first case, DNS (Domain Name System) directories in the other.

In their current state, though, each of those rich and diverse spaces has wound up with only a fraction of its resources being well-displayed to anyone but a native. The out-of-towner who visits New York consults a tour guide; the casual Internet user sees only those things that Google puts on the first page of hits from a search. In both cases, unseen forces limit the options that most users will ever even know that they have, let alone those that theyll actually use.

/zimages/5/28571.gifGoogle settled a class-action lawsuit alleging that it conspired with competitors to inflate advertising rates.Click hereto read more.

People are a lot more savvy about vested interests in the real world than they are about the biased advice they get in cyberspace. If a cab driver urges you to try a particular restaurant, youre likely to suspect that the driver is getting a bit of a finders fee to steer customers there.

The operations of a search engine, driven by something as intuitively reasonable but as subject to manipulation as Googles PageRank algorithm, are just as prone to a bias that serves someone else as well as the searcher—but the nature and mechanism of that bias are much less obvious to most users.

Google the exact phrase “search engine optimization,” and youll get nearly 65 million hits. The results arent talking about the theoretical possibility or the ethical repugnance of deliberately warping the fabric of cyberspace to make you fall into the desired fold. Theyre talking about the effective ways and means of doing it.

Im not suggesting that people should boycott Google. Quite the reverse: When our regional symphony orchestra board meeting was brainstorming public relations techniques, I suggested that the board stop putting its cumbersome URL on posters and advertisements and replace it with an exhortation to “Google Beach Cities Symphony!” Its easier to remember, and it speaks the native language of a demographic group that the orchestra would like to do more to engage.

The furor during May, though, concerning side effects of Googles algorithm changes—the so-called Big Daddy technology overhaul in January—should be a warning to anyone who depends on a Web presence or relies on speedy and well-informed access to Web resources. The completeness and consistency of the index and the search results, and even the minimal accuracy of operations such as searching for a specific phrase instead of the mere combined presence of certain words, are evidently at risk.

/zimages/5/28571.gifIs Big Daddy choking Google?Click hereto read more.

In the arms race between the search engines and the trash sites, my moneys on the trashers. You need scarce resources of people and capital to devise better algorithms and to execute them with useful speed against exploding numbers of sites with escalating complexity of content. Trash sites cost nearly nothing to set up, using tools and techniques that are widely shared.

At some point, therefore, the short, happy life of the free search engine thats actually useful to a professional researcher seems certain to come to an end. In the same way that academics and other professionals rely on peer-reviewed journals, not on tabloid newspapers, the enterprise professional will rediscover the value of the informed meta-source who maintains a good Rolodex of top-tier primary sources in any given field of interest.

Google is trying to ensure that its brand-name equity carries over into this evolving multitier market of search services—for example, through the Google Co-op unveiled in May that invites the creation of expert-filtered search result sets.

Individual enterprises, however, also would do well to tap into the growing recognition of the limits of algorithmic search—and to make their own sites the authoritative hubs and sources of first resort for information of interest to their customers and partners.

Technology Editor Peter Coffee can be reached at peter_coffee@ziffdavis.com.

/zimages/5/28571.gifCheck out eWEEK.coms for the latest news, views and analysis on enterprise search technology.