Who Should Decide the Relevance of Search Engine Results?

NEWS ANALYSIS: Who is most qualified to determine what Google Search results are most relevant? Should it be users, governments or even Google's competitors?

Google Search Relevance 2

Europe last week officially filed formal charges against Google for violating European antitrust laws. Specifically, regulators say that when European users search for products, Google favors its own Google Shopping service over competing sites and services.

America's own Federal Trade Commission (FTC) made similar accusations before concluding nebulously that Google did favor its own services, but consumer harm could not be proved.

I've been interviewing search experts and journalists on the question as well as talking to Internet users. Responses to these charges fall into three general groupings:

1. The charges are false. Google results objectively rank shopping services according to their relevance.

2. The charges are true. Google favors its own shopping services.

3. It doesn't matter if the charges are true or false. Google should be able to promote its own services higher than competitors because it's Google's search engine.

The most interesting version of response 2 came from my recent interview with Amir Efrati, senior reporter with The Information online news site. His opinion is that search engine rankings are all about relevance. Relevance is inherently subjective—an opinion. And that it's Google's opinion about relevance that is Google's product—it's the benefit you get from using Google Search.

It's an interesting point, especially in light of the fact that (as Efrati wrote in a recent column) the European case against Google is about antitrust, but unlike most antitrust cases it's not about harm to the consumers. As the FTC concluded, no evidence can be found that users are being harmed by Google's rankings.

Europe's five-year investigation into Google's "fairness" in placing search results in the order that it does concluded with Europe officially disagreeing with Google's opinion about which links are most relevant given a specific set of searches.

Once you deeply explore Europe's case against Google—or the FTC's case, for that matter—an elephant always shows up in the living room, which few acknowledge: The issue isn't about results that are fair or unfair, relevant or irrelevant, biased or unbiased.

The issue is: Who should decide what the results are when somebody searches Google?

Who decides now?

The average person on the street might believe that Google currently determines the rankings that its search engine produces. But Google actually is the third most influential actor here.

Most results are determined by the actions of Website owners and by Google Search users.

For example, when I enter the term "cupcake" into Google Search, the second result is the Wikipedia entry for "cupcake." On Bing, it's the first result.

The Wikipedia entry ranks high because any good search engine would develop an algorithm for a highly relevant, universally applicable resource like Wikipedia that is also visited by a huge number of users worldwide.

The high ranking of Wikipedia is determined almost entirely by Wikipedia and by users. The search algorithms simply facilitate the influence of Wikipedia and users on rankings.

Personalization is another way for users to influence results. The No. 1 Google Search result for the word "cupcake" for me is a local chain called Kara's Cupcakes. Because I live in Petaluma, Calif., and have demonstrated through my activity on Google Search and other sites my interest in organic foods, Google is showing me this particular result as the top one.

The person most responsible for that result is me.

Most of Google's algorithm kung fu exists to remove Google from direct involvement in determining what the search results are—most, but not all.

Successful sites that rank or filter things algorithmically from Google to Facebook to dating sites must and do tests obsessively using real users. Google, of course, does such testing and tweaks its algorithms, as the company claims, to provide the best user experience.

Perceived relevance is one aspect of user experience. But there are others.

Shopping results may go up or down the rankings depending on factors other than content or even popularity. The range of information provided or even the formatting of that information may result in what Google deems a bad user experience, so they might be "penalized" with lower ranking.

I'll pause here to remind you of what you might already know: Google's relentless improvement of the user experience is the only reason why Europeans voluntarily choose Google for more than 90 percent of their search queries.