Google: Cloaking Hurts Your Page Rank

 
 
By Clint Boulton  |  Posted 2007-12-14
 
 
 

MOUNTAIN VIEW, Calif.—Google co-founders Larry Page and Sergey Brin get all the fame and glory for launching their search engine nine years ago. But these aren't the guys you generally associate with educating the public about Google's search and various other products.

For that you have to turn to Marissa Mayer, vice president of search products and user experience for the company. Mayer started at Google in 1999, designing the search interface among other products, and her rise has been nothing if not meteoric.

Mayer's position has evolved to a blend of development and product management; she now has several initiatives under her purview, including Google News, Health, Toolbar, Labs and Earth, but she doesn't just sit at a desk programming or hit the road for customer tours.

It was Mayer who went on the Today Show Dec. 3 to announce the top 10 searches and then held a conference call detailing those keywords along with a tutorial on Google's Trends application the next day.

So, not only does Mayer work behind the scenes designing Google's search engine, but she is widely considered the face of the world's largest search engine. Mayer explained how she bridges the gap between designing Google search and presents it to users in an interview with eWEEK at the company's Mountain View, Calif., headquarters Dec. 12.

The PageRank issue is one of the biggest points of contention in the blogosphere now. Businesses making money from Google's ad program are complaining that changes to Google's search algorithm are affecting how their site is perceived to be valued, which in turn makes it more expensive for them to advertise. What do you want to say to these people by way of explanation?

Click here to read more about Google's Dataspaces technology.

People use search to find something online. If you're a small business owner, you need to use search effectively to get leads for your business, and we understand and respect that. That said, we really want to provide our users with the most relevant experience. So, cases where we see people's overall PageRank drop include when people provide some trickery to the search engine, where they'll realize it's the Google bot that's visiting their page and they hand us a different page than they would hand the user.

When we see things like that happening, it's basically no-holds barred. We have a page, we think we know what that page is about, we're going to index that and return those search results based on our knowledge, and that's totally different than what the user sees. It could be completely disjoined and that can really hurt user experience. Users will click on a link and say, how is this relevant to my search? So, when we see things like cloaking happen, then we'll treat it as a blank sky. We have no idea what's on that URL. They wouldn't tell us what's on that URL, so how are we supposed to put it in our search results? We still don't take it out, we just heavily downweigh it because we feel like we have no idea what the user is actually seeing.

When you say "we" do you mean the search algorithm that is detecting these anomalies and taking action, or does the algorithm detect cloaking and other trickery, report back to Google engineers who take action, influencing PageRank?

No, it's the algorithm.

Next page: Q&A with Google's Marissa Mayer

Some people assume that engineers sit here at HQ, find evidence of people gaming the system and take action.

It's unfortunate because one of the greatest Internet success stories was Yahoo, and Yahoo had their army of surfers who would surf around and categorize things into the directory. People read the stories about the Yahoo surfers and how they did their work, and, as a result, people think their search result pages are hand constructed. So they wonder: Who's in charge of the search result page for MP3? Obviously, Webmasters know better than this, but the general population actually believes that we have people here who are handwriting these pages and deciding results should move from result No. 4 to result No. 7, which we clearly don't.

We really value that our result pages are mathematically objective, and if we have a human there judging, that's a problem. Cloaking is one of the signals we look for. Wherever we send out a Google bot, we also send a regular user agent out, and when we do a comparison, if more than just the ads have changed, we know we're dealing with a site that's cloaking, and once that signal is set to yes, most of your scores get discounted proportionately. It's like a polynomial if-that.

What's top of mind for you? Which of the areas you oversee occupies most of your time these days?

I tend to go in little bursts. Yes, I'm managing the scale of my organization and building processes that make it easy for my employees to run fast and experiment and make the right changes. But at the same time, there are products that pop up that need more of my attention. For example, iGoogle, because it's growing so fast.

What do customers tell you they want to see improved with iGoogle?

They want to be able to see larger views of their Gadgets (Google's widgets) without leaving the page, which makes perfect sense because a lot of those little postage-stamp views can be small at times. They really are responding well to themes, which is our word for skins. They would like to see more themes and more of an ability to personalize them. So we're looking at how to roll up more themes and make it possible for people to participate and help us develop the themes.

To read more about Google adding blogs to Universal Search, click here.

And, users don't ask for it, but I think there is a big viral effect that we're missing, which is, once I add a piece of content to my home page and I like it, how do I share it with my friends? For example, we have something called the Gadget Maker, which makes it easy for people who don't know how to program to create a Gadget. Once I create that Gadget, how do I e-mail that?

You tipped off the public on the call about Google Trends that you could foresee opening up the APIs for that application, which would allow programmers to embed this tool in their applications. Any other APIs on the horizon that Google will share?

We really like being open. Our AJAX [Asynchronous JavaScript and XML] Search APIs have been really popular. Our Maps API has been popular; Gadget APIs has been popular. Now what we're looking at is how we can draw up a common theme and language across all of those APIs, so if you use one of the Google APIs, it's really easy to go and learn another one. Right now, they're being developed by different developers at different times, and they feel that way, so we're looking at how to unify those to make it easy for developers once they learn the basic framework of how Google APIs work.

You've come a long way at Google since starting as a programmer. What's the key to managing the intersection between the programming and consumer-facing worlds?

That's why we hire product managers that are computer scientists because we really think that to expose the best possible interface you need to understand the technical underpinnings, simplify them to death and make them very easy for users to use. But sometimes you make trade-offs. Should we release this product without this functionality? Will it matter? How will it work and look?

Check out eWEEK.com's Search Center for the latest news, views and analysis on enterprise search technology.

Rocket Fuel