eWEEK content and product recommendations are editorially independent. We may make money when you click on links to our partners. Learn More.
Web site operators are clamoring to understand what can best be described as an ongoing disturbance in the Google Force.
Googles search engine, once a clean, lean indexing machine, from a Webmasters perspective has been slipping badly lately.
Starting about two months ago, site operators have complained that their Web sites have suddenly disappeared from Googles index for no reason, tantamount to disappearing from the Internet.
Another common gripe is how a Web link mysteriously drops from a prominent place in search results to Page 43 or so. The list of gripes goes on.
Something is happening, although Google hasnt said what it is yet and likely wont on its annual Press Day, which is scheduled for May 10. Google usually makes a number of impressive announcements at this event. This year, theres some “product demos” in the offing, it promises.
But in the backdrop of all the shiny new Google stuff is the 800-pound gorilla of a question: Whats going on? Many people suggest that Big Daddys to blame.
On May 10, Google co-founder Larry Page addressed the issue, saying the problems took the company by surprise and it was now investigating what is going on. “We have a team studying it now,” he told an audience of journalists at Googles annual Press Day event.
Big Daddy is, in effect, a brand-new data center that Google uses to perform core search engine tasks like cataloging Web sites or serving up localized features. Google operates thousands of data centers across the globe.
The Big Daddy project stands out because search engines infrequently upgrade the computing and networking hardware. Instead, the focus is on new features to lure more site visitors, which translates to more search engine revenues.
By upgrading, Google can better compete against rivals Yahoo, the worlds leading Internet destination but second to Google in search, and Microsoft, whose online MSN unit manages the worlds No. 3 search engine.
The ultimate goal is a bigger slice of the $13.8 billion expected to be spent on Internet advertising in 2006, and even more in years going forward.
Every search engine is constantly tinkering, or buying new companies and absorbing their intellectual property. Its all to find new ways of attracting an ever bigger audience, which translates into higher advertising revenues.
But as all the Big Daddy disruptions seem to indicate, theres always growing pains as companies sharpen a competitive edge.
As screen-name JohnW writes in a SearchEngineWatch forum: Google “rolled out some new technology to fix some things, and in the process they seem to have broken some other things. There isnt much to say about it other than its broken—it is what it is and will be over when its over.”
Because of Big Daddy, the theory goes, Google must completely erase the databases where it keeps its 8 billion-plus Web pages. Then it will add Big Daddy and kick-start the indexing.
Next Page: Big Daddy comes knocking.
Big Daddy Knocks
That explains why sites disappear, for instance, a few days or weeks at a time. Its Big Daddy knocking, say several posters at this search industry forum (password and username are required).
Another theory about the disruptions is based on comments about a Google “server crisis” from Google CEO Eric Schmidt to the New York Times. From those words comes the idea Googles plumb out of data storage. That too explains the indexing issues.
Theres lots of speculation, and some quasi-confirmation from a prominent Google engineer, but Google didnt immediately respond to an e-mail seeking comment on the topic.
Much of whats known about Big Daddy comes from Google engineer Matt Cutts, who occasionally writes about the topic on his blog. A Google spokeswoman referred a reporter to Cutts blog in response to an inquiry about the recent problems.
At the Cutts blog, theres an oft-cited March 27 Cutts-agram sent at the height of the initial wave of complaints. In it, Cutts claims Big Daddys been fully deployed and he mentions how it will “stabilize.”
He also warns that sites finding themselves off Googles reservation can expect it to take just a few days to show up in the index again.
Meanwhile, at Press Day, the worst-kept secret has been about the debut of Google Health, one of dozens of Google-operated search engines focused on narrower topics.
A Google spokesperson confirmed that the company is testing the feature.
“We have been doing a variety of research in the health area, including how to improve the quality of health-related search results,” the spokesperson wrote in an e-mail. “We have nothing new to announce at this time.”
Theres also suspicion Google will unveil some more items from its partnership with computer manufacturer Dell Computers, based in Round Rock, Texas.
One dark horse rumor candidate is that Google will unveil a more sophisticated Internet phone service for its Google Talk instant message system. VOIP (voice over IP) is de rigueur for IM systems, yet Google Talk lags far behind competitors.
Microsofts MSN recently unveiled a new version with deeper phone capabilities, and Yahoo, of Sunnyvale, Calif., is also more and more phone-focused with each upgrade, so theres certainly competitive pressures to do so.
Editors Note: This story was updated to include information and comments from Google.