The Internets leading search engines and a major Weblog tool vendor are joining together to fight the onslaught of spam in the comment sections of blogs.
Google Inc., Microsoft Corp.s MSN division, Yahoo Inc. and Six Apart Ltd. announced late Tuesday that they are supporting a tag called “nofollow” to exclude links in blog comments from search-engine crawlers and to prevent spam posts from influencing search rankings.
Comment spam occurs when spammers attempt to dump Web site links into the comment sections of blogs. The spammers often use automated bots with the goal of trying to game search engines by increasing their link popularity. Incoming and outgoing links are a major factor in determining where a site ranks in search results.
The “nofollow” tag is already a part of HTML, but Google suggested using it as a way for blogs to tell search-engine spiders to ignore hyperlinks appearing within comment sections, trackbacks and referrer lists, said Anil Dash, vice president of Six Aparts professional network.
“Theres an economy around link spam because of the way the search engines work right now,” Dash said. “Until we address the underlying economics of it, theres not going to be a solution.”
Adoption of the tag by blog-publishing tools, and the support of the three largest search engines, should help take away the major incentive for spammers, since their links no longer would impact their search rankings, Dash said.
Google, in a blog post, said its search crawler has begun recognizing the tag.
“From now on, when Google sees the attribute (rel=”nofollow”) on hyperlinks, those links wont get any credit when we rank Web sites in our search results,” the posting said. “This isnt a negative vote for the site where the comment was posted; its just a way to make sure that spammers get no benefit from abusing public areas like blog comments, trackbacks and referrer lists.”
A Yahoo spokeswoman also said Yahoo Search will support the technique within its search index in “the coming weeks.”
MSN will begin recognizing the tag by the time it switches to its own search-engine technology later this year, said Justin Osmer, an MSN product manager. It also may add the support in upcoming builds of its engine in beta testing.
MSN is in the process of developing search technology to replace its use of Yahoo Inc.s results on its main search site.
As for the blog tools, Six Apart is rolling out “nofollow” support across its 6.5 million users. It released a plug-in for its Movable Type software that allows users to turn on the technique for their comment sections and trackbacks, Dash said. Its TypePad service also has begun supporting it, and its recently acquired LiveJournal service will implement “nofollow” within the next few days.
MSN Spaces, Microsofts recently launched blog service, also will support publishing of the tag, though the timing has yet to be determined, Osmer said.
“This is a particular case where the competitive barriers come down, and we join hands to combat this together,” Osmer said “[Comment spam] hurts all of us, and it hurts our customers and our consumers.”
Other backers include the Google-owned Blogger, WordPress, blojsom and Blosxom blog tools, and the Flickr and Buzznet photo-sharing services, according to Googles announcement. No details were provided on how those tools would support the effort.
Since the weekend, rumors of Googles support for the tag have circulated among leading blogs. The speculation started after Dave Winer, who publishes the Scripting News blog, wrote a vague post Friday about an upcoming announcement for a solution to “a problem on the Internet” and linked to a page from Google.
Winer said earlier Tuesday that cooperation among the leading search engines is needed to put a dent in comment spam.
“In general, fighting comment spam is something that the search engines are uniquely able to do something about,” he said in an interview with eWEEK.com. “All the defenses that the tool vendors do add, every one of them eventually fails because comment spammers want the PageRank.”
PageRank is Googles technology for analyzing links and other factors to determine a sites position in search results. All of the major search engines also consider links in determining rankings.
Last month, comment spam reached levels where it was knocking out Web servers at Web hosting companies that serve bloggers. Part of the rise was attributed to a bug in Six Aparts Movable Type software, which the company fixed.
But bloggers in general have noted an increase in comment spam, often forcing them to disable comments, use authentication services or review comments before they are posted.
About two weeks ago, Six Apart issued a 25-page guide to the comment-spam problem, Dash said. That opened a broader dialogue among search-engine and blog-tool companies, which led to the publishing of the “nofollow” attribute.
Beyond blogs, other sections of Web pages with contributed links, such as bulletin boards and guest books, could use the technique to nip spammers, Dash said.
Editors Note: This story was updated to include more information from an interview with Six Apart and an announcement from Google.