RSS Comes with Bandwidth Price Tag

By Matthew Hicks  |  Posted 2004-09-20 Print this article Print

As XML syndication grows in popularity, feed publishers are discovering unintended and unfortunate consequences: hits on bandwidth and scouring for solutions.

Popularity comes with price, as Web publishers of XML syndication feeds are learning the hard way. According to feed publishers, as the use of Really Simple Syndication news feeds grows so too does the bandwidth they consume and the demands they put on Web servers. Some Weblogs and technology Web sites are rethinking the way they publish their RSS feeds as they find that frequent requests from newsreaders, the applications that aggregate feeds, can strain their resources. "Publishers are being caught off guard by how popular and how fast this stuff grows," said Greg Reinacker, president and founder of NewsGator Technologies, of Denver, Colo., a newsreader developer. "Its sort of one of the prices you pay for being able to notify users quickly when new content is available. Theres a bandwidth cost to pay."
The debate over RSS bandwidth impact reemerged in the past two weeks after Microsoft Corp. started tinkering with its Microsoft Developers Network RSS feeds. Initially, the Redmond, Wash., software maker scaled back the content of its feed that had provided the full postings from its 967 bloggers.
The feed went from full text to a limit of 500 characters per posting earlier this month, catching the attention of bloggers and developers, some of whom decried the change. One of Microsofts own technology evangelists, Robert Scoble, wrote in his blog that "RSS is broken." Early reports pointed the blame at bandwidth, but Sara Williams, MSDN product unit manager, said the cause instead was a reevaluation of how to efficiently serve such a large, and ever growing, feed. "All of MSDN is a rounding error compared to all of the downloads on Windows Update," Williams said of the bandwidth. "The increased traffic we get because of RSS is completely negligible." But the increased traffic did raise eyebrows. Williams said the companys Web-hosting operations group had noticed that the file size for the MSDN blog page, being served by the full-text feed, had reached 400KB, a number far outside the typical range for a Web page. Read more here about how Microsofts honcho Bill is paying attention to RSS. That led MSDN to rethink its rollup RSS feed. Along with the main MSDN feed of all blog postings, each blog also has its own feed. Then last week, following calls from developers and bloggers to bring back full text, MSDN reversed course. The aggregated feed again is providing the full blog postings, though MSDN did continue to keep summaries, now up to 1,250 characters, for the feed that appears on the site. In the longer term, though, MSDN is looking to better segment the feed into topical ones so that developers can subscribe to a subset of blog postings on a topic like security or Visual Basic, Williams said. The way RSS is distributed and read also needs to evolve. "There are a bunch of opportunities to be smarter in the way clients ping servers for updates and the way servers cache RSS information as far on the edge as they can," Williams said. "Interesting innovations can happen, but at a high level the technology is further ahead than the tools." There is no single solution to the potential bandwidth bottleneck of RSS, experts say. Some, like Williams, suggest distributing RSS feeds throughout the Internet. Others want newsreaders to be more stringent in limiting the frequency of polling feeds. All seem to point to problems in RSS implementations as the culprit. "The problem is that the aggregators, the most popular ones, let users poll whenever they want, and thats not fair," said Dave Winer, a co-author of the RSS format and publisher of the Scripting News blog. "Content developers need to have a say in that." Click here to read how some marketing companies see RSS as an alternative to e-mail for ads. Most newsreaders are set by default to check every hour for updates, but most also allow users to change the interval to as little as every few minutes or seconds. Some even set lower intervals by default. "It just takes one poorly written newsreader to go out there and every 2 seconds query the page, and suddenly the Web server cant serve the normal page," said Patrick McGovern, director of, one of the Open Source Technology Groups developer Web sites. Next Page: Dealing With the RSS Bandwidth Problem

Matthew Hicks As an online reporter for, Matt Hicks covers the fast-changing developments in Internet technologies. His coverage includes the growing field of Web conferencing software and services. With eight years as a business and technology journalist, Matt has gained insight into the market strategies of IT vendors as well as the needs of enterprise IT managers. He joined Ziff Davis in 1999 as a staff writer for the former Strategies section of eWEEK, where he wrote in-depth features about corporate strategies for e-business and enterprise software. In 2002, he moved to the News department at the magazine as a senior writer specializing in coverage of database software and enterprise networking. Later that year Matt started a yearlong fellowship in Washington, DC, after being awarded an American Political Science Association Congressional Fellowship for Journalist. As a fellow, he spent nine months working on policy issues, including technology policy, in for a Member of the U.S. House of Representatives. He rejoined Ziff Davis in August 2003 as a reporter dedicated to online coverage for Along with Web conferencing, he follows search engines, Web browsers, speech technology and the Internet domain-naming system.

Submit a Comment

Loading Comments...
Manage your Newsletters: Login   Register My Newsletters

Rocket Fuel