Yahoo Open-Sources Traffic Server, Updates Hadoop

Yahoo has taken "its second major step in five months toward open-source cloud computing ... [by introducing] an open-source version of Traffic Server, a high-performance application server for builders of cloud services," the company said in a news release Nov. 2. "Traffic Server enables the session management, authentication, configuration management, load balancing and routing for an entire cloud computing stack."

Yahoo has moved to open-source its Traffic Server technology through the Apache Software Foundation.

Yahoo Traffic Server "follows the Yahoo Distribution of Hadoop as another example of Yahoo's ... commitment to open-source cloud computing initiatives. Yahoo has donated the Traffic Server code to The Apache Software Foundation through the Apache Incubator, and intends to build a robust community of developers around the open-source Traffic Server. Shelton Shugar, senior vice president of Cloud Computing at Yahoo, will be discussing the new technology [Nov. 3] at the Cloud Computing Expo" in Santa Clara, Calif., the company said.

"We see Traffic Server as an essential building block for cloud computing, and at Yahoo, it's integral to our edge services, online storage and cloud serving. The open-sourcing of Traffic Server is representative of our companywide commitment to sharing technology innovation with the open-source community, as well as our broader intention to continue to open-source our cloud technologies as they mature," Shugar said in the statement. "By releasing an open-source version of Traffic Server, we are sharing a core piece of technology with the open-source world, while also signaling our intention to build a community of developers to take it to the next level."

Chuck Neerdaels, vice president of storage and edge services at Yahoo, said the Traffic Server technology comes from Yahoo's acquisition of Inktomi in 2002. "And for two years it's been going through performance optimization, so it's a fairly scalable service manager," he said.

The Yahoo news release continued, "With the open-source version of Traffic Server, organizations can benefit from fast, reliable and scalable access to cached online content. In addition, Traffic Server enables speeded responses to requests for stored Web objects, such as files, news articles or images, reducing bandwidth usage and costs.

"The low-latency, extensible framework of Traffic Server makes it ideal for delivering Web traffic at high rates, and its 'plug-in' architecture makes it customizable to fit different system needs."

At Yahoo's recent Open Hack Day in New York Oct. 9, Sam Pullara, chief technologist at Yahoo, said the company would soon be open-sourcing Traffic Server.

"Yahoo's release of Traffic Server represents more than eight years of active use and quality engineering in a product that currently serves more than 30 billion Web objects a day across the Yahoo network. The company's global network of data centers allows Traffic Server to choose the closest servers to store and access cached content for increased speed. Traffic Server is widely deployed at Yahoo, capable of handling more than 30,000 requests per second per server and it currently serves more than 400TB of data per day," the Yahoo statement said.

In addition, the release said, "Yahoo is also announcing an update to the Yahoo Distribution of Hadoop, now deployed extensively in Yahoo data centers worldwide. Since the initial Yahoo Distribution of Hadoop was announced in June 2009, Yahoo has published multiple updates to the code. These include new features and bug fixes that continue to improve robustness, security, performance and operability of Hadoop for ongoing large-scale deployments."