Surviving a Web Pass Rush

IT team scores big during Super Bowl Crunch

Weeks before Super Bowl XXXV, as John Stiening and his team huddled to plan strategy, they made a cold-sweat-inducing discovery: They would get sacked.

Stiening and his engineers at eCreative Search Inc. weren?t diagramming Xs and Os—rather, they were trying to get a grip on capacity planning for the explosion of site visitors the big day would bring.

What they found was that the marketing department had more than doubled the amount of free passwords dished out via e-mail to potential visitors to ECS Web site, which runs an ad-industry search engine. They also discovered a calculation error in firewall configuration that meant they had half the capacity they thought they had. To top it off, the marketing department planned an all-out blitz to drive traffic to the site.

The fear of getting overwhelmed by hits had Stiening scrambling for help from the sidelines. His salvation? Handing off capacity woes and content delivery worries to one of the new breed of content delivery network services. By teaming up with Akamai Technologies Inc., of Cambridge, Mass., a mere three days before kickoff, ECS was able to weather the biggest spike in traffic in the companys history, as traffic on the Monday after the game shot up to 2 million hits.

While competing sites crashed on Super Bowl Sunday under the crush of traffic, ECS site "performed better than it ever has," said Stiening, who, until early this month, was director of technology at the Chicago company.

A growing number of online companies are finding it worthwhile to take advantage of content delivery network services. According to Joel Yaffe, an analyst at Giga Information Group Inc., in Cambridge, Mass., this market is expected to grow to between $150 million and $200 million this year, up from $120 million in 2000.

Content delivery network providers cache a portion of their clients content on servers in a widely dispersed network, thus offloading traffic from a clients own central servers and providing a better experience to the Web surfer, whose request for information typically takes fewer hops to reach the desired content.

Content delivery network providers also use complex algorithms to monitor traffic and direct requests along the least-congested routes. By dispersing content around the edge of the network—in spots closer to the end user—access times are reduced, and uptime is dramatically increased.

The cost to clients for the service is typically based on traffic and usage, plus a nominal setup fee that starts at about $2,000 with Akamai.

ECS needed this kind of help to capitalize on Super Bowl mania. During the Super Bowl, the ad industry kicks into high gear, debuting companies slickest ads before an estimated audience of 43 million households. The football finale is a make-or-break event for companies like ECS, and the companys marketing department planned to use it to promote ECS site and services, which include tools to locate the talent associated with any ad spot that airs.

Marketers sent out more than 23,000 e-mail messages to agency executives and production companies, providing each with a one-day-only password that would allow full access to the site. Agency execs would use their password to review videos of Super Bowl spots, the thinking went, and theyd be so impressed, theyd sign up for membership in ECS.

For a relatively small enterprise with finite bandwidth and server capacity—a normal day sees the site get about 70,000 hits—the 2,000 percent traffic spike that the Super Bowl wound up causing could have spelled disaster. If the new visitors didnt have a good experience at the site, the plan could turn into an expensive failure.

The first sign of trouble was when the tech group found that marketing had sent out far more passwords than originally planned. The situation grew more dire when the group found funky firewall configurations. "We realized [the firewall] would only support half as many connections as we thought, and [we] had only 100MB of capacity, when we believed wed have 200MB," Stiening said.

Less than a week before the kickoff in Tampa, Fla., Stiening approached Akamai, which hed previously spoken with about a long-term solution to ECS capacity challenges, and asked for help. Akamai technicians deployed their EdgeSuite service for ECS within two days. Technicians made changes to the sites Domain Names Systems, redirecting visitors requests to the closest of Akamais 8,000 servers. Basically, all media traffic was pointed back to an Akamai server, which checked its cache. Only if the Akamai server lacked the correct content did it need to pull it from the main ECS media server. That allowed Akamai to act as a shock absorber between requesting users and the companys primary Web server and eliminated congestion between the primary Web server and the Internet.

Stiening said the best proof of the new setup is that "our sales have been fantastic. ... We were able to keep our site up during the most strenuous of use. We were able to concentrate on our new site and our new collaboration tool. The fact that the site performed better than ever during those spikes was icing on the cake."