Avaya officials managing the sprawling WorldCup network for FIFA are holding their breath. So far, there have been no serious attacks on the network, but if there are, they are likely to happen at the peak of the tournament.
The Avaya team is taking no chances, since the Federation Internationale de Football Association wont accept even “a couple of minutes of downtime,” according to Doug Gardner, managing director for the Avaya FIFA World Cup Program IP network in Munich.
On the security side that means using automated tools—both commercial and open source—and humans to monitor for security events 24 hours a day.
“We spend a great deal of time monitoring the network in real time. We have people monitoring for security 24 by 7 here and off-site. In Munich, we have two people per shift,” said Tom Porter, chief of Internet security for FIFA and Avaya in Munich.
Automated tools include host intrusion detection, network intrusion detection, “a bevy of log analysis tools, anti-virus and we use honey networks—virtual networks that allow us to see malicious activity in real time,” said Porter.
On the network side, it also means putting a lot of redundancy into the network, which spans 64 venues, 12 stadiums, two FIFA hotel headquarters in Berlin and Frankfort, 11 other FIFA hotels, 11 organizing committees and an airport and railway station.
Each stadium owner, for example, is required to provide two separate, high-speed links into the ATM backbone provided by Deutsche Telekom.
Two separate cables are physically located on either side of the stadium, and the network is connected to two separate telecom points of presence.
“On match days, two hours before the match starts, we put people from the technical teams into the primary technical area where the core backbone switches and core voice switches are. Both of those are duplicated and they are redundant on either side of the stadium,” said Gardner.
With such precautions in place, Avaya was confident enough to guarantee 99.999 percent availability for the converged voice and data network.
Beyond the security risks—the network has been hit in past events by large DoS (denial of service) attacks—the big challenge to maintaining such high availability is the dynamic nature of the network itself.
“There are pieces of it going up and coming down all the time. You cant call time out and have a hack free moment,” said Porter.
“Because the network is so dynamic and the edges of this network are so hard to define, we spend lot of time looking at it in real time.”
The network spans 64 network nodes, between 30,000 to 45,000 devices ranging from phones and computers to printers and PDAs.
Roughly 140,000 users have connected to it at some point to date, and the network thus far has transported at least 7TB of data. With both wired and wireless connections, Avaya expects the network altogether will carry between 15 to 20TB of data.
The critical application it supports is the transmission of information and statistics on whats happening in a match gathered by spotters in the stadium.
The data is transmitted over the Avaya network to the broadcast center in Munich, where it is compiled and sent to commentators as an audio feed or as a ticker at the bottom of their TV screens.
“That is one of the most critical things our network does. Even a couple minutes outage would be totally unacceptable,” said Gardner.
The network also supports the fifaworldcup.com Web site, powered by Yahoo. The site is fed by a large number of Web journalists who post their stories on the matches to the site.
Those journalists access the network in media centers and on the pitch using Gigabit Ethernet links or wireless links.
One thing that has changed from the 2002 World Cup is the wireless connectivity built into professional digital cameras.
Many now have 802.11 wireless built into the camera, and 95 percent of the cameras used are digital. In 2002 the split was 50/50 film versus digital. At the same time, in 2002 wireless connectivity was an experiment. Now its just expected to be there, Gardner said.
With the games in full swing, and the network humming along nicely so far, the network operations team is not sitting back and resting.
To stay vigilant to attacks, the Avaya team is continuing the penetration testing it started before the games started.
“To continue testing the technologies we use to defend the network and keep people who are watching these things alert, we run vulnerability scans on a constant basis on different parts of the network,” said Porter, a self-described white hat hacker.
Thus far the network has seen about 10 minor security-related events, mostly non-malicious.
“Most of what weve seen are people bringing in viral or Trojan-infected laptops, which are scanned and cleaned before they are allowed to connect,” said Porter.
But the real test could be coming. “Our next guess is if someones going to try it, it will be prior to the quarterfinals or prior or during the semifinals,” said Gardner.