It has endured a lot of grief lately due to some well-documented crashes of its national flight plan-filing system, but the Federal Aviation Administration is finally starting to bring its Cold War-era IT systems into the new century.
In the last several months, the FAA has upgraded its legacy internal business systems to a new open-systems server and storage infrastructure supplied by Sun Microsystems and an IP network provided by Cisco Systems. These systems currently handle all the agency’s nonflight-related administrative functions, including the FAA’s human resources information, email, messaging, internal document routing and storage.
However, the word from the systems integrator, longtime government contractor GTSI, is that this new deployment is opening the eyes of key people at the FAA’s IT hierarchy and has so impressed them with its performance and scalability they are now considering bringing critical air traffic systems up-to-date with similar infrastructure by early next year.
The National Airspace Data Interchange Network’s current mainframe-based system, an integral part of the overall NAS (National Air Space) traffic system that processes an average of 1.5 million messages per day, is obsolete and has a history of technical issues. Travel disruptions due to these breakdowns are not out of the ordinary, according to knowledgeable air industry sources.
As a result, industry analysts and a number of former Federal Aviation Administration staff members said they believe there is heightened likelihood of a major air traffic stoppage, as was demonstrated three times this summer by the crash of the system head in Atlanta. They also are concerned about increasing vulnerability to terrorist cyber-attacks.
The most recent example of this happened on Aug. 26, when a corrupt file entered the flight plan system and brought it down for about 90 minutes during a high-traffic period late in the day on the East Coast. This was not an isolated incident, as the FAA’s chief administrator originally had told the media. Similar crashes occurred on Aug. 21 and in June, FAA records show.
People connected with this problem inside and outside the FAA agreed that the system needs to be upgraded as soon as possible. The main issues have been: a) agreement on what kind of system to install for the long term; and, of course, b) how to pay for it.
After more than five years of talk, research, evaluation, planning, testing and quality assurance work, it looks as though a breakthrough finally has been made.
FAA approves extension of upgrade contract
FAA approves extension of upgrade contract
On Sept. 23, the FAA exercised the second option year on its 2006 SAVES (Strategic Sourcing for the Acquisition of Various Equipment and Supplies) contract, which was approved by Congress, to fund the systems upgrade. The IDIQ (indefinite delivery, indefinite quantity) contract will total $63 million after all options are exercised.
To date, the FAA has spent about $23 million of that amount; GTSI is budgeted to spend about $13 million more this year.
The contract was awarded under the Federal Strategic Sourcing Initiative and is based on Office of Management and Budget mandates calling for agencies to consolidate their technology infrastructures. So far, the SAVES program has helped the FAA standardize on technology, source goods and services more efficiently, and effectively monitor IT spending, an FAA spokesperson said.
Sun’s open-source OpenSolaris/ZFS/SunFire server/Thumper storage infrastructure-which features built-in, state-of-the-art virtualization capability-was a key building block on which the FAA IT evaluation group settled. Some of the new software is already being used in the air traffic system; ZFS (Sun’s open-source Zettabyte File System) is being used in the FAA’s air traffic data center.
“The FAA uses a large quantity of Sun Solaris servers in a variety of configurations to support some of our noncritical business applications,” Andy Isaksen, manager of the Communications Infrastructure Engineering Team for NADIN and architect of the original mainframe system, said. “ZFS is being used on at least one service within the Air Traffic Organization Enterprise Data Center.”
Isaksen said, “NADIN, which is responsible for flight plan distribution … is nearing completion of our user migration waterfall. We began our migration to the new NADIN from our legacy system in March 2008 and the transition is scheduled to complete in early 2009. We are approximately 75 percent complete.”
Whatever infrastructure NADIN uses, it is responsible for all flight plan distribution for hundreds of airports, and it provides the gateway between the aviation community and FAA, Isaksen said.
Commercial aircraft of any type cannot take off with having filed a valid flight plan, one that includes destination, estimated flight speed, description of cargo, estimated altitude, weather conditions and other data points.
The FAA augmented its old Phillips DS714 mainframes in 2005 at the FAA data centers in Atlanta, Ga., and Salt Lake City with Stratus FTserver 6400s, which run on Intel Xeon processors. However, the NADIN system, which is compliant with National Institute of Standards and Technology 800-53 security controls and operates on a private network, will keep evolving to the Sun-Cisco implementation.
The custom-built NADIN application is not hardware- and operating system-dependent and can be compiled to run on many server platforms, Isaksen said. This includes Solaris, so the changeover was not a major issue.
System integrators perspective
System integrator’s perspective
“What the FAA is doing is common to what a lot of other [government] agencies are doing: They’re trying to do more standardization across their IT infrastructures,” said Tom Kennedy, vice president of sales at GTSI, the systems integrator selected by the FAA.
One of the main requirements in the GTSI contract is that FAA wanted more control of equipment purchases. GTSI worked with the FAA’s standardization committee, chaired by longtime FAA IT administrator Rick Jordan, to come up with the standards in upgrading the networking and server/storage parts of the systems, facilitate the buys and carry out the implementations, Kennedy said.
“Right now, we’re mostly working on the non-NAS side of the FAA’s IT,” Kennedy said. “What we want to do is show success on this side of the system, and then bring it to the NAS side.”
One of the main challenges GTSI faces is consolidation of storage devices, Kennedy said.
“They made a major investment in virtualization,” Kennedy said. “In their [previous] environment, they had disparate storage devices from multiple vendors, all across the FAA. They’re now upgrading or consolidating them, via the standards. Now they’ve honed in down to one platform.”
And that would be Sun-Cisco.
“SAVES was a pretty high initiative that came out of the [U.S.] CIO’s office. The first year, it seemed like there was some resistance [to the upgrade] internally, as people were getting comfortable with actually having [new] standards,” Kennedy said. “But since our implementation, we’ve done two times the volume [of data transactions] this year versus last year.”
That kind of performance will catch attention of bureaucrats every time.
Security: A Major Factor
Security: A Major Factor
In addition to the new Sun infrastructure, the FAA also has taken measures to tighten security from all access points.
ForeScout Technologies, a network access control and policy management provider for large enterprises, was selected to supply a number of its CounterACT network appliances to the FAA’s SAVES contract with GTSI.
CounterACT was approved as an agency standard by the FAA’s Technology Control Board. FAA networks throughout the United States are now using CounterACT to improve network access.
ForeScout President Gord Boyce said CounterACT combines clientless network access control and malicious threat detection to ensure that connected (and, importantly, connecting) devices are in compliance with network security policies and are free of self-propagating threats.
CounterACT seamlessly integrates into any network environment without requiring costly upgrades or infrastructure changes, Boyce said. It also enables enterprises to tailor enforcement actions to match the level of policy violations, ensuring that user disruption occurs only when it is warranted or required by the IT staff, he added.
“The FAA did a nine-month deep dive to make sure our product met their requirements,” Boyce said. “The meat of their business-side deployment is just now beginning. They expect to roll us out to the rest of their network over the next nine months.”
Not only will CounterACT give the FAA the security to lock down their network, Boyce said, but it also will allow “understanding as to what’s on their network, and the knowledge to know what their network looks like.”
CounterACT can see any device that attempts to obtain an IP address, Boyce said. “One of our biggest differentiators in the market is the fact that we are clientless. We don’t need to have any prior knowledge of a device as it connects to your network,” he said.
“Whether that’s an IP phone, an IP printer, a contractor that you’ve never seen before, a managed desktop or laptop-anything that wants to get an IP address, we’re going to be able to identify and interrogate it, and do some sort of a policy enforcement on it.”