eWEEK content and product recommendations are editorially independent. We may make money when you click on links to our partners. Learn More.
The Port Authority of New York & New Jersey lost 75 employee lives, its headquarters in Manhattan, a major data center, Internet connectivity and 2,500 PCs during the September 11, 2001, terrorist attack on the World Trade Center. At a New York trade show this week, an agency security officer spelled out some of the key lessons learned about disaster planning and recovery from the horrendous experience more than three years ago.
“I dont want anybody else to go through what Ive gone through—ever,” said Michael Frank, assistant director of the technology services department for the PA (Port Authority), a bi-state private/public agency that oversees the seaport, airports and many other major transportation and supply chain routes in New York and New Jersey.
All together, more than 2,800 people perished in the WTC disaster, Frank said during a session at the InfoSecurity Show called “When Bad Things Happen to Good People: Lessons Learned from the WTC Attack.”
On 9/11, the telecommunications infrastructure in downtown Manhattan went down, causing long-term outages to 300,000 telephone lines and 3.6 million high-capacity data circuits.
The region also faced “transportation disruptions affecting the delivery of goods and materials,” Frank told the audience. Specifically, the disruptions included “bridge and tunnel closings, increased vehicle inspections and intensified inspections of imported goods.”
Like untold numbers of other businesses in the area, the Port Authority suffered financial damages, too. Traffic dwindled at seaport and airport facilities, causing economic losses to the agencys customer base of airlines and other transportation firms.
Moreover, the Port Authoritys operations, health, safety and revenue systems had all been moving in more of an IT direction in recent years.
The Sept. 11 events destroyed the agencys mission-critical room for distributed systems; firewall, DMZ and IDS (intrusion detection system) security environments; 20 local area networks (LANs); its central telephone switch; and both of its remote-access environments.
The PA also lost connectivity to an outsourced systems administrations firm that maintained its SAP and PeopleSoft human resources environments.
About 2,500 employees were displaced to other offices, stretching their commutes to as many as two-and-a-half hours each way.
“Staff was stretched to deal with continuing demands—increased security at facilities, the demands of WTC clean-up and emotional stress.”
But on a brighter note, several of the PAs IT systems remained up and running despite the disaster, including all major operational, health and safety systems for non-WTC facilities; the financial/procurement system; and payroll.
“Checks were paid to more than 5,000 employees on schedule, two days after the event,” Frank said.
Most of the PAs sites did manage to stay in communication “due to redundancy of core data communications lines.”
Meanwhile, PA staff in lower Manhattan adopted “aggressive use of supplemental technology—cell phones, PDAs, BlackBerry devices.”
The PA established an EOC (emergency operations center) immediately, and then stepped to an upgraded EOC within a month of the event.
Within five days of Sept. 11, Internet connectivity was re-established. Within a week, “servers/applications/user data, which was backed up with the corporate solution, were restored.”
But in the aftermath of the disaster, the Port Authority also has made a number of changes to its IT policies and practices. “Some client departments made less than optimal decisions related to backup strategies,” Frank said. So, at this point, all departments are required to back up their data, “without exception.”
“Not all vendors/suppliers reacted to PA needs in the same way,” according to Frank, who was especially disappointed in how two of the PAs long-term IT suppliers responded when he tried to procure new PCs.
“First they asked, Whos the Port Authority? Then they said, We cant deliver,” he told the attendees.
IBM, in contrast, came through right away with both new computers and disaster-recovery services, he said.
“Know your [IT] vendors,” Frank advised. “At times like these, you really find out who your friends are.”
Frank made other suggestions, too. “People are job number one,” he said. When faced with disaster, companies should focus on employee morale, transportation and disruption issues. They also must allow time for “honoring those lost and attending to the injured.”
On the IT side, companies should “stick to your principles regarding IT security—authentication, information integrity, access control,” Frank said.
“Establish three data centers if economically and physically possible. Consider outsourcing applications to qualified vendors.”
Prior to the WTC disaster, the PA had already set up dual data centers, as well as a policy of “outsourcing lower-value-added activities [such as] PC break and fix, systems administration and help-desk call handling,” he said.
Also, as Frank sees it, the 9/11 disaster has afforded the agency some opportunities, such an acceleration of its server consolidation plans; outsourcing of part of its remote access solution; creating a new document management system; and expanding its work on consolidating logical and physical security credentials.
“[But] we are still very much concerned about both physical and IT security,” the security officer said. “Business continuity and resumption is critical to PA and the region. Threat readiness remains high.”