Ships Systems: Surviving the Storm, and the Recovery

Fighting against the ravages of nature and the extremes of weather has become part of the job for Jan Rideout, the vice president and sector CIO for Ships Systems at Northrup Grumman, the company that builds the latest, most powerful destroyers for the U.S. Navy.

Rideouts IT operation has been based at the companys shipyard in Pascagoula, Miss., where she supports that shipyard and another that the company runs in nearby Avondale.

Last year, Rideout found herself preparing for Hurricane Ivan—a storm that struck the Gulf Coast once, and circled back for another try. She successfully implemented her companys business continuity plan that kept the shipyard operating.

This year, with Hurricane Katrina, things were different. “With this storm we had less time,” Rideout said. “It was Friday morning when they started to change the predicted track and said it was going to New Orleans … We didnt have a lot of time to react.”

Still, Rideout had a plan. “Our prep was pretty much the same as we do for every storm,” she said, describing how her staff backed up multiple terabytes of critical data, and then prepared it for transport. “We sent our backups to Dallas. That made a big difference. We used to send them to Mobile,” she said.

/zimages/4/28571.gifClick hereto read more about how Hurricane Katrina tested disaster recovery plans.

Just as in the previous year, the backups went to a Northrup Grumman Corp. data center in the Dallas area where they would be safe from the storm that was bearing down on the Gulf Coast. She had to move fast.

“This was very quick,” she said. As in the previous storm, a few critical systems were backed up, but kept running in hopes that they would survive the storm. “We left e-mail and Blackberry servers running,” Rideout said, “but took everything else down, including the PBX.”

Then the IT staff had to protect the servers against rain that might come in through a broken window or damaged roof. “We covered all of the servers that were not running with plastic wrap,” she said.

Next, the problem of providing power to keep those servers running.

Last year, Rideouts IT staff had rented a standby generator, then switched to the shipyards internal power during the storm when the rented generator gave out. This year, things were better. “We did have generators this year,” Rideout said.

“At Avondale we had the generator installed so it would start automatically,” she said. “In Pascagoula we had to do a manual transfer to the generator.”

By then, the Northrup Grumman IT staff had done all it could do to protect the data center against the ravages of Hurricane Katrina. It was time to evacuate to Dallas, hoping that the steps that had worked so well last year would work again.

No such luck.

/zimages/4/28571.gifRead the full story on CIOInsight.com: Ships Systems: Surviving the Storm, and the Recovery