The Planet Prepares Data Center ‘Bunkers’ for Hurricane Ike

Catastrophic events such as storms, wars, financial meltdowns and disasters offer a unique opportunity for captivating observation (blow by blow news coverage) during the event and, later, exhausting review (post-event analysis).

Hurricane Ike is no exception and for anyone interested in data center operations, backup and recovery, redundancy and how the hell to maintain 10,000 servers amid a storm 900-miles in diameter, watch the folks at The Planet.

The Planet runs more than 10,000 servers in two data centers H1 and H2 in the Houston area. (The Planet runs another 48,000 servers total at H1, H2 and four data centers in Dallas.)

As Ike roars ashore the staff is preparing both sites for the storm and blogging the action in posts to a customer forum.

The posts paint a controlled but distressed picture as customers voice nervous concerns and staffers Eric Bush, Houston Data Center supervisor, and Kevin Hazard, a Web Hosting Evangelist (no relation).

Some of the rather pedestrian, but important preparations being recounted:

“Securing exterior windows and doors and checking the roof and lot for loose objects.Confirmed fuel levels and acquired an additional 10,000 gallons backup to supply both data centers and downtown headquarters for one week.Transferred Houston-based staff members to Dallas continue service and answer support calls.Supplied both sites with food and water for staff for several days.A ride-out team, including Bush, will stay “locked-down” at the datac enters through the storm. Their families were evacuated out of the area to relieve an added stress, Bush said.Several 10Gbps connections to Tier 1 bandwidth providers are being checked at both sites.“

The forum still reeks of complaints following an explosion and fire that shut down H1 in May and June. Many customers, still tender from that episode are weighing in on Ike.