NaviSite Says 85% of Servers Back Online

As of 12:30 EST Nov. 8, hosting company NaviSite said it was "on track" to bring up more servers online and had about 85 percent of its servers already up in its new data center.

The company has been clambering since Nov. 3 to fix a days-long service outage that has put tens of thousands of Web sites out of service.

The problem that has brought down an estimated 165,000 Web sites was the result of a botched data center migration that went horribly wrong on Nov. 3. Five days have passed and NaviSite is still struggling to resolve the issues.

Resource Library:

"Our best estimate timeline for completion of the migration process remain unchanged and [we] believe that all of the servers will be up and running by this evening—barring unanticipated exceptions with any individual machine(s)," said Mark Clayman, senior vice president of hosting services at NaviSite, in a Nov. 8 letter posted on the companys homepage.

Clayman said NaviSites engineers are working around the clock to bring sites back up as soon as possible. "We are continuing our all hands on deck approach to completing the migration."

The botched data center migration started when NaviSite planned to move the data center belonging to Alabanza, the company it acquired in August, from Baltimore to NaviSites headquarters in Andover, Mass. To save time NaviSite planned to physically move 200 servers from Baltimore to Andover, and then transfer data over the Internet from another 650 servers. But when the Internet transfer proved too lengthy (at which point panic might have begun to set in for NaviSites engineers), to save time the company decided to again physically move servers from Baltimore to Andover.

In the meanwhile, the Internet transfer ran into issues synchronizing Address Resolution Protocol Requests.

As of the afternoon of Nov. 7, more than half of Alabanzas 850 servers were back online, as were about 65 percent of the customers Web sites.

At least one angry customer has blogged about her experience during NaviSites historic five-day outage—and received an outcry of questions from other NaviSite customers. Cynthia Brumfield, on her IP Democracy blog, wrote that she has received phone calls and e-mails from "desperate, hollow sounding small business owners and Web hosting resellers asking for advice, or to let me know of their dire situations, or to find out what I know….Why have I received these phone calls and e-mails? Because Ive been blogging about the situation. And because NaviSite is still not answering customers phone calls or e-mails."

Brumfield said that one man called her to say that his businesses, which has 1,000 Web sites directly hosted by NaviSite, are now dead.

"NaviSite hasnt returned any of his calls, or e-mails," wrote Brumfield. "And despite promising to bring top management into the endless, useless conference calls NaviSite has been hosting since Sunday, this man told me that he had been on a call [Wednesday]…but no one from NaviSite chose to participate."

Check out eWEEK.coms for the latest news, views and analysis on servers, switches and networking protocols for the enterprise and small businesses.