Most of the desktop computers in the UKs Department for Work and Pensions were paralyzed for four days on Monday, when a failed upgrade took them offline. The outage, covering 75 percent to 80 percent of the DWPs 80,000 PCs, is one of the largest in the UK governments not entirely impressive IT history.
And possibly one of the most costly. According to staff reports, the outage occurred on Monday afternoon, disconnecting staff e-mail, benefits processing, and Internet and intranet connectivity. According to one, a limited network upgrade from Windows 2000 to Windows XP was taking place, but instead of this taking place on only a small number of the target machines, all the clients connected to the network received a partial, but fatal, “upgrade.”
Another source says that the DWP was trialing Windows XP on a small number (“about seven”) of machines. “EDS was going to apply a patch to these. Unfortunately the request was made to apply it live and it was rolled out across the estate, which hit around 80 percent of the Win2K desktops. This patch caused the desktops to BSOD and made recovery rather tricky as they couldnt boot to pick any further patches or recalls. I gather that [Microsoft Corp.] consultants have been flown in from the U.S. to clear up the mess.” EDS is also thought to be flying in fire brigades.
If these claims are true, the DWP could face grave difficulties in rolling all of its machines back to their previous, working state. Staff from Microsoft and EDS are reported to have been working around the clock to dig the department out of the pit, while speaking on the “Today” program Friday morning, a spokeswoman amusingly insisted that the departments systems had not in fact fallen over. They were working; it was merely the case that “80 percent of desktop computers are not connecting through to the mainframe systems.”
So thats cleared that up then. She added that the emergency payments system was “working perfectly.” The emergency system appears to have kicked in on Wednesday, and the department was preparing a press release on the matter Thursday. There was no sign of it when this story was published.
Reports coming in on Friday however suggest that at least some of the DWPs systems are coming back online.
For more IT news, check out The Register.
Check out eWEEK.coms for Microsoft and Windows news, views and analysis.