No Holiday for IT

By Deb Perelman  |  Posted 2007-11-21

No Holiday for IT

IT stands among the ranks of vital professionals—healthcare, public safety workers and government—for whom evenings, weekends and holidays are par for the workplace course.

However, without the glamour associated with saving lives, restoring heat to freezing homes or guiding people through the sky so they can be reunited with their loved ones for the holiday, few even realize that IT is keeping the lights on.

For IT professionals, the biggest holidays of the year are rarely a cause for celebration. Systems need to be upgraded when the office is shut down and all too often, servers partake in a Murphys Law, going haywire when there are the fewest people around to restore them.

Almost every long-time IT professional has at least one horror story to share.

"Back when I lived in Australia—my story is from an Easter weekend—which was the only weekend that the network could be shut down for long enough to restructure things," John Terpstra, an IT professional in Austin, Texas told eWEEK. "Of course, it was stinking hot outside, about 91 degrees. The door handle to the server room was so hot, I couldnt touch it. Inside, it turned out the air conditioning had broken, and all of the servers were still running, but from the moment I walked in, they started dying, one by one."

To read about five steps to better job security, click here.

Of course, Terpstra continued, none of the people who needed to be contacted could be, and while his job had not been to reorganize the hardware, this is what he ended up doing. Out of 24 servers, only five survived the weekend, and each of those died within three months of the incident.

Kevin Behr, the chief technology officer and managing principle at Assemblage Pointe in Lancaster, Penn. could commiserate, having his own story from when he worked for a large retailer a few years back.

"One of the rock star programmers wanted to leave a day early for the Thanksgiving holiday, having gotten all of his work done," Behr said. "Not realizing there had been a change freeze at the end of October, as the company did 60 percent of their business in the last two months of the year, hed put a lot of code changes on their site. Nobody knew that he did this, and when the operations guys who were left rebooted their servers, there were literally no items left for sale on their site."

These IT professionals ended up spending the days before Black Friday trying to figure out why they had nothing for sale on their site. They had to restore the site from the backup, and literally worked three days straight through the holiday weekend to get the site back up, Behr explained.

Of course, the biggest and most famous holiday weekend in which nearly every IT professional in the world was on the clock was the Y2K turnover.

"It was a horrible experience for all of us in any kind of finance," said a former independent consultant and IT specialist in New Jersey who asked not to be identified. "When that COBOL clock would turn over, we all had to be there in case someone didnt code it correctly. When midnight hit and everything was okay, we still had to watch the screens for four or five hours, while the clock turned over in different time zones. "That was how we spent our New Years Eve, while our friends were drinking champagne at parties."

These stories underscore the sadness that can accompany professionals, stuck fixing IT disasters while the rest of the world is out gallivanting.

Bill Light, an applications development and technical expert at the Permanente Medical Group in Oakland, will never forget one Thanksgiving weekend that stood out over two decades he worked at the San Jose water company. His team had a mainframe conversion to do that could only be performed over the holiday. They didnt finish until Monday morning.

Page 2: No Holiday for IT

No Holiday for IT

Once he finally made it home from his hellish weekend, someone remarked to him, "Isnt it great just to be thankful?"

"As if!" he thought. "We had a Thanksgiving pizza."

Holidays and weekend such as that one can take their toll on IT pros, even when a job offers them comp days or overtime pay, Light said.

"They tell you that you can take the next Thursday or Friday off, but youve still missed Thanksgiving dinner," he said. "You cant get that back next week."

Often, its not one or two missed holidays that cause technology professionals to hit their breaking point, but the combination of this and the lack of sympathy or thankfulness from the people who were lucky enough to spend the holidays with their loved ones.

The IT work force lacks resilience and durability. Click here to read more.

"So many companies dont realize that the people who get screwed are those left on the holidays or the less-senior workers trying to earn their stripes," Behr said. "Something always blows up and the people who leave on the lights are screwed. They have no context to try to figure out what went wrong."

In Terpstras case, his bosses did finally say thank you two weeks later when there was a period of accountability, but there was certainly no bonus.

"Their real concern was that it had been a very costly affair, and how it could have been avoided," he explained.

Behr argues that these repeated situations—especially considering that he estimates that 80 percent of outages are self-inflicted by companies—can eventually make IT professionals hate their jobs.

"There are a lot of really sharp folks doing their time, but too often the reward for good work is more work… often, these IT guys were worn out even before Thanksgiving came," he said. "Thanksgiving is just one piece of the pie."

In many cases, the smallest human acts from managers and bosses can make a huge difference in IT worker morale, reassuring them that their employers arent just taking advantage of them.

"I had a boss one year who brought down a plate of dinner from his house, and a bottle of wine, even though it was forbidden to have any alcohol on the job," Light said.

The boss hadnt given him the night off or a reprieve from the work that the company needed done, but he had shown some thankfulness and understanding, something that Light has never forgotten.

Check out eWEEK.coms Careers Center for the latest news, analysis and commentary on careers for IT professionals.

Rocket Fuel