BlackBerry Devices Up and Running Again

BlackBerry users can breathe a sigh of relief today, April 18, now that their BlackBerry devices are back up and running. However, its not clear that all e-mail messages that formed a massive backlog on RIMs servers have been delivered yet.

Blackberry service for most North American users went out around 8 p.m. EST on April 17. At that point, delivery of most e-mail messages slowed to a trickle for some users. Many users lost all BlackBerry e-mail completely.

At the time, RIMs Web site failed to acknowledge any problem, and the companys customer service number played a recorded message acknowledging that some users might have problems with e-mail delivery.

At no time has RIM acknowledged the extent of the problem, nor has RIM posted any details of the North American outage on the company Web site, www.rim.com. The magnitude of the outage was developed from individual user reports, and questioned by network administrators, said Kevin Michaluk, CEO of Crackberry.com, a news site devoted to the portable device.

This afternoon, however, RIM did issue a brief statement through its public relations agency. The statement said, "A service interruption occurred Tuesday night that affected BlackBerry in North America. E-mail delivery was delayed or intermittent during the service interruption. Phone service on BlackBerry handsets was unaffected.

"Root cause is currently under review, but service for most customers was restored overnight and RIM is closely monitoring systems in order to maintain normal service levels."

RIM declined to explain the outage further, or to provide a spokesperson for questions.

Resource Library:

"It was funny," said Michaluk. "At first, everyone thought it was their own device. They took out batteries, to see if that would help. They couldnt use their Web browsers."

Michaluk said that the confusion spread as the outage persisted and the first kind of reports were that it was carrier specific. But Michaluk said that he didnt lose e-mail connectivity completely. "I was still getting the occasional e-mail but my Web browser wasnt working," he said.

An informal poll of Blackberry users in the Washington area showed that most lost their e-mail late last night, but received the backlogged e-mail during the early morning hours. A few Washington area Blackberry users, however, reported no loss of service.

"It had something to do with their data management network," Michaluk said. "Carriers were affected to varying degrees. Some people got no e-mail; some got it in waves; during the wee hours of the morning people started getting caught up."

RIM has been quoted as saying that the problem with the delivery system has been fixed, but that the company is releasing e-mail slowly so as not to overwhelm its system.

"RIM said they had it under control, but they had a huge backlog of e-mails," Michaluk said, "They couldnt just turn it on because that might bring down the system."

RIM has been widely criticized about its lack of communication with users and network administrators during this outage, as well as during outages that have happened in past years. During this outage, there is no evidence that the company made any attempt to notify users as to the nature of the outage, its expected duration, or how the company planned to handle its return to service.

Poll: BlackBerrys, laptops blur work-home balance. Click here to read more.

"They were pretty snooty about it," Michaluk said. "I think they probably went into fixing mode right away and didnt get in touch with people. I think that was bad. Administrators were clueless; the IT people were wondering why their BlackBerry wasnt working. This was not good communications with users. I dont think it was handled as well as it could be."

BlackBerry devices are widely used by a wide variety of commercial and government organizations. In the past, government BlackBerry use has been heavily criticized by oversight groups including Congress because it exposes government communications to a single point of failure, as was demonstrated Tuesday night.

Providers of other push e-mail services, including Microsoft, have been quick to point out that their e-mail services are not subject to such outages.

"RIMs system is based on network operation centers and there are four or five of them around the world that they route their e-mail through. But its a first generation system with single points of failure," said John Starkweather, product manager for Microsofts Windows Mobile group.

"The way our system works is a distributed network model," Starkweather explained. "If a single server or piece of a network goes down, the network can be rerouted so that one failure does not stop business."

"Not only do you have those single points of failure but you have security questions where you have that content sitting outside your control," Starkweather added, noting that governments in some countries wont allow the use of RIMs product for that reason.

Starkweather said that a lot of companies got comfortable with these first generation systems, but added that in todays business environment in which e-mail has become mission critical, more redundancy should be required.

"IT organizations understand that for their PCs and servers, they have backups," Starkweather said. He said that e-mail systems should also have some kind of backup.

Analyst Jack Gold agrees that some sort of backup plan is needed if a company is going to depend on a BlackBerry or other mobile e-mail device.

Gold, principal analyst for J. Gold Associates in Northborough, Mass., said that the nature of the backup strategy depends on the business and on how mission critical e-mail is. "There are some industries where having a BlackBerry is more mission critical than others."

Gold recommended one strategy for the next time RIM loses its NOC and all the BlackBerrys go down. Noting that the phones on the BlackBerry devices still worked fine, he suggested sending out messages to let everyone know what was happening.

"I think one thing you can do that wont cost you much is to send everyone a SMS text message letting them know its down," he said.

However, Gold, a BlackBerry user himself, cautioned against making too much of the BlackBerry outage. "As annoying as it is, in the overall scheme of things, its not that big a deal," Gold said. "What it really means is that you didnt get e-mail for a couple of hours. The majority of time it was out was at night," he said, noting that many people never realized that they were without service until they read about it the next day.

"We shouldnt overblow this," Gold said. "Its actually a pretty reliable service. Most people who have corporate e-mail have their servers go down more often than BlackBerry."

Gold said that the real problem was that RIM didnt tell anyone what was going on. He suggested that RIM take some steps to make sure that such an outage doesnt happen again. "RIM has been talking for years about having backup NOCs, but apparently something catastrophic went down. Had they had another NOC, they could have failed over," Gold said. But he also pointed out that the problem with having a failover NOC is that theyre very expensive, and might not solve all of the problems.

Gold mentioned that other mobile e-mail systems can have their own problems. "Microsoft is also susceptible to server failures and I suspect that it has a high rate of failure compared to BlackBerry," he said.

Plus, Gold noted that to use the Microsoft solution, youd have to change all of your devices over to Windows Mobile. He also mentioned that Good has the same problem with depending on NOCs that RIM has.

"What I would suggest for most users," Gold said, "is dont overreact."

Editors Note: This story was updated to include information and comments from RIM, Microsoft and analysts.

Check out eWEEK.coms for the latest news, reviews and analysis on mobile and wireless computing.