Verizon 4G Outage Keeps High-Speed Network Customers in the Dark

 
 
By Wayne Rash  |  Posted 2011-04-27
 
 
 

Verizon 4G Outage Keeps High-Speed Network Customers in the Dark


Sometime in the wee hours of April 27, the unthinkable happened. The Verizon Wireless 4G LTE nationwide data network crashed, taking all of the company's 4G devices with it. Who'd have thought that this brand-spanking new network boasting the latest in communications technology would simply, well ... stop. But it did. 

By sunrise customers were reporting in a variety of forums that the high-speed data access they depend on had simply vanished. At this point, the cause of this massive outage remains a mystery to the public, mainly because Verizon Wireless has chosen to communicate with the outside world using only a few brief and cryptic tweets.

One tweet explained that Verizon Wireless was aware that the LTE network was out. It may have been a small comfort to hear Verizon still existed and was capable of responding to the problem, but it offered little in the way of help to the thousands of customers who rely on this service. Later in the day we were told through another tweet that Verizon Wireless engineers were working to solve the problem. This is certainly tweeting the obvious. 

Finally, late in the day, Verizon again tweeted, this time to let the world know that it had found the problem and would try to fix it.  Perhaps it's asking a lot from Verizon Wireless executives, but surely they must know more than this. And if they don't know more, then perhaps it's time that the stockholders took a long, hard look at the management. 

Let's face it. In this day of reasonably modern network management, it's impossible that Verizon Wireless would be unaware of the outage within a few minutes of when it happened. It's also impossible that they wouldn't try to find the problem and fix it. So why share that information and nothing else, and why the medium of Twitter? I know for a fact that the company has employees who can write statements with a cogent explanation of what's going on, so why not tell us? 

The problem now, of course, is that speculation is running rampant. On one forum the speculation was that terrorists had hacked into Verizon Wireless, shut down the 4G and had stolen everyone's phone numbers. On another forum it was suggested that this was somehow related to the theft of information from millions of PlayStation owners. Somebody else suggested that it was the storms in the Midwest. One forum user thought maybe Verizon Wireless forgot to plug in the 4G computer. I might bet on that last one. 

So what really happened? If I had to guess (and that's what this is, since no actual information is available), I'd guess that there's an obscure configuration problem somewhere in the code that handles the provisioning of the 4G tower equipment or in the switching equipment farther upstream.

Outage Shakes Trust in Verizon's New 4G Service


 

It's unlikely to be a provisioning bug in the ThunderBolt phones because the same outage also took out Verizon's mobile hotspots. It's unlikely to be something wrong with just one type of data handling equipment because there is more than one type. But what is the problem? 

Again, if I had to guess, I'd suspect it's the switches that handle the traffic in each metro area. One of the tweets suggested that the 4G service would return one market at a time, and that in turn suggests that there's a common point of failure for each market, or at least a piece of equipment common at some regional level that went toes up (to use the technical term). Beyond that, I don't know, and apparently neither does anyone else outside of the Verizon Wireless engineering staff. 

I hope at some point those engineers will tell us what happened. But in the meantime, now that you can't use your ThunderBolt to watch movies on the beach, perhaps it's time to reflect on the wisdom of trusting one company's new technology with the family jewels. Chances are, given how new Verizon's 4G network is, that you didn't. In fact, chances are pretty good that you're rocking along on some 3G service somewhere, feeling relieved that you didn't upgrade. 

If that's the case, reach around and pat yourself on the back. There's usually little reason to rush into a new and unproven technology, and even when there's a good reason, you should be careful. If you really need that 4G bandwidth for business purposes, then try two or three 4G carriers, and get a feeling for which one meets your needs the best. And remember that meeting your needs includes meeting your reliability needs.  

I know that this means you'll likely have a few Verizon Wireless devices, as well as some from Sprint and a few from T-Mobile. That's OK. That way you won't have to worry about being totally without communications when the aliens come and steal all of the Verizon phone numbers. Chances are those aliens won't also get your Sprint phone numbers. 

Meanwhile, let's hope that Verizon Wireless gets the 4G problem solved and fixed along with how to keep it from happening again. Let's also hope Verizon is candid enough to explain what happened to its network. These things do happen, but they shouldn't happen twice.

Rocket Fuel