Can You Hear Me When It Counts?

By Wayne Rash  |  Posted 2006-09-18

Can You Hear Me When It Counts?

To John Graves, wireless communications is a lot more than Verizon Wireless catchy "Can You Hear Me Now?" tag line. Its about emergency responders hearing one another when it counts.

Graves, program director for the Government Emergency Telecommunications Service, which is part of the Department of Homeland Security, is one of the people trying to answer a key question revolving around wireless networks: Why are cellular networks among the first to fail during an emergency?

"In emergencies, its not just emergency workers using wireless, its everybody," said Graves in Arlington, Va. "Everybody wants to call home. People pick up the phone and make a lot of telephone calls. This causes the phone network, including wireless, to get congested."

According to Graves, the telephone network is built to have a call success rate of 99 percent on the busiest hour of the busiest day of the year. The problem: During national emergencies—think Hurricane Katrina and the Sept. 11, 2001, terrorist attacks—that success rate can drop as low as 10 percent.

When wireless companies suffer outages due to equipment damage and lost connections to the public service telephone network, PSTN (Public Switched Telephone Network) calls are routed to functioning networks, overloading them.

The challenge is closing that gap. The issue is on the front burner of everyone from the industry association CTIA, which had its Wireless IT & Entertainment conference in Los Angeles Sept. 12-14, to wireless carriers and first responders. The race is on to prepare for the next big emergency, and its not clear wireless companies can get it together due to interoperability, allegiance to serving customers and lack of communication among key players. As things stand today, there is no coordination among the companies preparing for the Next Big One. The bottom line: The industry may not be ready for the next emergency, whether it be a massive weather event, an earthquake, a terrorist attack or a pandemic.

In the meantime, each provider—ranging from Cingular Wireless to Sprint Nextel to Verizon Wireless to T-Mobile USA—is attacking the issue differently. Some are working to make sure that their networks can survive, others are making sure that they can coordinate the rebuilding process, and a few are focusing on the needs of first responders. "Always have a Plan B," said Josh Lonn, regional director of development for the South region at T-Mobile in Frisco, Texas. "We have to spread our risk as much as we can."

The companys Plan B: Assume the worst. T-Mobile had already moved important switches out of harms way months ahead of Hurricane Katrina. In addition, the company hardened its cell sites, installing generators in every location possible and creating redundant communications. T-Mobile, unlike the other wireless providers in the area, was safe because of advance planning. Even then, T-Mobile network technician Louie White had to protect a massive switch with plastic and endure the winds and rain whipping through the Galleria building in Metairie, La. The switch and White survived, enabling T-Mobile to provide wireless communications in New Orleans and the surrounding area.

Joe Farren, director of public affairs for CTIA, told eWEEK his organization is working closely with the Department of Homeland Security and has a disaster preparedness and recovery certification program to ensure "members have taken a number of steps to prove that theyre ready to respond to a disaster. So far, the big four national carriers have been certified," said Farren, whos based in Washington. "Others are going through the process as we speak."

Indeed, wireless giants are working to bolster networks ahead of the next emergency.

Sprint Nextel Vice President for Public Sector Programs Chris Hackett said the company, in Reston, Va., is eyeing vulnerable markets such as Florida and "hardening the cell sites." Hackett said the companys hardening program includes putting generators into every cell site possible and arranging for fuel. Hackett said the Nextel system, which provides the ability to talk directly from device to device without a switch, is critical for first responders, and he wants to make sure that cell service is maintained during an emergency.

Capacity crunch

According to industry experts, there are three things necessary to allow people to communicate during an emergency: Keeping existing capacity running, adding capacity where possible and prioritizing communications steps such as hardening cell sites and switches, which is already under way by wireless carriers. Placing critical equipment out of harms way and having workers on standby are also important in keeping existing communications running.

Because there is more demand for communications during an emergency, capacity is critical, said experts. Using temporary cell sites to supplement or replace the existing infrastructure and adding alternate forms of communications such as amateur radio also can take the load off the commercial communications system.

Perhaps the most challenging part of the equation in ensuring critical communications take place is cultivating a willingness among commercial operators to modify their phone systems to give users with critical needs priority. Graves is also in charge of the WPS (Wireless Priority Service), a program for prioritizing wireless traffic. Graves said that on 9/11, there was already a priority service in place for wired phones but that capability was lacking for wireless phones. The White House then ordered a wireless priority service. He said that within two months, T-Mobile and then Cingular were able to provide wireless priority in New York and Washington. "It took a year for the first carrier, T-Mobile, to offer nationwide service," Graves said.

Graves said that now all GSM (Global System for Mobile Communications) carriers offer nationwide WPS access. "We have Cingular, T-Mobile and Nextel since IDEN [Integrated Digital Enhanced Network] is basically GSM," he said. Its taken longer for the CDMA (Code Division Multiple Access) carriers to offer WPS, and, so far, only Verizon Wireless has the service available nationwide. Graves said that Sprint will start next spring.

In the United States, the WPS works by managing the queue for calls coming into the wireless network, or calls to a wireless phone. If theres a wait for a slot on the cell, the WPS system puts the highest priority calls in the queue first. However, WPS does not bump existing calls in the United States, even though it has the capability to do so.

To read about what CTIA is doing to prepare for emergencies, click here.

Ultimately, Graves is responsible for providing communications services to support the continuity of government. To accomplish this, the National Communications System, the government organization that makes sure that wireless carriers are able to work together, helps set the standards.

Expanding the communications capability can mean either adding more cell sites or providing better access to land lines. Depending on the nature of the emergency, expanding access to land lines may be problematic. "Mother Nature is pretty hard to outsmart," said Bill Smith, chief technology officer for Atlanta-based BellSouth, which was hit hard during Hurricane Katrina. "Well-prepared doesnt mean youre invincible," said Smith, noting that BellSouth thought it was prepared for a disaster, but the company wasnt ready for a major underwater cable to be taken out.

When capacity is severed, wireless companies are left with mobile cell sites, provided they can be connected to sites with switches. These mobile facilities called COWs (cell on wheels) or COLTs (cell on light trucks) can be driven into an area where coverage is needed and put into operation. In some cases, they can connect to a terrestrial T-1 line and, in other cases, use a microwave or satellite link. Nextel, for example, has a dedicated satellite COLT thats designed for long-term operation in a remote area and can be driven to the place where its needed, locked down and left to run on its own. T-Mobile, on the other hand, has even developed a "Cell on Hummer" in which the company can provide cell service anywhere a Hummer H1 can go.

Next Page: The carriers lay out their plans.

Carriers Lay Out

Their Plans ">

Planning for the future

Each carrier interviewed by eWEEK said it was preparing for emergency communications, but there was little agreement on what it meant to be ready.

For instance, Cingular Wireless has spent $17 million on ensuring it can fix a damaged network quickly.

"We have purchased emergency response equipment including [MACH (mobile access command headquarters)]. Its an office on wheels that provides satellite connectivity, data, voice and video," said Cingulars Director of Continuity Planning and Crisis Management Tina Brown, whos based in Atlanta. Cingular now has two MACH units, the larger of which can house up to 30 people.

MACH 2 is a smaller unit for quick deployments. MACH 1, which is about 1,000 square feet, is built into an expandable 65-foot trailer designed to work with an external communications unit. "These two vehicles are very high-tech, military-grade communications facilities," said Brown, adding that the units can handle radio, VOIP (voice over IP) and video with capacity for 6M-bps transmission rates.

The company also is taking steps to make sure that the worst problems of the past wont be repeated. "We built a switch outside of New Orleans thats not quite as vulnerable," Brown said.

Brown said that Cingular is stepping up its plans to get ready for major events when theyre known in advance. "For [Tropical Storm] Ernesto, we deployed part of our equipment to Columbia, [S.C.,] to stand by to see where the storm was going to go and how intense it was going to be," she said. Brown said that the deployment included the smaller MACH 2 command unit, the emergency communications units, two equipment trailers and two RVs for sleeping. Generators were placed in Raleigh, N.C., she said.

While Cingular is adding generator backup power to some of its cell sites, the company isnt going as far as some others. "Were making sure we have N+1 redundancy on our backup power," explained T-Mobiles Lonn. "Thats two times the redundancy in case our generators fail." Lonn said that T-Mobile is investing heavily in permanent generators for backup power and in backups for the telephone company backhaul. The company has had a fleet of COWs, COLTs and portable generators for some time but is continuing to invest in more, Lonn said.

Verizon Wireless is taking a similar direction. "Reliability has always been in the back of our minds," said Hans Leutenegger, area vice president for network for the South, in Charlotte, N.C. "You have to be thinking reliability well before a disaster, but you have to have already done it when the disaster comes."

Leutenegger said that Verizon Wireless has permanent generators at every cell site where theyre allowed, and he added the rest have portable generators. In addition to power, Verizon Wireless is making sure switch locations can withstand Category 5 hurricanes in Florida, he said. Verizon Wireless also uses two special fixed command centers, one on each coast for backup.

Sprint Nextel is in an unusual position. Its the communications company of choice for many first responders because of the companys support for direct communications between handsets. This ability for the first responders that the company focuses on to communicate without the need for the wireless switches or cell sites simplifies Sprint Nextels solution but doesnt eliminate the problem of needing to provide power for its cell sites. According to Hackett, his company has made it a point to locate critical infrastructure where its safe from most threats. In addition, he said that the company has developed the ability for switches to back each other up, and to move traffic to switches that are less crowded. Sprint Nextel also keeps a fleet of COLTs and emergency generators standing by in case they lose power or cell sites.

As strange as it may seem, one of the biggest challenges to preparedness for the wireless carriers is communications. But in this case, its communications with government entities; first responders; NGOs (nongovernmental organizations) such as the Red Cross, Salvation Army and others; their customers; and one another. Unfortunately, those efforts have not met with success in some cases. Each of the wireless companies tells stories of employees being prevented from restoring service because word never reached the people enforcing access or curfews.

BellSouths Smith said phone service restoration after Katrina was delayed because police wouldnt let the companys technicians splice fiber after curfew. Smith said there needs to be a uniform credentialing plan.

Likewise, each of the companies has said that getting help from the government is very difficult and that, even when promised, such help rarely comes through. As a result, these companies have had to resort to hiring private security organizations such as Blackwater USA for security because police or National Guard troops never arrived when promised.

Meanwhile, some companies arent used to working together. There is some progress on this front as Verizon Wireless and T-Mobile both routinely open their networks to anyone who can connect with them. "During Katrina, a lot of carriers were having problems with switches," Leutenegger said. "We allowed all customers to access our network. We just turned off authentication for all of our competitors, allowing them to make calls freely." The move is part of Verizons emergency checklist.

What have businesses learned from the Hurricane Katrina disaster? Click here to read more.

A Cingular spokesperson told eWEEK that his companys policy was to open its network. Steve Mondul, deputy assistant to the governor of Virginia for commonwealth preparedness, just wishes companies could do a better job of communicating with the state. Mondul said wireless companies havent been planning enough for compatibility, something the government at all levels actually needs to see happen. "Compatibility is not a standard part of the planning. It needs to be institutionalized so that everyone knows that its happening," said Mondul, in Richmond, Va.

While Mondul said he welcomed the efforts by the wireless companies to bring in equipment, he said that planning and coordination are still needed. "You dont want your generators on towers near major evacuation routes and major command centers to be running out of gas," he said.

Mondul said that planning is critical in working with state and local governments. "It would be really good to have the company technical reps meet so we can determine critical areas and needs and address them in the planning cycle rather than in the response cycle," Mondul said. "We need to develop a conceptual plan to deal with it, rather than having no plan."

Of course, the level of interaction with government varies by wireless company. Sprint Nextel, for example, has invested heavily in planning sessions with governmental entities at all levels. According to Sprint Nextels Hackett, the company routinely conducts seminars for governments, first responders and others to discuss how to create and implement a communications plan. Then the company conducts exercises to practice its responses.

While many in government wish that the wireless carriers would communicate among themselves more effectively, there are limits on whats possible. For instance, too much communication among wireless carriers can be seen as collusion by antitrust authorities. In other words, two wireless carriers could reach an agreement on how to support each other in case of a major emergency and then be charged with antitrust violations for talking to each other.

However, there is at least one means for wireless carriers to work together. The National Coordinating Center for Telecommunications, or NCC, was created in 1984 after the breakup of AT&T to ensure the federal governments communications needs were met. Brian Carney, manager of the NCC, which is part of the DHS, said carriers could cooperate through his group.

On the local front, municipal and state governments have to find ways to work with wireless companies in their emergency operating centers and put lines of communication in place before a disaster happens. And, of course, the carriers themselves have to do as much as they can to be ready. "The reality is that its a dangerous world," said Lonn. "Im confident that weve done all we can and have a great ability to respond. Id never say were perfect, but I feel a heck of a lot better now than I did two years ago."

Check out eWEEK.coms for the latest news, reviews and analysis on mobile and wireless computing.

Rocket Fuel