eWEEK content and product recommendations are editorially independent. We may make money when you click on links to our partners. Learn More.
In October and early November, Ernylee Chamlee monitored fire officials moment by moment, as they called in air tankers, trucks and manpower to fight wildfires raging in Southern California–from her desktop computer in Sacramento.
Chamlee is staff chief of operations, command and control for the California Department of Forestry and Fire Protection (CDF). She is in the control seat in one of the few real-time enterprises whose effectiveness is measured by life and death. Only by using the kind of decision support systems allowed by intelligent software and digital communications lines could she track any resource deployed, from airplanes to engines to firefighters to supplies.
But as in many corporations, “real-time” for the CDF is a relative term. Except for the minutes it takes to enter data, the CDF can instantly track the fire resources deployed. Costs are a different matter. Californias resource management application doesnt connect with the states financial systems. Bottom line: The dollar-and-cents impact of fighting fires comes about every 24 hours. When Californias firefighting system was developed, moving resources from point A to point B faster was priority..
Replacing the guesswork behind allocating resources is a system dubbed the Multi-Agency Incident Resource Processing System (MIRPS). “MIRPS helps us make sure everything is covered and that we can get the resources where we need them,” says Chamlee. “The main thing is that we have to cover for new fires.”
Many companies have been pursuing the goal of becoming a real-time enterprise. But the busy fire season in the West shows one of the few instances where second-by-second decision-making is an absolute requirement. But even Californias system, which is a model for a national firefighting edition, slowed as it hit its peak capacity of 500 users during the wildfires, officials say.
According to the CDFs Nov. 26 tally, the wildfires torched 739,597 acres, destroyed 3,631 homes and led to 22 fatalities. Expenses for fighting the fires are more than $150 million and rising.
Heres how MIRPS works. When a fire breaks out, officials determine what they need. If its clear that a fire is out of control, a commanding officer can ask for reinforcements via radio or phone. From there, a dispatcher takes the call and keys the information into the processing system, which keeps an up-to-the-minute inventory of people, equipment and services available throughout the state. Then, an operations chief such as Chamlee can draw from fire stations in nearby locales first, then draw on faraway crews in Northern California, for instance, as required.
The custom-developed system allows firefighters to access databases on servers using “dumb” machines. The information they request and see is managed on-screen by Windows Terminal Server and Citrix software. Data is communicated over a private network managed by SBC Communications and MCI.
Next Page: Challenges to being fully “real time.”
Challenges to Real Time
When the systems capacity was strained by the volume of fires in Southern California, the answer was to add two Windows Terminal Servers, says Russ Nichols, manager of applications for the CDF.
Although information is manually extracted from MIRPS to calculate costs, it cant calculate the tab on the fly. California does come up with daily estimates, however. Financial analysts take information from MIRPS, say 25 engines deployed in San Diego, and manually add it to a financial system where costs are added up once a day.
Another factor that prevents the operation from being as “real-time” an enterprise as firefighters would like is the delivery of information to dispatchers. At the beginning of a fire, dispatchers are generally sent instructions and requests by phone or radio. For extended attacks, such as fires that last for days, California has a system called Incinet, an on-the-scene Unix server that connects to MIRPS to help manage resources.
Finally, the simple need to enter new information such as location of the fire, identification of a fire engine assigned to it and last day off for a crew slows down response.
“Were as real-time as the data entry is if were swamped,” says Nichols, adding that during the wildfires, it sometimes took 15 to 20 minutes to add data such as type of resource requested, availability and destination into the system. If there was a request for an out-of-state resource, data had to be entered twice.
That data entry time could be shortened if California decides to link MIRPS with a similar countrywide system called ROSS, the National Interagency Resource Order and Status System, run by The National Wildfire Coordinating Group. Without a link or merger with that system, Chamlee and other California dispatch chiefs cant order an air tanker – an airplane retrofitted to dump water or fire retardant on fires– from Oregon directly. To get that air tanker or a similar piece of equipment from out of state, information is pulled from MIRPS and manually typed into ROSS, which scans the country to find it.
California officials now will study whether to integrate MIRPS with ROSS. Both systems track resources and their availability and status in real-time. Neither can yet deliver instant cost calculations, but ROSS uses standard accounting codes that can be downloaded into state systems. ROSS also is built on standard Internet technologies such as Java.. MIRPS, by contrast, is a custom application delivered over a closed state network.
ROSS, in general, is more sophisticated than MIRPS. For instance, it treats the procurement of people and equipment the same way it handles supplies and inventory. Officials can procure all items directly from vendors, through ROSS. They can also receive and process bills. Costs can beascribed to the locale where fires are fought. California firefighters cant do this in MIRPS.
Within the state, however, financials are a less-pressing consideration. Why? Counties in California have a mutual aid system where resources are moved from one to another without charge. The theory is that if a San Francisco engine is given to San Diego in a wildfire, the favor may be returned at a later date.
Next Page: Is ROSS ready for primetime?”
ROSS Ready for Primetime
?”>
The biggest question for the California fire defense team: Is ROSS ready for primetime? MIRPS project manager John King says its unlikely that the CDF will use ROSS next fire season, but 2005 is a possibility.
ROSS was developed by Lockheed Martin for the National Wildfire Coordination Group and designed to accommodate multiple federal agencies and all 50 states, most of which had paper systems to route resources.
This year represented the first real test of the national system. When a long fire season ended last month, ROSS had allocated and managed more than 150,000 resources such as people, engines and machines across 50 states, says ROSS project manager Jon Skeels.
According to Skeels, ROSS relies heavily on the Java programming language to tie together a wide range of applications—Oracle databases, IBM Websphere and Hyperions Brio business intelligence software—found in corporate systems.
Skeels, however, is hostage to the machinations of Congressional funding. When federal budget deliberations dragged on, Skeels had to postpone a migration to the 9i edition of Oracles database system as well as plans to add storage area networking and processing power. Those upgrades will occur before the 2004 fire season.
ROSS users paid the price over the summer when the existing system crashed for 3.5 hours in June, from a database overload. Meanwhile, network performance was abysmally slow in June because there was “too much stress on too-small servers,” says Skeels. That fact prompted many dispatchers to go back to the friendly confines of faxes and paper forms.
The culprit for the poor performance: an incorrect configuration setting in an Oracle database that keeps a running diary of requests. “There were a few days where ROSS was a four-letter word around here,” says DeneenCone, dispatch center manager for the state of Arizona, which was dealing with a slew of wildfires as performance lagged. “We were using a dual system of ROSS and paper to be fail-safe, so it still worked out.”
In Oregon, the slow performance led Randall Baley, dispatch coordinator for the Oregon Department of Forestry, to revert to “slow-time” from real-time. “We were just more comfortable with paper in the heat of battle,” he says. “ROSS has potential, but the speed of the system wasnt enough.
So how slow is ROSS? In theory, with ROSS, dispatchers take reports from the scene, check availability of resources, place orders for new equipment and personnel, and thats it.
By contrast, Baleys paper system involves filling out five different forms to request equipment, supplies and crews. These forms are then faxed or the contents read over the phone to a regional fire coordination center, where details are manually re-entered on forms. From there, details are again faxed or transcribed to a state coordination center.
Baley says the paper system can take at best a half hour to get an air tanker headed toward California. Requests for engines and supervisors could take “numerous hours” to fill. If running properly, ROSS can process that air tanker order in minutes because data is only entered once. The time frame for other resources depends on availability.
Skeels acknowledged the problems, which he contends have been corrected with additional hardware and a database fix. He also installed a plan of action to prevent future problems. “During that time [ROSS users]went back to paper,” says Skeels. “No additional acres were lost.”
Cone say ROSS functioned as promised after its June problems. Skeels is trying to quantify the returns from ROSS by tracking time and the number of orders moved.
Based on surveys with users, Skeels estimates ROSS led to better than a 50% decrease in the time needed to take and process orders. Skeels didnt have an average time for the paper system, but dispatchers like Baley say the time to fax something five times spans hours. Skeels says 2003 was largely a year to gather and create a baseline set of statistics for ROSS.
In the meantime, Skeels will continue to develop ROSS and fight a cultural war with champions of a slow, decades-old yet effective paper system.
“All ROSS is doing is taking what we were doing all along and automating it,” says Cone. “Most people see it as all bells and whistles, but it really updates an archaic system. A lot of old fire dogs think itll fail for a while, though.”
Discuss This in the eWEEK Forum