Pinched by budget cuts in 1998, NASAs Goddard Space Flight Center was under pressure to improve operations while reducing costs.
The center had already reduced control-room coverage for some of its small explorer satellite missions, from 24/7 to 8 a.m. to 5 p.m., five days a week, in order to save money. But that change made some people nervous.
“We lost the ability to maintain 24/7 operations, which increased the risks for missions if something went wrong,” says Rick Saylor, the lead development engineer for small explorer mission (SMEX) satellites at the time. “We didnt have insight into the health of the satellites, and we wanted to mitigate that risk.”
If one of the $65 million satellites experienced any problems, NASA engineers wouldnt know about it until the next business day. The satellite, meanwhile, could switch into safe mode, shutting down all but essential operations to stay in orbit. But it would cease gathering information until the problem was corrected.
Jeff Fox, now president of wireless solutions provider Mobile Foundations, was working for NASA on a general research project at the time. NASA then asked Fox to develop a solution that would limit the risks for the upcoming TRACE (Transition Region and Coronal Explorer) mission. It would be the first time the agency would use an 8-to-5 monitoring schedule for a new mission. All other operations that had abandoned 24/7 on-site monitoring had ceased delivering primary data, although they still sent information.
NASA needed a solution that could notify engineers who were not in the control room about events occurring in the spacecraft; provide online summaries of data sent to ground stations when the craft passed over the earth several times each day; and alert engineers to any trouble the satellite might experience.
Lotus Domino Tracks Engineers Fox and colleagues developed the Spacecraft Emergency Response System (SERS) using Lotus Domino.
“Lotus Domino was the best available product, and we didnt want to reinvent software that was already out there,” says Fox. He added a wireless component, which wasnt available on Domino. (Lotus plans to release Domino Everyplace, with wireless capabilities, in the second half of 2001.)
Several times per day, each satellite monitored by SERS downloads scientific data it has collected and “health and safety” data about the satellites components. The health and safety data is forwarded to a Domino server at Goddard Space Flight Center.
The SERS program analyzes the health and safety data and generates alerts appropriate to the urgency of any problems it finds. A low battery might generate only an e-mail alert, while a sudden voltage surge would require an immediate pager alert.
SERS generates incident reports and alerts different engineers depending on the type of problems, time of day and other rules.
“It automates many routine tasks, provides rapid situational awareness about what has happened, and provides ways to react to the data within a wireless environment,” says Fox.
If a satellites gyroscope shuts down, for example, SERS will contact the engineer on call via a two-way pager, PDA, cell phone, etc. If that engineer cant be reached, SERS will contact his or her backup. Personnel schedules and their communications devices are stored in profiles the staff creates in a Domino database.
The engineer can reply to the message and get more details about a problem at a secured Web site, then return to the control room to fix the problem. For security reasons, the engineer cannot control spacecraft from his wireless device or home computer.
Initially, SERS issued alerts for just about everything. Filters were slowly added so that the alerts became more discriminating. During the first six months of the TRACE project, about 1,500 alerts were transmitted but only six required an engineer in the control room.
Mission Accomplished “SERS performed flawlessly,” says Saylor. “Im amazed how smooth it went.”
Fox attributes its success to a customer-centric design that was developed as a result of continuous consultation with engineers at NASA. “They were part of the development team and that makes them feel it is their software.”
Technically it is. Because NASA paid for development of SERS, the agency owns the software, Fox says. Mobile Foundations cant charge NASA for the software, but it can collect labor costs involved in developing and operating the system.
SERS is currently deployed on about a half-dozen NASA satellites, which cost the agency about $20,000 each per year, and the Hubble telescope, which uses SERS to track anomalies and generate automated reports (not for wireless messaging).
Mobile Foundations currently is working with NASA to deploy SERS on several upcoming missions, including the Triana satellite. Triana will orbit around Earth from more than one million miles away, tracking pollutants, vegetation and solar flares 24 hours a day.
The mission is far more complex than any current SERS projects. Triana will gather data continually and employ several workstations that will interface with SERS.
So far, SERS has saved NASA an estimated $1 million per year, according to Saylor. The Goddard Space Center currently uses about 10 engineers in its control room for several SMEX satellites, Saylor explains. Without SERS, it would need about three times as many engineers working in shifts around the clock.
Looking ahead, Fox hopes SERS will be deployed on “lights out” NASA missions, where no person sits in the control room, though a couple of engineers monitor the mission from remote sites. Mobile Foundations has plans to add a Java visualization application to the solution and expand its customer base.
It has developed a Groupware-Enabled Automated Response System to provide private enterprises with mission-critical solutions similar to SERS.
Fox foresees markets in telemedicine, search-and-rescue missions, earthquake monitoring and law enforcement—all services requiring quick responses from certain people.
“Let us automate things that are difficult but mundane,” says Fox.