Steps 4 And 5

Sure, a team can test its code and still not find all the problems. But too often, that observation is used as a reason to avoid further testing, not just in development but after the code’s put into use.

Any given program can be tested for reliability, security and performance when it’s completed. But software can be tested even when it is just a “component” of a system.

Testing tools are widely available from such firms as Empirix, Mercury Interactive, Parasoft and Software Development Technologies (SDT). But, says Gosling, “people don’t use them.”

Testing ties up personnel, and adds to a project’s overall cost.

Since many organizations wait until the end of development to test, projects that are just about to come in “on time” and “within budget’’ often fail to do either.

Krasner, Guttman, Gosling and others agree that one solution can be a software version of the independent, not-for-profit Underwriters Laboratory, which reviews electronic equipment. Such an independent service would provide a seal of approval that a given piece of software or a software-based system is safe. Vendors who find safety to be a fundamental feature of their product—those whose software runs equipment that affects human lives, for instance—would voluntarily submit their products to the lab. If the software checked out as safe and reliable, it would be stamped as suitable for life-critical applications.

Independent testing isn’t exactly new. For more than two decades, the National Software Testing Lab in Blue Bell, Pa., for instance, has been creating and managing tests for everything from servers to wireless devices to software applications. Its clients include Dell, Intel, Nokia and the Canadian government. Keylabs of Linton, Utah, which says it has done work for American Airlines, Charles Schwab and Visa, provides similar services.

But there’s no generally accepted seal of approval for software.

Perhaps the biggest reason mediocre software persists—and threatens lives—is that individuals and corporations keep buying it.

“People put up with it,” says Jonathan Jacky, a scientist working at Microsoft Research.

Software might be the only product designed by a group of people called engineers that’s released and known to be imperfect. No one expects a building to fall, a bridge to collapse, a train to derail or a plane to crash. When any of those fail, shock is followed by accusations, inquiries, penalties, and, sometimes, legislative efforts to make sure the problem doesn’t recur.

Not so with software. According to the Cutter Consortium, an information-technology consultancy, almost 40% of 150 software-development organizations it polled last year said they didn’t believe their organizations had an adequate program in place to ensure that their software was high quality.

Cutter senior consultant Elli Bennatan notes that 29% said their companies didn’t have a quality-assurance professional on staff with any real authority, 27% said their companies didn’t conduct formal quality reviews, and 24% didn’t bother to collect software-quality metrics.

And 32% said their companies released software with too many defects. “If you don’t demand quality, you don’t get it,” SQI’s Krasner says. In effect, users and developers of software must begin demanding quality, and backing those organizations that certify developers, such as SEI, or those that support development of reliable code, such as the SCC.

Otherwise, it will be lawyers of victims, like those in Panama, and legislators or regulators that will be demanding it—in civil court and in statehouses.

Or, in the worst case, in the penal code.

---

Taking Action
To find out more about the Sustainable Computing Consortium, including how to join the organization, contact Larry Maccherone, Associate Director, CyLab, (412) 268-1715; LMaccherone@cmu.edu.

To find out more about the Software Engineering Institute and the Capability Maturity Model, go to www.sei.cmu.edu/cmmi/ or e-mail customer-relations@sei.cmu.edu.

For information on ICCP certification, go to www.iccp.org.

For information on IEEE certification, go to www.computer.org/certification/.