Software Testing is Important to Quality
Software testing is important to quality
I have been involved in testing and quality for almost my whole 25+ year career. I know how important testing is to quality. It is a matter of poignant pride for me that, in the two lost Shuttle missions, software failure was not the cause-thanks to the legions of software testers who worked on the mission control center systems.
However, there's less software involved in a shuttle mission than in driving your Prius to the grocery store. Seriously. Software for late 1970s and early 1980s hardware is orders of magnitude simpler and smaller than software for 2010-era computers. You could not run an iPhone or a Prius on the computers that run the shuttle.
And, even when computers were smaller and simpler, you could not exhaustively test the systems. Glenford J. Myers recognized this fact in The Art of Software Testing, the first book on software testing written in 1979. Whether testing cars or data centers, software testing is a necessary but insufficient means to quality.
Failures are not proportional to defect size
This brings us to the next lesson from Toyota, though it is by no means company or culturally specific. We have clients around the world. It is common across borders, companies and cultures for people to forget that complex systems can exhibit unpredictable, and in some cases, catastrophic, failures. It is also common for people to forget that failures are not proportional to the size of the defect.
To see examples, one can consult the Internet for the answers to four questions: Why did a SCUD missile evade the Patriot missiles and hit a troop barracks in the first Gulf War? Why did the first Ariane 5 rocket explode? Why did not one, but two, NASA Mars missions fail? Why did the Therac-25 kill cancer patients? In each instance, the answer is discouragingly simple: an infinitesimally small percentage of the code proved defective.
Again, size deceives. If you knock a rivet out of a bridge, does the bridge fall? No. If you nick a wire in a single suspension cable, does the bridge fall? No. If you carve your name in a facing stone on a pillar, does the bridge fall? No. Yet, some software fails for similarly small defects involving just a few lines of code.