Zero Defects: The Only Acceptable Standard

Peter Coffee: Quality isn't just a matter of counting bugs versus time; it's a matter of appreciating the high leverage of the worst defects.

Im going to begin this letter with a nasty, nitpicky example thats definitely a case of throwing stones from inside a glass house. I know that the sin Im about to chastise is one that we sometimes commit ourselves at eWEEK, but thats just too bad: The example Ill now cite of someone elses carelessness is the one that happened to be at hand when I needed it.

I was recently researching the product line road map for the newly merged Hewlett-Packard and Compaq, about which Ill share some critical comments in my eWEEK column a week from today. On my way to that Web page, however, I passed through the "Merger Information" portal page on the HP site, where I encountered the promise of "Infrastucture Solutions." (Perhaps the spelling error has been fixed by now, but I have a screen photo if I ever need to prove that I saw it.)

I dont know what it feels like to be infrastuck, but theyll have to pay me to try it.

OK, OK, I know that this is a nasty piece of nitpicking, but theres a reason why I take up your time by mentioning it. That entire HP Web page only contains 172 words of text, and one of them—a stand-alone hyperlink label, right out there in the open—is wrong. Thats a defect rate of almost six per thousand, and thats the most charitable measure: If we choose paragraphs as our units of work, the defect rate jumps to almost 48 per thousand.

Thats almost five times the threshold (ten defects per thousand lines of code) at which Steve McConnell, author of the exemplary "Code Complete," recommends review to determine if an entire module should be redesigned or reimplemented. Steves reasoning? "Modules with high defect rates are more expensive and time-consuming to deliver than less error-prone modules. Normal modules cost about $500 to $1,000 per function point to develop. Error-prone modules cost about $2,000 to $4,000 per function point to develop...

"If development speed is important, make identification and redesign of error-prone modules a priority... If its poorly structured, excessively complex, or excessively long, redesign the module and reimplement it from the ground up. Youll shorten the schedule and improve the quality of your product at the same time."

Quality isnt just a matter of counting bugs versus time; its a matter of appreciating the high leverage of the worst defects, as carefully studied at CeBASE, the Center for Empirically Based Software Engineering. And this isnt a matter of aesthetics versus the practical realities of getting the code shipped on time; the most attention-getting measurement in a January 2001 article based on CeBASE studies is that "Current software projects spend about 40 to 50 percent of their effort on avoidable rework." Dang. Doesnt that one hit you hard?

I once challenged the mind-set that defects are unavoidable, a viewpoint exemplified by a Microsoft statement that "bugs are inherent in computer science." I got a lot of derisive feedback when I took the opposite position, but Im unrepentant: I find my viewpoint also held by development guru Terence Colligan, in his essay "Nine Steps to Delivering Defect-Free Software," which begins: "Believe Defect-Free Software is Possible."

Colligan continues, "The defect-free engineers...expect their code to have no defects. When a (rare) bug is found, they are very embarrassed and horrified. When they encounter bugs in other peoples code, they are disgusted."

Which is how I feel about typographical errors on Web sites that are built by people whose only job is to do such things correctly. Im sorry if the resulting example seems nasty and unfair.

But Im not very sorry.

Tell me why I should be more forgiving.