Zero Defects: The Only Acceptable Standard

By Peter Coffee  |  Posted 2002-05-20 Print this article Print

Peter Coffee: Quality isn't just a matter of counting bugs versus time; it's a matter of appreciating the high leverage of the worst defects.

Im going to begin this letter with a nasty, nitpicky example thats definitely a case of throwing stones from inside a glass house. I know that the sin Im about to chastise is one that we sometimes commit ourselves at eWEEK, but thats just too bad: The example Ill now cite of someone elses carelessness is the one that happened to be at hand when I needed it. I was recently researching the product line road map for the newly merged Hewlett-Packard and Compaq, about which Ill share some critical comments in my eWEEK column a week from today. On my way to that Web page, however, I passed through the "Merger Information" portal page on the HP site, where I encountered the promise of "Infrastucture Solutions." (Perhaps the spelling error has been fixed by now, but I have a screen photo if I ever need to prove that I saw it.) I dont know what it feels like to be infrastuck, but theyll have to pay me to try it.
OK, OK, I know that this is a nasty piece of nitpicking, but theres a reason why I take up your time by mentioning it. That entire HP Web page only contains 172 words of text, and one of them—a stand-alone hyperlink label, right out there in the open—is wrong. Thats a defect rate of almost six per thousand, and thats the most charitable measure: If we choose paragraphs as our units of work, the defect rate jumps to almost 48 per thousand.
Thats almost five times the threshold (ten defects per thousand lines of code) at which Steve McConnell, author of the exemplary "Code Complete," recommends review to determine if an entire module should be redesigned or reimplemented. Steves reasoning? "Modules with high defect rates are more expensive and time-consuming to deliver than less error-prone modules. Normal modules cost about $500 to $1,000 per function point to develop. Error-prone modules cost about $2,000 to $4,000 per function point to develop... "If development speed is important, make identification and redesign of error-prone modules a priority... If its poorly structured, excessively complex, or excessively long, redesign the module and reimplement it from the ground up. Youll shorten the schedule and improve the quality of your product at the same time." Quality isnt just a matter of counting bugs versus time; its a matter of appreciating the high leverage of the worst defects, as carefully studied at CeBASE, the Center for Empirically Based Software Engineering. And this isnt a matter of aesthetics versus the practical realities of getting the code shipped on time; the most attention-getting measurement in a January 2001 article based on CeBASE studies is that "Current software projects spend about 40 to 50 percent of their effort on avoidable rework." Dang. Doesnt that one hit you hard? I once challenged the mind-set that defects are unavoidable, a viewpoint exemplified by a Microsoft statement that "bugs are inherent in computer science." I got a lot of derisive feedback when I took the opposite position, but Im unrepentant: I find my viewpoint also held by development guru Terence Colligan, in his essay "Nine Steps to Delivering Defect-Free Software," which begins: "Believe Defect-Free Software is Possible." Colligan continues, "The defect-free engineers...expect their code to have no defects. When a (rare) bug is found, they are very embarrassed and horrified. When they encounter bugs in other peoples code, they are disgusted." Which is how I feel about typographical errors on Web sites that are built by people whose only job is to do such things correctly. Im sorry if the resulting example seems nasty and unfair. But Im not very sorry. Tell me why I should be more forgiving.
Peter Coffee is Director of Platform Research at, where he serves as a liaison with the developer community to define the opportunity and clarify developers' technical requirements on the company's evolving Apex Platform. Peter previously spent 18 years with eWEEK (formerly PC Week), the national news magazine of enterprise technology practice, where he reviewed software development tools and methods and wrote regular columns on emerging technologies and professional community issues.Before he began writing full-time in 1989, Peter spent eleven years in technical and management positions at Exxon and The Aerospace Corporation, including management of the latter company's first desktop computing planning team and applied research in applications of artificial intelligence techniques. He holds an engineering degree from MIT and an MBA from Pepperdine University, he has held teaching appointments in computer science, business analytics and information systems management at Pepperdine, UCLA, and Chapman College.

Submit a Comment

Loading Comments...
Manage your Newsletters: Login   Register My Newsletters

Rocket Fuel