Cost of Quality
Cost of quality
So, you want to find lots of bugs, especially critical bugs, and also do so much more cheaply than the alternative: customers and users finding bugs in production. To measure this, use a technique called cost of quality to identify three main costs associated with testing and quality.
1. Cost of detection: the testing costs which you would incur even if you found no bugs. For example, setting up the test environment and creating test data are activities that incur costs of detection.
2. Cost of internal failure: the testing and development costs which you incur purely because you find bugs in prerelease testing. For example, filing bug reports and fixing bugs are activities that incur costs of internal failure.
3. Cost of external failure: the support, testing, development and other costs which you incur because you release systems with some number of bugs (just like everyone else). For example, much of the costs for technical support or help desk organizations and sustaining engineering teams are costs of external failure.
Calculate the average costs of a bug in testing and in production, as explained below:
1. The average cost of a test bug (ACTB) = the cost of detection + cost of internal failure divided by the number of test bugs.
2. The average cost of a production bug (ACPB) = the cost of external failure divided by the number of production bugs.
As I mentioned in a previous Knowledge Center article, the average cost of a bug found during prerelease testing is well below the average cost of a production bug-often by a factor of two, five, ten or more. The bigger the difference, the more optimized your quality assurance efforts are from a financial point of view. In addition, the more expensive it is for your organization to deal with bugs in production, the more it should invest in prerelease testing.
As you've seen in this article, quality need not be an elusive, subjective, unmanageable element in your projects. You can define quality objectives, derive important questions related to these objectives, devise metrics, set quality goals and measure quality progress. Organizations of all sizes-from small startups to large global enterprises-have already taken these steps toward quantitative quality management. You, too, can go beyond gut feel and rabbit's feet to set-and achieve-quality goals for your IT projects.
Rex Black is President of RBCS. Rex is also the immediate past president of the International Software Testing Qualifications Board and the American Software Testing Qualifications Board. Rex has published six books, which have sold over 50,000 copies, including Japanese, Chinese, Indian, Hebrew and Russian editions. Rex has written over thirty articles, presented hundreds of papers, workshops and seminars, and given over fifty speeches at conferences and events around the world. Rex may be reached at firstname.lastname@example.org.