Hardening Expectations for Software Quality

Opinion: Throw cheap CPU cycles, not costly coder attention, at targets of rising complexity.

A hardware manifestation of a software bug, for example the Pentium FDIV error of 1994, is so rare that people can still remember that specific incident more than 10 years ago. In contrast, imagine saying "that Windows bug in 1994" (or, for that matter, "that Unix bug in 1994") and having anyone know exactly what you meant.

But complex semiconductors "are basically all software until you fabricate them," observed Susan Marie Kunz, founder and CEO of Solidware Technologies in Boulder, Colo., during a conversation we had last week. It seems odd, she said, that semiconductor quality stays abreast of rising complexity while enterprise platform and application software struggles to keep up.

Closing that gap is the goal of Solidwares debut product, Splat, introduced today and available for trial download from the companys site at www.solidw.com. The companys vision for the product, Kunz said, is to elevate software quality with the same strategies of aggressive automation, task prioritization and cost-effective risk reduction that she and her colleagues bring from their prior experience in semiconductors.

Growth of complexity and scale, Kunz said, "is emerging on the software side with open source and other developments that leave you not quite sure of code quality; modules increasingly arent designed initially to work together, so use cases arent comprehensive," she observed.

The problem of software complexity is compounded, Kunz added (and I emphatically agree), by the industrys rapid uptake of multithreaded hardware platforms and software strategies. "When you introduce multithreading, a lot of code will break," she said. "You dont have to make that move, but all the new CPU architectures … if you dont use it, performance will definitely be impacted. The companies that make their code work well in a multithreaded environment will definitely have an advantage over those that dont -- but its definitely a risk, from a defect point of view."

A concurrent growth of complexity, she said, comes with the move from 32- to 64-bit systems. One application, she said, went from less than a CPU-year to more than 500 CPU-years of testing required to achieve comparable coverage.

Too many software testing tools, Kunz said, fail to live up to their potential because its up to coders to keep track of the opportunities to use them. A tool may automate the performance of the test itself, but identification of when and what to test is no simple thing.

"Theres no reason why a person should have to say, Oh, I messed with memory, I have to run Purify. If I know you did that, and I know you have Purify, that should kick off automatically," Kunz urged. "We ask customers, What would make your life easier? They so often say, Just tell me when I touch my code, what could I have broken?"

I absolutely agree with that goal, and I hope that readers of this column will share with me their assessments of how close Solidwares Splat comes to achieving it -- and what else theyre using toward that end.

Tell me what youre afraid of breaking at peter_coffee@ziffdavis.com.