Microsoft System Prioritizes Tests

By Timothy Dyck  |  Posted 2002-11-25 Print this article Print

As software grows larger, ways to test that software need to keep pace.

As software grows larger, ways to test that software need to keep pace. Microsofts Programmer Productivity Research Center has developed a way to use binary comparisons between two versions of a compiled program to infer which software tests should be run on it.

"We built a test prioritization system," said Microsoft Distinguished Engineer Amitabh Srivastava. "The idea is very simple. You may have tens of millions of lines of source code, but you dont change them all," Srivastava said. "When every test is run, we put probes into the program and store [the test coverage map] into a repository. ... If you give me two versions of kernel32.dll, I can tell you whats changed. Now I know which parts of the program [the tests] are covering, and I prioritize the tests."

This mapping between changes in software and the most relevant tests to check those changes is now in use in the Windows, SQL Server and Exchange product groups.

In just a few minutes, developers can run a small number of tests that are most likely to find any bugs the developers might have introduced.

"We are now catching the defects on the developers desktop without them entering the system," said Srivastava. "If we can stop the defect from entering the system, its very cheap."

Srivastava and fellow researcher Jay Thiagarajan presented a paper on their technique in July at the International Symposium on Software Testing and Analysis in Rome.

Timothy Dyck is a Senior Analyst with eWEEK Labs. He has been testing and reviewing application server, database and middleware products and technologies for eWEEK since 1996. Prior to joining eWEEK, he worked at the LAN and WAN network operations center for a large telecommunications firm, in operating systems and development tools technical marketing for a large software company and in the IT department at a government agency. He has an honors bachelors degree of mathematics in computer science from the University of Waterloo in Waterloo, Ontario, Canada, and a masters of arts degree in journalism from the University of Western Ontario in London, Ontario, Canada.

Submit a Comment

Loading Comments...
Manage your Newsletters: Login   Register My Newsletters

Rocket Fuel