Microsoft System Prioritizes Tests
As software grows larger, ways to test that software need to keep pace.As software grows larger, ways to test that software need to keep pace. Microsofts Programmer Productivity Research Center has developed a way to use binary comparisons between two versions of a compiled program to infer which software tests should be run on it. "We built a test prioritization system," said Microsoft Distinguished Engineer Amitabh Srivastava. "The idea is very simple. You may have tens of millions of lines of source code, but you dont change them all," Srivastava said. "When every test is run, we put probes into the program and store [the test coverage map] into a repository. ... If you give me two versions of kernel32.dll, I can tell you whats changed. Now I know which parts of the program [the tests] are covering, and I prioritize the tests."
This mapping between changes in software and the most relevant tests to check those changes is now in use in the Windows, SQL Server and Exchange product groups.