Anti-malware Testing Guidelines Important First Step

The Anti-Malware Testing Standards Organization has published a set of best practices and guiding principles for testing security software. Call it a much-needed first step.

The group published two documents on its Web site, setting the foundation for the uniform testing regime the security industry needs. Both documents are the fruits of discussions by more than 40 security experts, product testers and members of the media from across the globe.

The first of the two is titled the “AMTSO Fundamental Principles of Testing” and lays out a number of basic rules, such as requiring testers to validate whether test samples have been correctly classified as malicious and mandating that tests be open and transparent.

The second document is a set of best practices for dynamic testing of host-based security products and stresses, among other things, the importance of keeping logs of what happened during the test. It also offers advice on subjects such as sample collection, measuring results and handling user-product interaction.  

While the documents do not get down and dirty into step-by-step guidelines as to how tests should be conducted, they represent an important first step for an organization with plenty of work to do. Founded in May, the AMTSO was established to respond to concerns that product tests were not keeping up with either technology innovations or the malware the products were designed to fight.

Though its recommendations are voluntary, testing standards are needed to give real meaning to the product reviews circulating around the Internet. Offering direction on subjects such as choosing malware samples can make a big difference.

“From my perspective, I think [one] of the most important things is sample selection,” said Dave Marcus, director of security research for McAfee’s Avert Labs. “What is exactly the sample set that you’re using to test with? That’s a real important question. That probably causes more problems in testing than almost anything else.”

Next: The Issue of Behavioral Detection >>