Opinion: Some people treat anti-virus protection as a commodity these days, but it's getting harder to do, not easier. Is wild list testing enough anymore?
Comparative testing of anti-virus products is complicated, and very few people outside of the vendors do it.
The most popular measure of performance is a list of viruses called the Wild List
, and some popular tests and certifications are based on it. But, unlike viruses, the wild list updates very slowly. In fact, everything in it is at least one month old, probably much older.
You have to have harder tests than that to have real confidence in anti-virus protection, although controversy ensues when you do. In the early hours, even days, of a virus outbreak, researchers are often in disagreement about the nature of the attack. They disagree about the number of variants, whether an attack is new or a variant of an old attack, and what mitigating circumstances may apply.
David Coursey writes an open letter to virus writers. Read it here.
This early time is the most important and dangerous time in a viruss life, when it gets a foothold in the wild, not a month from then. For this reason, the larger security companies have a genuine advantage in their ability to respond to new threats. When I see malware protection from little companies or even looser affiliations, I dont get a warm fuzzy about their ability to respond quickly.
A recent article by a Kaspersky researcher
expresses some of the testing concerns well. In some tests of these products for PC Magazine in which I have participated we have worked with AV-Test.org
, a German research firm with its finger on the pulse of the virus threat and the industry response to it. I cant get into detail about the research results because they charge for it, but I can guarantee you that efforts like ClamAV
arent typically among the first responders to new threats.
The big threeMcAfee, Symantec and Trend Microarent always perfect in this. I specifically remember Mydoom.A as a low point for Symantec. But these companies all have multiple research teams around the world and the ability to respond quickly.
What about smaller anti-virus companies?