Among the vendors failing to receive VB100 certification are McAfee, Trend Micro, Sophos, Webroot, Norman, BitDefender and PC Tools.Some of the biggest names in the anti-virus industry have flunked detection
tests for known malware samples on Windows Vista Service Pack 1.
Seventeen of 37 anti-malware products pitted against "in the wild"
viruses on the latest version of Vista failed to obtain VB100 certification, an
industry benchmark used to rate product quality, according to test results
released by Virus Bulletin.
Among the software products failing to nab VB100 certification were McAfee
VirusScan Enterprise, Trend Micro Internet Security, Norman Virus Control,
Sophos Anti-Virus, Webroot Spy Sweeper with AntiVirus, Alwil Software's Avast,
BitDefender Antivirus 2008, PC Tools AntiVirus and VirusBuster Professional.
To gain VB100 from Virus Bulletin's testers, a product must detect 100 percent
of malicious Trojans, bots and viruses from a batch of "in the wild"
samples maintained in the WildList Organization International's database. The WildList contains a listing of
viruses collected and reported by virus hunters during actual computer attacks.
The basic requirements for a VB100 passing grade are that a product detect,
both on demand and on access, in its default settings, all malware known to be in
the wild at the time of the review, and generate no false positives when
scanning a set of clean files.
For live coverage of this
year's RSA Conference, click here.
However, as the results show, several brand-name
anti-malware labs are still missing virus samples linked to known attacks.
In McAfee's case, for example, the company's VirusScan Enterprise 8.5.0i was
described as simple and dependable with solid integration of Windows Vista's
UAC (User Account Control) feature. Virus Bulletin's John Hawes said the
product's detection rates were "dependably excellent" during the
tests until a single sample of the W32/Virut strain reared its ugly head.
Since that sample was in the WildList set, that was enough to deny McAfee a
VB100 award, Hawes explained.
Trend Micro Internet Security, a three-user anti-malware product that retails
for $49.99, also scored well on some detections but Hawes said some false
positives led to the failing grade. "A small number of file infectors were
missed in the WildList set and a couple of items in the clean set were labeled
as 'TROJ_Generic.' As a result, Trend does not qualify for the VB100 award on
this occasion," he said.
Webroot Spy Sweeper with AntiVirus shares signatures with Sophos Anti-Virus; both
failed because some samples of the tricky Virut variants were not detected.
While these results are a public relations embarrassment for the bigger
anti-virus vendors, analysts say the results should be taken with a grain of
salt.
"[T]here are a couple asterisks worth noting," Paul Roberts,
senior analyst in The 451 Group's enterprise security research unit, wrote in a
research note. "First of all: The platform in question—Vista SP1—was
released shortly after the deadline for product submissions
to VB. VB reviewer John Hawes … is up front about that fact that not every
anti-malware vendor was even able to get a copy of SP1 for testing before
submitting their wares to VB for certification."
Roberts added, "Certifications like VB100, which are based largely on
static file analysis, have gone a long way towards sustaining the signature-based
detection model when others might serve consumers and enterprises better."
He said most anti-virus companies already do blend behavior and signature-based
detection methods, but warned that companies that rely heavily on the former,
like BitDefender, tend to do worse on tests like the VB100.
"Does that mean BitDefender provides inferior protection to a company
like, say Kingsoft, which did receive the award? Hardly, but the lack of
certification still becomes a hook on which to hang competitive claims. Bottom
line: You get punished for not using signatures, even if that's the right or
most effective thing to do," Roberts said.
Roberts called for new testing methods to help "end the illusion of
competence that current testing models perpetuate" and raise the bar for
malware detection among established vendors.
