Is AntiVirus Technology Headed
For Obsolescence?"> I was involved in the closest thing, to my knowledge, to such a comparison. We settled on this as a technique: We installed all the products on identical systems on the same day, updated their definitions, and then saved the system images with Symantec Ghost for later. A month or two later we restored the images to test systems which were disconnected from the Internet, and used them to scan new viruses that had come out since the initial product installation. If the scanners found any of them they would have to do so through heuristics.
The results were awful, and several of the products raised no red flags at all for the 8 viruses we tested. The best performer was McAfee, which raised suspicions on two of the files.
Of course, heuristics are one of those things that people say will get better in the future because of "research." I would argue that the future bodes better for brute-force pattern matching than it does for "intelligent" heuristic scanning because of hardware trends. In the last few years, hardware has gotten drastically faster yet desktop software has not grown in complexity sufficiently to consume the extra CPU power. And were only on the doorstep of the 64-bit, super-parallelism era of CPUs, one which should improve pattern matching even more, far more than they will benefit heuristics.
As if I needed more convincing to have no confidence in heuristic scanning, today I got a fishy email with an attachment. It looked like a virus, except that the attachment made it through my antivirus scanner. Scratch that - it made it through 3 different antivirus scanners, the Norton on my desktop, the Panda at my gateway, and a third one at my ISP (I dont know who the vendor is for that one). Turns out its the new Sobig.E (see iDEFENSEs description of Sobig.E for more details) which is in a new pattern file that came out from Symantec hours later, but I got 5 of these emails before the cavalry came over the hill. This episode shows the real problem with the system of virus definitions: The Internet has allowed attacks to spread so fast that they can outpace the ability of the definition development and distribution system to keep conscientious users up to date. This is a real concern. Of course, this episode also had my heuristic scanners failing me one more time. Security Supersite Editor Larry Seltzer has worked in and written about the computer industry since 1983.
My copy of Norton Antivirus says it protects against 64,201 viruses (6/25/03 definitions). Is there really a problem with protecting against many times that? Last week I argued that antivirus products should also be scanning for the stuff we call spyware and fight, for some reason, with other programs. My copy of Spybot Search and Destroy says it protects against 7454 "problems" although many of them are cookies. I have a spam filtering program that separately scans the same e-mail my antivirus scans. It would actually be more efficient for one program to do all this work.