Time for the Anti-virus Business to Talk Testing

Opinion: When you can't trust your anti-virus products, you're in a fundamentally bad position. McAfee's got some explaining to do, but don't assume they're uniquely incompetent.

I have to say Im glad I dont use McAfees anti-virus products on any of my desktops. My weekend, like so many of yours, would have been ruined. I do use it on my gateway protection box and havent had any problems that I know of. Why do these things always seem to happen over the weekend?

In an era when many anti-virus companies release updates many times a day, perhaps every hour, its easy to dismiss the importance of testing. These companies are in a tough spot: Their mandate to provide coverage as soon as possible runs directly in conflict with their need to test their products before releasing them to the public.

It does seem to me as if effective testing has become impossible for such products, but when the situation gets as bad as it got with McAfee its time to lose sympathy for the vendor. Development mistakes are inevitable, and thats why you have to test any product, even anti-virus definitions, that goes out to the public. Obviously, McAfee didnt test these definitions, at least not very carefully.

A faulty virus definition update from McAfee erroneously flags several widely used software programs as a virus outbreak. Click here to read more.

Its sort of unfair for me to pick on another vendor at this point, but Kaspersky updates its definitions hourly. How can it possibly test them? Symantec, on the other hand, has a terrible reputation for slow updates. It only recently moved to a daily schedule, and only for its newest products. Like a few other vendors, Symantec issues beta definitions; in fact, theyre available to the public for download (the company calls them "Rapid Release Virus Definitions").

Resource Library:

One would expect this process to improve the quality of Symantecs products, and I bet it does. And yet even Symantec isnt immune to problems. The company has had several incidents of security vulnerabilities in its own products, but nothing as serious as the false positives McAfee experienced.

Testing of this sort is not easy to do. Note that this specific error is present only with the on-demand scanner, not the on-access scanner. So if all it had done was an on-demand scan of several typical Windows systems, it would have found this problem.

But to find problems in the on-access scanner, it would probably have to test by running all the programs thoroughly enough so that all the files get used. Just in case you think this is easy or quick I can assure you, based on a lot of testing experience, that it is neither. And the longer it takes, the longer it keeps protection from potentially serious threats from customers.

As a general matter, tests like this are parallelizable, which means you can get better throughput by splitting up parts of them among several computers. So you can get more thorough testing by throwing more test development, equipment and administration money at the problem. But my guess is that no anti-virus company is going to be willing to wait as long as the testing people want them to.

Ziff Davis Media eSeminars invite: Learn how to proactively shield your organizations against threats at all tiers of the network, Symantec will show you how, live on March 21 at 4 p.m. ET. Sponsored by Symantec.

Imagine a serious threat situation. Word comes out in the back channels that all major anti-virus researchers share that theyve got samples of a threat that could spread quickly and do great damage. The samples are shared, the vendors generate signatures, some of them release them, and others test. Im as guilty as anyone of criticizing vendors for being slow to release updates, but we need to take this episode as a warning that its possible to release them too quickly too.

No question about it, McAfee has a lot to explain and a lot to demonstrate now about its quality control if it wants customers to feel good about trusting their computers to the companys protection. But we shouldnt stop there. Before we go overboard emphasizing the speed of attacks and the necessary speed of protection, lets start hearing from vendors about how they protect us from their own mistakes. This wont be the last time.

Security Center Editor Larry Seltzer has worked in and written about the computer industry since 1983.

Check out eWEEK.coms for the latest security news, reviews and analysis. And for insights on security coverage around the Web, take a look at eWEEK.com Security Center Editor Larry Seltzers Weblog.

More from Larry Seltzer

Reader Comments: Time for the Anti-virus Business to Talk Testing >>> Be the FIRST to comment on this article!

>>> More Network Security & Hardware Articles          >>> More By Larry Seltzer