Time for the Anti-virus Business to Talk Testing

 
 
By Larry Seltzer  |  Posted 2006-03-13 Print this article Print
 
 
 
 
 
 
 

Opinion: When you can't trust your anti-virus products, you're in a fundamentally bad position. McAfee's got some explaining to do, but don't assume they're uniquely incompetent.

I have to say Im glad I dont use McAfees anti-virus products on any of my desktops. My weekend, like so many of yours, would have been ruined. I do use it on my gateway protection box and havent had any problems that I know of. Why do these things always seem to happen over the weekend? In an era when many anti-virus companies release updates many times a day, perhaps every hour, its easy to dismiss the importance of testing. These companies are in a tough spot: Their mandate to provide coverage as soon as possible runs directly in conflict with their need to test their products before releasing them to the public.

It does seem to me as if effective testing has become impossible for such products, but when the situation gets as bad as it got with McAfee its time to lose sympathy for the vendor. Development mistakes are inevitable, and thats why you have to test any product, even anti-virus definitions, that goes out to the public. Obviously, McAfee didnt test these definitions, at least not very carefully.

A faulty virus definition update from McAfee erroneously flags several widely used software programs as a virus outbreak. Click here to read more.
Its sort of unfair for me to pick on another vendor at this point, but Kaspersky updates its definitions hourly. How can it possibly test them? Symantec, on the other hand, has a terrible reputation for slow updates. It only recently moved to a daily schedule, and only for its newest products. Like a few other vendors, Symantec issues beta definitions; in fact, theyre available to the public for download (the company calls them "Rapid Release Virus Definitions").

One would expect this process to improve the quality of Symantecs products, and I bet it does. And yet even Symantec isnt immune to problems. The company has had several incidents of security vulnerabilities in its own products, but nothing as serious as the false positives McAfee experienced.

Testing of this sort is not easy to do. Note that this specific error is present only with the on-demand scanner, not the on-access scanner. So if all it had done was an on-demand scan of several typical Windows systems, it would have found this problem.

But to find problems in the on-access scanner, it would probably have to test by running all the programs thoroughly enough so that all the files get used. Just in case you think this is easy or quick I can assure you, based on a lot of testing experience, that it is neither. And the longer it takes, the longer it keeps protection from potentially serious threats from customers.

As a general matter, tests like this are parallelizable, which means you can get better throughput by splitting up parts of them among several computers. So you can get more thorough testing by throwing more test development, equipment and administration money at the problem. But my guess is that no anti-virus company is going to be willing to wait as long as the testing people want them to.

Ziff Davis Media eSeminars invite: Learn how to proactively shield your organizations against threats at all tiers of the network, Symantec will show you how, live on March 21 at 4 p.m. ET. Sponsored by Symantec. Imagine a serious threat situation. Word comes out in the back channels that all major anti-virus researchers share that theyve got samples of a threat that could spread quickly and do great damage. The samples are shared, the vendors generate signatures, some of them release them, and others test. Im as guilty as anyone of criticizing vendors for being slow to release updates, but we need to take this episode as a warning that its possible to release them too quickly too. No question about it, McAfee has a lot to explain and a lot to demonstrate now about its quality control if it wants customers to feel good about trusting their computers to the companys protection. But we shouldnt stop there. Before we go overboard emphasizing the speed of attacks and the necessary speed of protection, lets start hearing from vendors about how they protect us from their own mistakes. This wont be the last time.

Security Center Editor Larry Seltzer has worked in and written about the computer industry since 1983. Check out eWEEK.coms for the latest security news, reviews and analysis. And for insights on security coverage around the Web, take a look at eWEEK.com Security Center Editor Larry Seltzers Weblog. More from Larry Seltzer
 
 
 
 
Larry Seltzer has been writing software for and English about computers ever since—,much to his own amazement—,he graduated from the University of Pennsylvania in 1983.

He was one of the authors of NPL and NPL-R, fourth-generation languages for microcomputers by the now-defunct DeskTop Software Corporation. (Larry is sad to find absolutely no hits on any of these +products on Google.) His work at Desktop Software included programming the UCSD p-System, a virtual machine-based operating system with portable binaries that pre-dated Java by more than 10 years.

For several years, he wrote corporate software for Mathematica Policy Research (they're still in business!) and Chase Econometrics (not so lucky) before being forcibly thrown into the consulting market. He bummed around the Philadelphia consulting and contract-programming scenes for a year or two before taking a job at NSTL (National Software Testing Labs) developing product tests and managing contract testing for the computer industry, governments and publication.

In 1991 Larry moved to Massachusetts to become Technical Director of PC Week Labs (now eWeek Labs). He moved within Ziff Davis to New York in 1994 to run testing at Windows Sources. In 1995, he became Technical Director for Internet product testing at PC Magazine and stayed there till 1998.

Since then, he has been writing for numerous other publications, including Fortune Small Business, Windows 2000 Magazine (now Windows and .NET Magazine), ZDNet and Sam Whitmore's Media Survey.
 
 
 
 
 
 
 

Submit a Comment

Loading Comments...
 
Manage your Newsletters: Login   Register My Newsletters























 
 
 
 
 
 
 
 
 
 
 
Rocket Fuel