eWEEK content and product recommendations are editorially independent. We may make money when you click on links to our partners. Learn More.
The anti-spyware business has a chip on its shoulder and its hard to blame them, especially with people like me arguing that their whole category of product is, or should be, superfluous.
Lately there has been some controversy over testing practices for spyware. Shortly after the controversy over Consumer Reports anti-virus tests peaked, Alex Eckelberry of Sunbelt Software showed that in their anti-spyware tests Consumer Reports didnt actually test against any spyware. They used a simulator named Spycar to perform suspicious behaviors.
This is philosophically in line, to a degree, with their use of specially written malware for the anti-virus tests. I do think its an interesting test and would be useful if it were part of a more comprehensive test that included tests of actual spyware.
The same could be said for their fake virus testing: It would be much more interesting if they had tested against actual viruses as well, although, as Eckelberry points out in the blog entry above, Consumer Reports also misused Spycar in certain ways.
Ill get a little defensive myself for a moment and disagree with Alexs implication that I was “fighting back” with Consumer Reports. I still havent actually read their tests, although Im trying to get my hands on a copy without having to subscribe.
I havent defended their actual tests and thought I made it clear that I didnt assume they did a good job. I only defended the idea of experimenting with lab-created viruses as a way of testing heuristic functionality.
As Dr. Solomon has pointed out in the same thread to which Eckelberry refers, we dont even know if the files that Consumer Reports generated are actual viruses because they havent shared them with anyone. Im just assuming that they were for the sake of argument. And since testing heuristic protection is a difficult thing to do, Consumer Reports approach seems like an experiment worth making.
I will agree wholeheartedly with Eckelberry that any good test of malware detection should include tests against actual real-world malware, at least in cases where the tests purport to show the effectiveness of the products in blocking it.
In years past I have been involved with tests where we had trouble obtaining a library of malware against which to test, but this isnt as hard as it used to be. There are many such repositories now and Ive had offers from researchers of access in case I needed it. Im sure Consumer Reports could have had similar access if they looked for it.
And in early August, presaging all this, Eric Howes (also of Sunbelt) let out a lecture on “The Wretched State of Anti-Spyware Testing.”
Make sure to read at least the top two messages, and Howes other messages are also interesting. Its really a heck of a set of guidelines to testing malware, and Im saving a copy for the next time Im involved in a test.
Notice that I just referred to “malware” and not “spyware.” I immediately noticed in Howes rules that there is nothing peculiar in them to spyware. They refer just as well to other forms of malware.
And this brings me back to my whole problem with a separate anti-spyware business. The jobs of anti-virus software and anti-spyware software are so similar that the same testing procedures apply to both.
Its worth quoting Eckelberrys blog entry at this point:
b) Remove spyware.
c) Block new spyware, hopefully before it infects your system.
I could say the same thing for anti-virus software. Why should there be two programs performing these tasks for different categories of malware, the distinctions between which are of interest only to academics?
Even the terms are anachronistic: Most of the programs blocked by anti-virus software arent viruses; these days they are trojan horses and worms. And to the best of my knowledge (correct me if Im wrong Alex) most of the programs blocked by anti-spyware these days are adware, not spyware.
Theyre both useful, and the major anti-virus programs are trying to subsume anti-spyware functionality, although I cant say how well theyre doing it.
Perhaps the anti-spyware guys should take the same approach from the other side. Id trust anti-virus protection from Sunbelt Software. Why shouldnt I when theyve been doing essentially the same task on different threats for years?
Incidentally, they do sell server-based protection that includes anti-virus; Ive been using their Ninja product on my Exchange 2003 Server for anti-spam, and it also scans e-mails with (optionally) both Authentium and BitDefender.
I havent seen anything resembling a virus get through these defenses. Some time soon I hope to write more about my experiences with Ninja and other work Ive done on my own server, but Im really happy with Ninja.
The anti-virus business hasnt made a lot of friends lately, but at least its still competitive. That tells me that theres even more room for competition, and perhaps were all better off with the anti-spyware guys running the show.
Security Center Editor Larry Seltzer has worked in and written about the computer industry since 1983.