As I work on a couple of upcoming reviews of anti-spam products for PC Magazine Ive been struggling to define my own standards for anti-spam products. In my previous reviews of such products Ive always stated not only that the number of false positives was the most important measure of the product, but that the products should be held to a standard of near-perfection.
Heres my reasoning: If you assume that the product will generate a non-trivial number of false positives—by which I mean legitimate, non-spam messages mistakenly classified as spam by the product—then you must check the products blocked mail folder periodically in order to separate out the real mail in there. Ive always thought that if you have to look through all the spam anyway, you may as well look at it in your inbox. This is the reason that after all the reviews Ive written, I still dont run an anti-spam product myself.
However, I think Ive finally lost my patience with this approach. Things have gotten so bad lately that Im willing to take some risks. I havent changed my opinion that false positives are the most important measure of an anti-spam product and that all anti-spam products will generate some of them. At the same time, if you have to check the blocked mail folder, you may as well not run the product. Thats why I think I wont bother checking the blocked mail.
Some time soon, Ill start running one of these programs and Ill just put my faith in it. Thats the answer: you really have to have faith in order to run an anti-spam product sensibly. Im going to assume that some amount of legitimate mail, perhaps from you, dear readers, will not make it through.
Perhaps its a mark of the times that this situation doesnt bother me in the same way it used to. Heres my reasoning (or rationalization): Even if I dont run an anti-spam product some amount of real mail will not make it through. Im sure Ive mistakenly deleted many a real message in the process of manually deleting spam.
In addition, not all false positives are created equal—many legitimate messages will not be missed. Some of them are commercial solicitations to which I once subscribed. Some of them are jokes sent by friends and relatives. Some may be real product announcements from vendors or public relations firms; this is a big part of the risk Ill run.
When I talk to people about the anti-spam products theyre running and many of them say they are very happy with them. Often Ill hear them say, simultaneously, that the product generates no false positives and that they dont have to bother checking the blocked mail folder.
But make no mistake about it: If youre not checking the blocked mail folder you dont know how many false positives the product is generating. How could you? This is also at the heart of what makes anti-spam products so difficult to test. In an upcoming column Ill discuss some of the issues that it the most challenging category of software to test.
Again, it all comes down to a matter of faith. If youre not going to double-check the results of your anti-spam software youre putting yourself at the mercy of the product. By the same token, you have to be suspicious of vendor claims for the accuracy of their products. Spam blocking is becoming a mandatory software task, but its still a problematic one.
Security Supersite Editor Larry Seltzer has worked in and written about the computer industry since 1983.
More from Larry Seltzer
