Security Software Reviews Done Wrong
Updated: Consumer Reports review State of the Net survey was supposed be a guide to Protect yourself online, from the biggest threats to the best solutions. Consumer Reports examined spyware, firewalls, anti-virus, Web security, phishing and identity theft, but its methods proved troublesome and vendors such as Symantec are up in arms. Security consumers should be, too. Bad information on security software coming from such a trusted source in security software reviews is doubly bad.I don't do a lot of reviews anymore, but I spent about 13 years in the reviews business, testing a wide variety of products. A badly-done, badly-thought out review hits me like fingernails on the blackboard. So it is with the recent Consumer Reports story on computer security and accompanying review of computer security software (the full story is available only to subscribers). [Editor's Note: Consumer Reports responded to this story after it was published. Please see the end of the story for the full text of their response.]
It's normally bad form to dump on another publication's work in the same field, but this particular story really bothers me. I knew about it before and I must admit that I was spurred into writing this story by a blog posting by Symantec's David Cole complaining about the review. I've had my share of bad experiences with Symantec products so I'm not inclined to give them free PR, but Cole's points are quite valid.