eWEEK content and product recommendations are editorially independent. We may make money when you click on links to our partners. Learn More.
Whenever I’ve done anti-virus or anti-spyware testing in the past, getting live samples has always been something of a challenge. I’ve made several attempts to archive malware for testing purposes, but often found the samples to be antiquated when the time comes to do a test. This makes it difficult to extrapolate differences between products being tested, as signatures were created long ago for my malware.
Similarly, Web sites that I have counted on in the past as a guaranteed source of malware have come and gone, making it a drag to find new, active strains on the Web on short notice. And Google’s newfound commitment to vetting links can also get in the way. Even some of the most traditionally pernicious adware vendors have made strides to clean up their acts.
This time, I tried a more cutting-edge approach to collecting malware for use in eWEEK Labs’ tests. Cameron Sturdevant recently met with a company called Robot Genius, which operates a specialized malware-seeking Web crawler called RGcrawler. RGcrawler seeks out every Windows executable file, ActiveX control and .zip file on the Web, then downloads the code to Robot Genius’ network for automated installation and testing. From these tests, Robot Genius can identify known malware or suspicious behavior to positively ID real threats available on the Web.
Robot Genius gave us a list of links to almost 300 malware-infected executables. Looking for a wide diversity of malware, I culled that list down to 35 samples by eliminating many similar entries that originated at the same Web site. I looked for a combination of known threats that Robot Genius could classify plus unknown threats identified as malware by their behavior.
Trying to ensure that we had legitimate malware, I took those 35 samples and ran them through the scanners at VirusTotal, which checks user-submitted malware against 31 different signature engines to identify possible threats. Of the 35 samples, six came back with a clean bill of health from VirusTotal. While this is not conclusive proof that these six samples were benign, I nonetheless eliminated those six from consideration in our tests to avoid the chance of holding a false positive from Robot Genius against the products tested.
This left me with a total of 29 vetted malware samples. These files represented a mixture of adware, spyware, worms and Trojans with files sizes ranging from only 27KB to over 9.5MB.
To put the anti-malware solutions through their paces, I focused my testing on detection of new threats, rather than elimination of existing ones. To test the real-time detection, I copied the samples to the local hard drive through a variety of means — from a Windows file share, from a USB drive and via VMware’s drag-and-drop copy between virtual host and client.
For the samples that survived the real-time scan, I conducted a full disk scan to attempt to find inert but present-on-the-disk malware, like when a bundle stays resident in the browser cache even if the original file is long gone.
For samples that made it through rounds 1 and 2, I then attempted to install each of the remaining samples onto the protected system, to see whether the anti-malware solution would catch on as the bundle was exploded out or as new content was downloaded from the Web during installation, and to see whether malicious behavior (like a process injection or write to the local host file) triggered an alert. I installed each threat individually on a fresh virtual instance, trying to ensure that any detections were specifically related to the particular sample being tested.
Note: I had some problems with Microsoft Forefront Client Security and VMware Workstation 5.5 right from the start. As I reported in the review, Forefront’s real-time detection did not work correctly on a virtualized Windows XP SP2 client instance. I found that I could copy malware to the Forefront-protected instance willy-nilly — from a file share, over the Web or from a thumb drive — and Forefront acted like nothing was wrong until I ran a full disk scan.
To counter this, I performed tests 1 and 2 on a Forefront-protected laptop computer. However, I still performed test 3 on the VMware-based instance to ease the process of resetting the system after each infection attempt. Spot checks of the installation of a couple strains on the laptop revealed identical behavior in both instances.