IT administrators affected by the flawed antivirus update pushed out April 21 by McAfee may have a lot of work ahead of them cleaning up computers. But there are some lessons to be learned from the fiasco that admins should file away for the future, experts told eWEEK.
IT operations for a number of organizations were disrupted when McAfee pushed out a .dat file update for its antivirus products. The buggy update, which was meant to address an attack targeting Windows PCs, incorrectly identified svchost.exe as a Windows virus detected as W32/wecorl.a, and affected computers running Windows XP Service Pack 3.
According to McAfee, .5 percent of its corporate customers were impacted, and an even smaller percentage of customers of its consumer products were hit as well.
But among those enterprise users were police, hospitals and universities whose XP computers downloaded the file and were greeted with a Blue Screen or DCOM error, followed by shutdown messages.
For organizations, the moral here may be to take a more cautious approach to applying antivirus updates. But doing so can open up its own can of worms, noted Jason Miller, data and security team manager at Shavlik Technologies.
"Deploying AV updates can be quite challenging given that antivirus definition files are extremely time-sensitive," Miller said. "The longer IT administrators wait to deploy new definition files, the more likely user machines are vulnerable to the last viruses. In addition, antivirus vendors typically release multiple definition files daily. A good practice is to test the new definition files as they become available before rolling them out to the network, but this may not be feasible for all companies due to the frequency of the updates and testing time."
Many organizations will first bring updated signature files, AV updates and patches into a safe sandbox for internal testing and validation, said Peter Schlampp, vice president of marketing and product management at Solera Networks.
"This sandbox would represent the general makeup of the overall organization-various operating systems, various versions, hardware [configurations], etc.," Schlampp said. "By exposing a select number of machines to the updates, they get an idea of how the systems will respond to the updates. Only after this internal validation will they then coordinate the updates throughout the entire organization."
The incident will make many think twice about moving to a "cloud-based" update system in which PCs just accept an automatic push of updates from AV vendors without any gating by enterprise IT, opined Gartner analyst John Pescatore.
While Pescatore characterized McAfee's response as quick and properly apologetic, he compared the original mistake to an aspirin company providing pills that end up causing headaches.
"This one incident doesn't really mean we need to go back to what was common five years ago, i.e. fully QA-ing [quality assurance testing] every signature update before pushing out," Pescatore said. "However, waiting a few hours to see if anyone else reports problems before you push out, or at least simple QA on the standard supported corporate image through a reboot, is still a prudent approach."
In an additional twist, ESET reports that attackers have begun poisoning search engine results to lead unsuspecting victims looking for information about the situation to sites pushing rogue antivirus software.
"Juraj Malcho, our head of lab in Bratislava, reports that in a Google search ... he got three malicious hits in the top 10 ... and 11 in the top 20," blogged David Harley, ESET's director of malware intelligence. "Subsequent searches using different search strings are finding even more hits, so right now, Google is well and truly poisoned."
McAfee published a tool earlier April 22 that suppresses the driver causing the false positive by applying an Extra.dat file and restoring the svchost.exe. Organizations looking for information about workarounds and fixes can find it here.