News Analysis: Windows Vista only had 12 vulnerabilities in its first six months, making Linux distros look buggy by comparison, but analysts aren't convinced.
According to the numbers given in a new report from Microsoft, Windows Vista has blown away all the major enterprise Linux distributions and Mac OS X as far as having the smallest amount of serious security vulnerabilities in the six months since its release. The numbers were compiled by Jeff Jones, the security strategy director in Microsofts Trustworthy Computing Group
"The results of the analysis show that Windows Vista continues to show a trend of fewer total and fewer High severity vulnerabilities at the 6-month mark compared to its predecessor product Windows XP (which did not benefit from the SDL [Secure Development Lifecycle] and compared to other modern competitive workstation OSes (which also did not benefit from an SDL-like process)," Jones wrote in a blog posting
about the report on June 21.
In the report, available as a PDF download on Jones blog, Jones compares the number of vulnerabilities of critical, medium and low severity that have been discovered in Vista with those found in Windows XP, Red Hat Enterprise Linux 4 Workstation, Ubuntu 6.06 LTS, Ubuntu 6.06 LTSReduced Component Set, Novell SUSE Linux Enterprise Desktop 10.8, Novell SLED 10Reduced Component Set and Apple Mac OS X v10.4.
The score, according to Jones: In the first six months of the Vista life cycle, Microsoft has released four major security bulletins that address 12 total vulnerabilities affecting Windows Vista.
In comparison, the most popular Linux distribution, Red Hat Enterprise Linux 4 Workstation, was swamped with 129 publicly disclosed bugs in shipping components, 40 of them "High Severity." During the first six months, Red Hat fixed a total of 281 vulnerabilities in RHEL4 Workstation. Eighty-six of those fixed were rated "High Severity" by the NIST (National Institute of Standards and Technology) in the NVD (National Vulnerability Database).
By Jones count, Vista seems to be a nigh-impregnable fortress. But counting vulnerabilities is not the best metric, say analysts and Microsoft observers.
"I get nervous about counts," said Michael Cherry, an analyst with Directions on Microsoft. "If we get obsessed about vulnerability counts we almost put pressure on them to manipulate the count. To not report things. I wish we had a better metric than counting."
At any rate, vulnerability counts are somewhat subjective, Cherry pointed out. "Lets say youre working on a module of code. You go in to fix problem A and while youre fixing problem A you find problem B. Do you count those as two problems or one? I can make a case for it being counted either way," he said.
Besides, its hard to base a trend on a six-month security assessment, Cherry said. Most operating systems have a 10-year life cycle, and so far Vista has had limited deployment.
It could also be that there are more operating system guardians for Linux distros and Mac OS X, argued Joe Wilcox, editor of Microsoft Watch.
More cops on the beat means that more criminals get caught.
When asked if more vulnerabilities could mean more thorough code inspection, Austin Wilson, director of Windows Client Security Product Management for Microsoft, based in Redmond, Wash., demurred in addressing the possibility. "I cant speak for Linux distributions; its a good question to ask them," he said. "Im certainly happy to talk about Vista."
Microsofts Jones admitted that many think its unfair to count the vulnerabilities for all of the components for the product that Red Hat ships and supports as Red Hat Enterprise Linux 4 WS; hence, he inspected both full-component versions of the Linux distributions as well as stripped-down builds. "To accommodate that idea, I will additionally analyze a reduced set of RHEL4WS components that deliver functionality comparable to Windows XP and exclude other optional components," he said.
"Linux distribution vendors add value to their workstation distributions by including and supporting many applications that dont have a comparable component on a Microsoft Windows operating system," he continued. "It is a common objection to any Windows and Linux comparison that counting the optional applications against the Linux distribution is unfair, so Ive completed an extra level of analysis to exclude component vulnerabilities that do not have comparable functionality shipping with a Windows OS.
"You may read Red Hat and WindowsDefining an Apples-to-Apples Workstation Build for more details, but basically I install an RHEL4WS computer and I exclude any component that is not installed by default, which includes all optional "server" components that ship with RHEL4WS. I additionally exclude text-Internet, graphics (the Gimp stuff) and office (OpenOffice) and Development Tools (gcc, etc.) installation groups. I use the rpm command to list out all packages that get installed and use that package list to filter vulnerabilities."
Jones described the result as a Gnome-Windows workstation that includes standard system management tools and Firefox for browsing, sound and video support, but excludes all server packages, as well as OpenOffice and other optional components that a Windows system wouldnt have by default.
He compared the security performance of this reduced RHEL4WS build to Vistas. During the first 6 months, Red Hat fixed 214 vulnerabilities affecting the reduced RHEL4WS set of components. Sixty-two of those addressed were of high severity. At the end of the six-month period, a total of 59 publicly disclosed vulnerabilities in the reduced set of components did not yet have a patch from Red Hat, 12 of them rated high severity.
"So, though the reduced component set of RHEL4WS did have a better six-month period than the full product, Red Hat customers did face a reasonably large number of vulnerabilities in the first six months," Jones wrote.
As far as Ubuntu 6.06 LTS (Long-Term Support) goes, Jones said it had 29 vulnerabilities already publicly disclosed prior to the June 1, 2006 availability date. Seven of the nine high-severity issues were fixed one week later on June 8. Furthermore, during the first six months, Ubuntu fixed 145 vulnerabilities affecting Ubuntu 6.06 LTS, 47 of which were rated high severity in the NVD. At the end of the six-month period, there were at least 20 publicly disclosed vulnerabilities in Ubuntu 6.06 LTS that did not yet have a patch available from Ubuntu.
A reduced-component build of Ubuntu 6.06 LTS had 74 vulnerabilities in its first six months, Jones said, 28 of which were deemed high severity. At the end of the six-month period, a total of 11 publicly disclosed vulnerabilities in the reduced set of components did not yet have a patch from Ubuntu, two of which were rated high severity, he said.
Novells SLED 10 (SUSE Linux Enterprise Desktop 10), released on July 17, 2006, had "at least 23 vulnerabilities" already publicly disclosed prior to the ship date, and Novell provided fixes for 20 of these in the first six months, Jones said. Of those, five flaws were high severity.
During the first six months, Novell fixed a total of 159 vulnerabilities affecting SLED 10, of which 50 were rated high severity in the NVD. At the end of the six-month period, there were at least 27 publicly disclosed vulnerabilities in SLED 10 that did not yet have a patch from Novell, six of them high severity.
For the reduced component build of SLED 10, in its first six months, according to Jones count, Novell fixed 123 vulnerabilities affecting the reduced SLED10 desktop set of components. Forty-four of those addressed were of high severity. At the end of the six-month period, a total of 20 publicly disclosed vulnerabilities in the reduced set of components did not yet have a patch from Novell, six of them rated high severity.
As for Mac OS X, Mac OS X v10.4 had 10 vulnerabilities already publicly disclosed prior to the April 29, 2005 ship date and Apple provided fixes for nine of these during the first six months after shipment. Three of the vulnerabilities were high severity. During the first six months, Apple fixed a total of 60 vulnerabilities affecting Mac OS X v10.4, of which 18 were rated high severity in the NVD. At the end of the six-month period, Mac OS X v10.4 still had 16 publicly disclosed vulnerabilities that did not yet have a patch available from Apple, three of them rated high severity.
How Vista stacks up.