eWEEK content and product recommendations are editorially independent. We may make money when you click on links to our partners. Learn More.
Editors Note: This story is Part 2 in a series of three stories about Microsofts Linux and open-source lab.
Microsoft Corp. seems to be moving away from focusing on the actual number of security patches and updates that it and its software competitors release.
Instead, it is concentrating on making it easy and efficient for customers to obtain the security fixes and update their systems.
Bill Hilf, who is director of Platform Technology Strategy at Microsoft and heads its Linux and open-source lab, told eWEEK in a recent interview that “the differentiator for customers is not the number comparison, but which vendor makes the patching and updating experience the least complex, most efficient and easiest to manage.”
Mark Cox, security response team leader at Linux vendor Red Hat, agrees, saying that one of the top reasons machines are ensnared by security exploits is that they dont obtain the latest security updates. “So it follows that to protect users a vendor needs to make security updates as easy and painless as possible, across the entire application stack.”
That is why Microsofts Linux lab simulates production environments—across open-source software, Microsoft software and other commercial software. It has built tests and analysis tools to look at how frequently those systems need to be patched and what the impact of that is.
Microsoft has an update model known as “Patch Tuesday” where patches and updates are issued once a month unless they are critical and need to be released earlier. This model is different from those of the various Linux and other commercial software vendors.
As such, the lab has taken various commercial Linux distributions, running a variety of workloads, and simulating the Patch Tuesday model. At the same time, the lab runs the same workloads and system configurations on a separate set of servers that are patched via the normal model from the commercial distribution vendors.
“Looking at various models is the most important area of patch update work were doing in the lab right now. In total, this type of data gives us a deeper understanding of not just how different vendors do patch updates, but also what the impact is to real workloads in a real data center,” Hilf said.
Hilf also stressed that this is not a one-time thing for Microsoft, which is running similar scenarios on an ongoing basis using the latest versions of Red Hat Enterprise Linux and Novell SUSE Linux, as well as the Mandriva, Gentoo, Debian and Ubuntu Linux versions. It also tests a wide variety of Unix systems and BSDs (Berkeley Software Distributions).
Next Page: Testing patch distributions.
Testing Patch Distributions
Asked what information Microsoft has gleaned from this research, Hilf said, “Overall, we found there are pros and cons to the way commercial Linux distributions manage patches and updates. Many of the issues we saw on the Linux distributions came up when we added additional open-source software to the system.”
The number of updates for the many software distributions is also less important to Hilf than the bigger picture, which shows that it is not just Microsoft software that has to be regularly patched and updated. “Patching and updating is part of life in the data center regardless of the operating system or platform software,” he said.
That being said, he pointed eWEEK to the data the different software vendors have provided on the Web. “On the security front,” he said, “the number of security bulletins/advisories issued in 2005, including all severity ratings, ranged from 168 bulletins for Red Hat Enterprise Linux 4 to 134 for Red Hat Enterprise Linux 3 and 55 for Microsoft.
“Again, this really shows that patching, particularly for security, is not a Microsoft problem, but something that affects all operating system and platform vendors,” Hilf said.
But Red Hats Cox dismissed this comparison out-of-hand, saying that the Microsoft and Red Hat advisories were at different levels of abstraction, so no direct comparison is meaningful.
“Even vulnerability counts normalized, say, to CVE names are hard to compare given the difference in software shipped by each vendor,” Cox said. “Although we shipped 168 security advisories for RHEL4 in the year, only 17 of the underlying vulnerabilities were of critical severity [using the same scales as Microsoft for vulnerability severity].”
Of those 17 critical vulnerabilities, Red Hat made fixes for every one of them available to customers via the Red Hat Network within two days of the vulnerabilities being known to the public, with 87 percent of them being available the first day.
“These sorts of statistics give customers a much better feeling for the risk and exposure theyll be taking when choosing a platform,” he said. “Of course, we could reduce the number of advisories by batching issues into a single update every month, or by not fixing those vulnerabilities rated as low severity, but that is actually detrimental and increases the risk to customers. Were not going to play the numbers game with our customers.”
In addition, Cox pointed out that Red Hat is often far quicker to respond to security issues affecting its customers. In late 2005 when flaws were found in Macromedias Flash Player, Red Hat took responsibility for providing users with a vulnerable version of the Flash plug-in and made an update available, he said.
Red Hat Enterprise Linux customers who had installed the Flash Player got their update by using the Red Hat Network or through their automated updates in the usual way, so no special actions were required. Customers who had installed the player even got a customized notification from Red Hat Network telling them which of their systems needed updating, Cox said.
Next Page: Responding to security alerts.
Dealing with Security Alerts
“Microsoft customers were left on their own,” Cox said. “For several days the only way customers could find out about this issue was from the Microsoft security team Weblog or if they read something in the press about Flash vulnerabilities and realized they had it installed. Later, Microsoft issued an advisory telling customers to visit the Macromedia site to obtain an update.”
Interestingly, Microsofts Hilf has a personal Red Hat workstation in his office that he uses on a daily basis. He selected a random week in October to provide a snapshot of the updates made to his Red Hat Enterprise Linux workstation over that period. He found that, between Oct. 6, 2005, and Oct. 11, 2005, his workstation was updated 66 times.
“I chose those dates randomly,” he said. “I use this system daily, so it was literally a snapshot of a given workweek. All this illustrates is that patching and updating are part of any living software system. It is part of the nature of modern software: Things change, bugs happen, features get added, and software needs to get updated.”
But Red Hats Cox pointed out that the second update release for RHEL4 was issued Oct. 5, resulting in a very large number of updated packages over the period of a day or two, “which is what Hilf saw. We only issued two Update releases for RHEL4 in 2005, so he was quite unlucky in his choice of a random snapshot,” he said, tongue in cheek.
Over that six-day period, only three security updates were released, one rated “important” and two rated “moderate,” Cox pointed out, adding that from the release of Red Hat Enterprise Linux 4 in February 2005 until Jan. 5, 2006, just 15 of the total 169 security errata package updates for the year were for issues rated “critical.”
Hilf also downplayed the significance of the number of updates, saying: “Our focus isnt a counting contest; it is to understand the models, the architecture for patching, and the manageability of the process. So I got a load of patches from Red Hat on my Linux workstation over the course of six days in October. Was that a big deal? Not really.”
Thats because Linux distributions update at a package or component level, so a user is often notified about updates more than Windows users at the component level.
When Red Hat releases an update (rather like a Windows service pack), it issues separate advisories for each package updated, giving users the ability to obtain all the updates or to select updates based on their own criteria, Cox said. “Customers may decide to only update for critical and important security issues, for example, and can do so easily using the Red Hat Network,” he said.
Later this year, Hilf said, he will have about two years of data, “and I expect to have more quantifiable data at that point.”
Next Page: Analyzing OS security.
Checking OS Security
But the security work does not stop there. Another ongoing lab research project is analyzing security across various operating systems, and Hilf and his team routinely do a security screen when they install any operating system in the lab.
“Like many other Linux folks, we use a tool called nmap to scan the operating system as soon as we install it and put it on the network. So, after a fresh install, we port scanned Red Hat Enterprise Linux 4, Novell SUSE Linux Enterprise Server 9, IBM AIX5L, Sun Solaris 10 and Windows Server 2003 SP1,” Hilf said.
What surprised the team most was how many services they found open after a fresh install on the commercial Unix systems. Both Red Hat Enterprise Linux 4 and Windows Server 2003 SP1 had four open services; Novell SUSE had nine open services; AIX5L had 18; and Solaris 10 had 20.
“So, for example, we found services like FTP, exec, login, telnet and finger enabled and open by default. Typically, these should be disabled by default for most environments due to both their security mechanisms and security track records. Most environments should use something much more secure than telnet for remote console access,” he said.
But, Hilf stressed, it is important to note that IT administrators would rarely deploy a default server “out-of-the-box” into a production environment. Most IT professionals would do a similar audit and configure the security of their server for their environmental needs.
“But, since we do this analysis, we found it interesting to note this default profile. From a Microsoft perspective, I think this is a good demonstration of our investments in trustworthy computing and improving our security development lifecycle,” he said.
But Chris Ratcliffe, director of marketing for Suns Solaris, countered that the standard Solaris 10 install profile leaves certain services switched on for compatibility reasons. “Having said that, there are two other things to bear in mind: Firstly, with Solaris 10, system administrators have a choice of installation profiles. For example, the Reduced Networking Metacluster install option [also known as Warm Brick Mode] creates a minimized Solaris image to which security administrators can then add the functionality and services they actually need,” he said.
Solaris 10 also has a Service Manager Profile known as Generic Limited Networking, which turns off almost all unencrypted remote communications to the system, Ratcliffe said.
Secondly, not all services “are created equally. In Solaris 10, remote applications such as rsh, rcp, telnet, Solaris Secure Shell and others are Kerberos-enabled. They make use of a standards-based common API for high-performance, systemwide cryptographic routines. This framework provides a single point of administration and uniform access to hardware-accelerated cryptographic functions when available,” he said.
In addition, most Solaris users perform system installs and configurations in an environment where this is not an issue. “In todays environment, any vendor not taking a systemwide approach to security is making a big mistake. Any vendor that hasnt really considered the implications of security in the networked environment in their operating system design has a lot of catching up to do,” he said.
Trustworthy Computing is about far more than just what ports or services are open, Ratcliffe said, adding that while this is certainly a factor, more than 80 percent of security violations come from inside companies, so vendors have to take a much broader view.
Solaris 10 provides a host of security features previously only found in Suns military-grade Trusted Solaris. Sun has also digitally signed the operating system, automatically checked file integrity and has made its services more secure, he said.
“In an upcoming update to Solaris, well add the ability to lock a system down so that only valid, signed executables from a list of sys admin configurable trusted authorities will be allowed to run. Rogue applications, Trojan horses and viruses simply will not execute,” Ratcliffe said.