The study found that there are fundamental differences in the Linux and Windows models, and while there are advantages to Linux componentization, there are also downsides in terms of complexity, time to manage and cost, according to Ryan Gavin, Microsofts director of platform strategy
The goal of the study, conducted by Herbert Thompson of Security Innovations Inc. and titled "Reliability: Analyzing Solution Business Needs Change," is to "help IT administrators make informed and business-savvy deployment decisions by identifying meaningful differences in real-world reliability between these platforms," said Thompson in the executive summary. "We also look at the broader implications of the model used by the two vendors and how the third-party community builds upon those systems."
The full report is expected to be posted here on Wednesday.
Thompson makes clear that neither the study nor its findings are final or conclusive, but rather a starting point for further work. "The sample, although too small to provide conclusive statistical comparisons, illustrates the methodology and begins to shed light on some key model differences between the platforms," he said.
"A welcomed next step would be a more expansive study based on this foundational methodology with a larger sample size, additional business requirement scenarios and that looks at a wide array of platforms," he said.
The study is part of Microsofts "Get the Facts" campaign, which was launched last year and is designed to give customers information about the advantages of using its Windows operating system over Linux, its open-source competitor. Many of the statements and "facts" have been challenged by the Linux and open-source community.
Gavin defended the need for the study, even if it is just a starting point. "There is a big need for a larger understanding of how we think and talk about reliability and to have a consistent dialogue about what things matter when you are talking about a reliable system," he said.
"One of the reasons we worked with Security Innovations is because they are more academic and research-oriented, and we wanted a repeatable methodology that could be applied again and again," Gavin said.
Thompson will also be reaching out to Novell given that Novells version of Linux was used in the study, and he will be "asking for their validation, having them help review the model and provide input into how this could have been done differently or better," said Gavin. "Ill also make the same offer to Red Hat [Inc.], and Im willing to fly their engineers to his lab to have them go through this and tell us ways we can do this better."
While this is an enterprise e-commerce scenario, there are many other workloads and scenarios that could have been picked, he said, adding that this is just the start and he would love to partner with Red Hat or IBM or Novell to explore some ways this could be done better.
Microsoft will also willingly publish and distribute any document that comes out of the process, even if it details the companys unhappiness with the way the study was conducted. "I have no hesitation in committing to doing that," Gavin said.
While neither Red Hat nor Novell could be immediately reached for comment, Microsoft has previously suggested that the Open Source Development Labs work with it on a research project, an idea that was rejected outright by OSDL CEO Stuart Cohen.
Thompson said what differentiated the methodology presented in his study from component-based studies was that it considered the reliability of a system holistically, where a system was viewed as a collection of smaller parts, each of which could be benchmarked and studied in isolation.
"We conducted an experiment pitting Windows 2000 Server against SUSE Linux Enterprise Server 8, simulating the one-year period from July 1, 2004, to June 30, 2005," Thompson said in the summary. "During this period, we simulated the evolution of an e-commerce company that has changing business requirements while continuing to maintain security through patch application. At the end of the period, both systems are then transitioned to the more recent versions of their respective operating systems, Windows Server 2003 and SUSE Linux Enterprise Server 9."
Security patches were applied in one-month increments, while new business requirements appeared at three-month intervals. The experiment was conducted by three expert Windows administrators on the Windows side and three expert SUSE Linux administrators on the Linux side.
The study considered one evolution scenario: an e-commerce company that must move its site from basic purchasing to a personalized, history-driven portal. These requirements were created with the help and input of some of the worlds largest e-retailers, and the experiment followed the life of two systems, one Windows-based and one SUSE Linux-based, over a simulated one-year period.
Next Page: What the study found.
Research Assistant
