eWEEK content and product recommendations are editorially independent. We may make money when you click on links to our partners. Learn More.
Microsoft Corp. will release Wednesday the findings of another survey it has sponsored, this time comparing two platforms—its Windows Server System and Novell Inc.s SUSE Linux Enterprise Server—under evolving business requirements over an extended period of time.
The study found that there are fundamental differences in the Linux and Windows models, and while there are advantages to Linux componentization, there are also downsides in terms of complexity, time to manage and cost, according to Ryan Gavin, Microsofts director of platform strategy
The goal of the study, conducted by Herbert Thompson of Security Innovations Inc. and titled “Reliability: Analyzing Solution Business Needs Change,” is to “help IT administrators make informed and business-savvy deployment decisions by identifying meaningful differences in real-world reliability between these platforms,” said Thompson in the executive summary. “We also look at the broader implications of the model used by the two vendors and how the third-party community builds upon those systems.”
The full report is expected to be posted here on Wednesday.
Thompson makes clear that neither the study nor its findings are final or conclusive, but rather a starting point for further work. “The sample, although too small to provide conclusive statistical comparisons, illustrates the methodology and begins to shed light on some key model differences between the platforms,” he said.
“A welcomed next step would be a more expansive study based on this foundational methodology with a larger sample size, additional business requirement scenarios and that looks at a wide array of platforms,” he said.
The study is part of Microsofts “Get the Facts” campaign, which was launched last year and is designed to give customers information about the advantages of using its Windows operating system over Linux, its open-source competitor. Many of the statements and “facts” have been challenged by the Linux and open-source community.
Gavin defended the need for the study, even if it is just a starting point. “There is a big need for a larger understanding of how we think and talk about reliability and to have a consistent dialogue about what things matter when you are talking about a reliable system,” he said.
“One of the reasons we worked with Security Innovations is because they are more academic and research-oriented, and we wanted a repeatable methodology that could be applied again and again,” Gavin said.
Thompson will also be reaching out to Novell given that Novells version of Linux was used in the study, and he will be “asking for their validation, having them help review the model and provide input into how this could have been done differently or better,” said Gavin. “Ill also make the same offer to Red Hat [Inc.], and Im willing to fly their engineers to his lab to have them go through this and tell us ways we can do this better.”
While this is an enterprise e-commerce scenario, there are many other workloads and scenarios that could have been picked, he said, adding that this is just the start and he would love to partner with Red Hat or IBM or Novell to explore some ways this could be done better.
Microsoft will also willingly publish and distribute any document that comes out of the process, even if it details the companys unhappiness with the way the study was conducted. “I have no hesitation in committing to doing that,” Gavin said.
While neither Red Hat nor Novell could be immediately reached for comment, Microsoft has previously suggested that the Open Source Development Labs work with it on a research project, an idea that was rejected outright by OSDL CEO Stuart Cohen.
Thompson said what differentiated the methodology presented in his study from component-based studies was that it considered the reliability of a system holistically, where a system was viewed as a collection of smaller parts, each of which could be benchmarked and studied in isolation.
“We conducted an experiment pitting Windows 2000 Server against SUSE Linux Enterprise Server 8, simulating the one-year period from July 1, 2004, to June 30, 2005,” Thompson said in the summary. “During this period, we simulated the evolution of an e-commerce company that has changing business requirements while continuing to maintain security through patch application. At the end of the period, both systems are then transitioned to the more recent versions of their respective operating systems, Windows Server 2003 and SUSE Linux Enterprise Server 9.”
Security patches were applied in one-month increments, while new business requirements appeared at three-month intervals. The experiment was conducted by three expert Windows administrators on the Windows side and three expert SUSE Linux administrators on the Linux side.
The study considered one evolution scenario: an e-commerce company that must move its site from basic purchasing to a personalized, history-driven portal. These requirements were created with the help and input of some of the worlds largest e-retailers, and the experiment followed the life of two systems, one Windows-based and one SUSE Linux-based, over a simulated one-year period.
Next Page: What the study found.
Page 2
While the sample size of administrators was too small to provide conclusive statistical comparisons, the results highlight some fundamental differences in the Linux and Windows models, the study said.
The results of this initial study showed some interesting patterns, Thompson said in the report. One of the most heavily touted benefits of Linux is its high modularity and the granularity of control that administrators have over a system. “In the experiment, we found that such flexibility also leads to ambiguity for administrators in terms of paths to follow when resolving conflicts,” Thompson said.
“On the Linux side, each administrator pursued vastly different paths to resolve dependency conflicts that arose when new components were installed. The result was solutions that grew in complexity and heterogeneity rapidly over time,” he said.
Multiple pathways can be both an asset and a weakness, allowing on the one hand highly skilled administrators to solve problems using greatly varied approaches, but on the other leading to the “personalization” of systems that could make issues like administrator substitution problematic, the report said.
“The Linux solutions also quickly went out of support from both the distribution vendor and third-party solution vendors as individual components [such as MySQL] were upgraded to meet third-party solution needs,” Thompson said.
In contrast, Microsoft has pursued a philosophy it calls “integrated innovation,” where much of the core system functionality is incorporated with the operating system itself.
During the experiment, all Windows administrators followed a fairly homogeneous route to both install patches and apply component upgrades for the simulated changing business requirements, Thompson said in the report.
“For the administrator trials, our choice of requirements was based on a common evolutionary scenario, one vetted and confirmed by analysts and corporations in the e-commerce space. Our choice of third-party components was based on market leadership in the enterprise markets,” the report said.
While the experiment followed only one scenario with a small set of administrators, the results did highlight some key model differences in Windows and Linux, Thompson said. “Our hope is that the methodology presented here will serve as a base that others can expand on and adapt to their own business needs,” he said.
“The foundation has now been laid, and it needs to be taken to the next level and tried as a database scenario, as a business applications scenario,” Microsofts Gavin said.