Microsoft-Sponsored Study Gives Nod to Windows in Reliability

By Peter Galli  |  Posted 2005-11-16 Print this article Print

Microsoft sponsors yet another study, this one on reliability that highlights fundamental differences in the Linux and Windows models.

Microsoft Corp. will release Wednesday the findings of another survey it has sponsored, this time comparing two platforms—its Windows Server System and Novell Inc.s SUSE Linux Enterprise Server—under evolving business requirements over an extended period of time. The study found that there are fundamental differences in the Linux and Windows models, and while there are advantages to Linux componentization, there are also downsides in terms of complexity, time to manage and cost, according to Ryan Gavin, Microsofts director of platform strategy The goal of the study, conducted by Herbert Thompson of Security Innovations Inc. and titled "Reliability: Analyzing Solution Business Needs Change," is to "help IT administrators make informed and business-savvy deployment decisions by identifying meaningful differences in real-world reliability between these platforms," said Thompson in the executive summary. "We also look at the broader implications of the model used by the two vendors and how the third-party community builds upon those systems."
The full report is expected to be posted here on Wednesday.
Thompson makes clear that neither the study nor its findings are final or conclusive, but rather a starting point for further work. "The sample, although too small to provide conclusive statistical comparisons, illustrates the methodology and begins to shed light on some key model differences between the platforms," he said. "A welcomed next step would be a more expansive study based on this foundational methodology with a larger sample size, additional business requirement scenarios and that looks at a wide array of platforms," he said. The study is part of Microsofts "Get the Facts" campaign, which was launched last year and is designed to give customers information about the advantages of using its Windows operating system over Linux, its open-source competitor. Many of the statements and "facts" have been challenged by the Linux and open-source community. Gavin defended the need for the study, even if it is just a starting point. "There is a big need for a larger understanding of how we think and talk about reliability and to have a consistent dialogue about what things matter when you are talking about a reliable system," he said. "One of the reasons we worked with Security Innovations is because they are more academic and research-oriented, and we wanted a repeatable methodology that could be applied again and again," Gavin said. Thompson will also be reaching out to Novell given that Novells version of Linux was used in the study, and he will be "asking for their validation, having them help review the model and provide input into how this could have been done differently or better," said Gavin. "Ill also make the same offer to Red Hat [Inc.], and Im willing to fly their engineers to his lab to have them go through this and tell us ways we can do this better." While this is an enterprise e-commerce scenario, there are many other workloads and scenarios that could have been picked, he said, adding that this is just the start and he would love to partner with Red Hat or IBM or Novell to explore some ways this could be done better. Microsoft will also willingly publish and distribute any document that comes out of the process, even if it details the companys unhappiness with the way the study was conducted. "I have no hesitation in committing to doing that," Gavin said. While neither Red Hat nor Novell could be immediately reached for comment, Microsoft has previously suggested that the Open Source Development Labs work with it on a research project, an idea that was rejected outright by OSDL CEO Stuart Cohen. Click here to read eWEEKs interview with Cohen on why he nixed Microsofts joint research offer. Thompson said what differentiated the methodology presented in his study from component-based studies was that it considered the reliability of a system holistically, where a system was viewed as a collection of smaller parts, each of which could be benchmarked and studied in isolation. "We conducted an experiment pitting Windows 2000 Server against SUSE Linux Enterprise Server 8, simulating the one-year period from July 1, 2004, to June 30, 2005," Thompson said in the summary. "During this period, we simulated the evolution of an e-commerce company that has changing business requirements while continuing to maintain security through patch application. At the end of the period, both systems are then transitioned to the more recent versions of their respective operating systems, Windows Server 2003 and SUSE Linux Enterprise Server 9." Click here for a review of SUSE Linux Enterprise Server 9. Security patches were applied in one-month increments, while new business requirements appeared at three-month intervals. The experiment was conducted by three expert Windows administrators on the Windows side and three expert SUSE Linux administrators on the Linux side. The study considered one evolution scenario: an e-commerce company that must move its site from basic purchasing to a personalized, history-driven portal. These requirements were created with the help and input of some of the worlds largest e-retailers, and the experiment followed the life of two systems, one Windows-based and one SUSE Linux-based, over a simulated one-year period. Next Page: What the study found.

Peter Galli has been a financial/technology reporter for 12 years at leading publications in South Africa, the UK and the US. He has been Investment Editor of South Africa's Business Day Newspaper, the sister publication of the Financial Times of London.

He was also Group Financial Communications Manager for First National Bank, the second largest banking group in South Africa before moving on to become Executive News Editor of Business Report, the largest daily financial newspaper in South Africa, owned by the global Independent Newspapers group.

He was responsible for a national reporting team of 20 based in four bureaus. He also edited and contributed to its weekly technology page, and launched a financial and technology radio service supplying daily news bulletins to the national broadcaster, the South African Broadcasting Corporation, which were then distributed to some 50 radio stations across the country.

He was then transferred to San Francisco as Business Report's U.S. Correspondent to cover Silicon Valley, trade and finance between the US, Europe and emerging markets like South Africa. After serving that role for more than two years, he joined eWeek as a Senior Editor, covering software platforms in August 2000.

He has comprehensively covered Microsoft and its Windows and .Net platforms, as well as the many legal challenges it has faced. He has also focused on Sun Microsystems and its Solaris operating environment, Java and Unix offerings. He covers developments in the open source community, particularly around the Linux kernel and the effects it will have on the enterprise.

He has written extensively about new products for the Linux and Unix platforms, the development of open standards and critically looked at the potential Linux has to offer an alternative operating system and platform to Windows, .Net and Unix-based solutions like Solaris.

His interviews with senior industry executives include Microsoft CEO Steve Ballmer, Linus Torvalds, the original developer of the Linux operating system, Sun CEO Scot McNealy, and Bill Zeitler, a senior vice president at IBM.

For numerous examples of his writing you can search under his name at the eWEEK Website at


Submit a Comment

Loading Comments...
Manage your Newsletters: Login   Register My Newsletters

Rocket Fuel