Vendors trying to compete with Microsoft on performance benchmarking should give up and throw in the towel. Theres no hope for success.
Microsoft will win every possible benchmark war in every major category. For those of us who count “The Terminator” as an influential movie, let this put it into perspective: “[Microsoft] cant be bargained with. It cant be reasoned with. It doesnt feel pity or remorse or fear. And it absolutely will not stop … [until it wins every benchmark war].”
The latest Microsoft smear is in the application server space, which Microsoft has been stomping on since 1998. Not only has Microsoft blurred the definition of an application server (the term once referred to systems used for serving up Java servlets and JavaBeans), its also handing out embarrassing performance losses to the vendors that created the software category.
The latest benchmark is from The Middleware Co. (www.middleware-company.com), a reputable organization that started the also-reputable The Server Side Web site (theserverside.com). Since there are few ways to legitimately benchmark J2EE and .Net applications together, The Middleware Co. took the Java Pet Store application and “fixed” it to make it a more legitimate test.
There are some flaws with the test of course, but, overall, this is well done. Two issues that Java programmers will not easily get over are that Microsoft helped fund the test and that different databases were used. There are thousands of excuses for Microsoft winning this particular benchmark, but, overall, this is as legitimate as they come.
So, is there any hope? Not really. Microsoft will almost always win the battles of ease of development and benchmarking. Microsoft and .Net, however, would get throttled in a large distributed test, but that kind of test would be enormously expensive and would probably be set up only by large private companies that are testing it for their own requirements.
Microsoft, however, is a two-faced company. When its hungry, it can do great things—such as .Net. When its not hungry, its greedy. Consider how Microsoft is reaming its customers on Office licenses or XP product activation. Take advantage, but be wary.
What kinds of benchmarking are you doing? Write to me at john_taschek@ziffdavis.com.