IBM Wallops BEA in Faux Benchmark

The application server space, once on the verge of drying up, is revving up once again to be a full-fledged slugfest. Customers dont really care about the slugfest part—theyre just looking for some deals—but everyone appreciates the entertainment.

The main contenders are BEA, IBM, Oracle and Sun, and the hubris begins with them. IBM believes it has only a single competitor in the application server space: BEA. Then IBM digs up a March Giga analyst report (which can be accessed via www.eweek.com/links, as can the other two reports mentioned below) that shows both companies have 34 percent of the market. Thats a far cry from WebSpheres single-digit numbers just two years ago.

Oracle thinks its in a mano a mano battle with IBM. Then Oracle finds a Hurwitz report that says its on top of all application server companies, including IBM. Suns iPlanet division—at one time the owner of J2EE application server technology—sees a long road ahead and is focused on the applications part of application servers.

Microsoft, obviously not in the J2EE space but desperate to be listed as competition, is trying to figure out how to call Windows an application server without productizing it. Microsoft will probably dig out last months Meta report on application servers that shows Microsoft will have 30 percent of the enterprise market in two years.

It appears that only BEA sees that this race is not winnable without establishing partnerships, something the company did its best to avoid just a few years ago. Unfortunately, BEA is now locked in a hopeless performance benchmarking battle that is useless to most customers. In this case, BEA is being walloped by IBM in the ECperf application server test hosted by The ServerSide (www.theserverside.com). The most interesting part of this test is not that WebSphere got such high scores (see ecperf.theserverside.com/ecperf) but that IBM finally put its benchmarking where its mouth is. IBM ran the test with a Linux system and used DB2 on the back end. Otherwise, its just a benchmark that was manipulated by engineers who threw more hardware into the mix (18 processors this time) to get the appropriate score.

Somewhere, I suppose, between the analyst reports, the media hype and the faux benchmarks, lies some truth. May the best technology win. (Watch for eWeek Labs application server evaluation in the May 13 issue.)

What is the truth? Write to me at john_taschek@ziffdavis.com.