IBM Wallops BEA in Faux Benchmark

By John Taschek  |  Posted 2002-05-06 Print this article Print

Somewhere between the analyst reports, the media hype and the faux benchmarks, lies some truth.

The application server space, once on the verge of drying up, is revving up once again to be a full-fledged slugfest. Customers dont really care about the slugfest part—theyre just looking for some deals—but everyone appreciates the entertainment.

The main contenders are BEA, IBM, Oracle and Sun, and the hubris begins with them. IBM believes it has only a single competitor in the application server space: BEA. Then IBM digs up a March Giga analyst report (which can be accessed via, as can the other two reports mentioned below) that shows both companies have 34 percent of the market. Thats a far cry from WebSpheres single-digit numbers just two years ago.

Oracle thinks its in a mano a mano battle with IBM. Then Oracle finds a Hurwitz report that says its on top of all application server companies, including IBM. Suns iPlanet division—at one time the owner of J2EE application server technology—sees a long road ahead and is focused on the applications part of application servers.

Microsoft, obviously not in the J2EE space but desperate to be listed as competition, is trying to figure out how to call Windows an application server without productizing it. Microsoft will probably dig out last months Meta report on application servers that shows Microsoft will have 30 percent of the enterprise market in two years.

It appears that only BEA sees that this race is not winnable without establishing partnerships, something the company did its best to avoid just a few years ago. Unfortunately, BEA is now locked in a hopeless performance benchmarking battle that is useless to most customers. In this case, BEA is being walloped by IBM in the ECperf application server test hosted by The ServerSide ( The most interesting part of this test is not that WebSphere got such high scores (see but that IBM finally put its benchmarking where its mouth is. IBM ran the test with a Linux system and used DB2 on the back end. Otherwise, its just a benchmark that was manipulated by engineers who threw more hardware into the mix (18 processors this time) to get the appropriate score.

Somewhere, I suppose, between the analyst reports, the media hype and the faux benchmarks, lies some truth. May the best technology win. (Watch for eWeek Labs application server evaluation in the May 13 issue.)

What is the truth? Write to me at

As the director of eWEEK Labs, John manages a staff that tests and analyzes a wide range of corporate technology products. He has been instrumental in expanding eWEEK Labs' analyses into actual user environments, and has continually engineered the Labs for accurate portrayal of true enterprise infrastructures. John also writes eWEEK's 'Wide Angle' column, which challenges readers interested in enterprise products and strategies to reconsider old assumptions and think about existing IT problems in new ways. Prior to his tenure at eWEEK, which started in 1994, Taschek headed up the performance testing lab at PC/Computing magazine (now called Smart Business). Taschek got his start in IT in Washington D.C., holding various technical positions at the National Alliance of Business and the Department of Housing and Urban Development. There, he and his colleagues assisted the government office with integrating the Windows desktop operating system with HUD's legacy mainframe and mid-range servers.

Submit a Comment

Loading Comments...
Manage your Newsletters: Login   Register My Newsletters

Rocket Fuel