What Linuxs TPC Victory Doesnt Show

By John Taschek  |  Posted 2001-05-28

Ah. Some Linux advocates must smell the sweetness of victory after the operating system came out on top in a recent TPC benchmark. That the benchmark is flawed, is more appropriate as a hardware test and is not relevant for operating system comparisons be damned!

Experienced IT professionals know exactly what the TPC is and how its useful. Theyre not sucked in by the hype that the Transaction Processing Performance Council both detests—because it calls attention to the flaws in the benchmark—and loves—because it needs the attention. Those knowing IT professionals will smile and move on.

The rest of the world, meanwhile, will split: Half will chalk up a victory for the underdog, and half will get cranky and talk about how expensive that Linux system really is or about how Microsoft dominates the other nine positions in the TPC-H 100GB Top 10. Microsofts IIS, for example, dominates all the TPC-W Web eCommerce benchmarks, but Apaches Web server is the one thats spectacularly successful in the market.

Lets be clear. Although the TPC was intended to be a hardware test, software companies adopted it for their own good. Oracle crushed many competitors by using the TPC to its advantage in the early to mid-1990s, when it was a flagging company on the verge of losing momentum to Sybase and Informix.

Oracle now despises the tests and barely participates in them because of the implications that the company itself started. Microsoft, however, sinks unlimited resources into the entire series of tests—not just the TPC-H but also the TPC-C, TPC-D and TPC-W. Thats why it "wins" these comparisons.

If Linux advocates claim the TPC-H score as a victory, theyll also have to acknowledge defeat in the other tests. The reality is that none of it matters.

This test is far more important to Silicon Graphics. It shows the companys commitment to Linux, and it shows that SGI can still produce great hardware. Unfortunately, the media and the generally inflamed public responding by the thousands to the message boards forgot about SGI, which very well could be saved by Linux.

There is a Linux "victory," however. Before Microsoft gets angry and starts pouring all its resources into retaking the performance lead, Linuxs amazing score shows that at least two companies with substantial resources (one far more than the other) are willing to go to bat for Linux and prove its value to the enterprise. It also shows that Linux wont be a bottleneck in large multiprocessor systems. It shows that DB2 is a useful decision-support database (although I question why Informix pulled its TPC-H scores in February). It doesnt, however, show that Linux is "faster" than Windows.

Rocket Fuel