What Linuxs TPC Victory Doesnt Show

 
 
By John Taschek  |  Posted 2001-05-28 Email Print this article Print
 
 
 
 
 
 
 

Ah. Some Linux advocates must smell the sweetness of victory after the operating system came out on top in a recent TPC benchmark.

Ah. Some Linux advocates must smell the sweetness of victory after the operating system came out on top in a recent TPC benchmark. That the benchmark is flawed, is more appropriate as a hardware test and is not relevant for operating system comparisons be damned!

Experienced IT professionals know exactly what the TPC is and how its useful. Theyre not sucked in by the hype that the Transaction Processing Performance Council both detests—because it calls attention to the flaws in the benchmark—and loves—because it needs the attention. Those knowing IT professionals will smile and move on.

The rest of the world, meanwhile, will split: Half will chalk up a victory for the underdog, and half will get cranky and talk about how expensive that Linux system really is or about how Microsoft dominates the other nine positions in the TPC-H 100GB Top 10. Microsofts IIS, for example, dominates all the TPC-W Web eCommerce benchmarks, but Apaches Web server is the one thats spectacularly successful in the market.

Lets be clear. Although the TPC was intended to be a hardware test, software companies adopted it for their own good. Oracle crushed many competitors by using the TPC to its advantage in the early to mid-1990s, when it was a flagging company on the verge of losing momentum to Sybase and Informix.

Oracle now despises the tests and barely participates in them because of the implications that the company itself started. Microsoft, however, sinks unlimited resources into the entire series of tests—not just the TPC-H but also the TPC-C, TPC-D and TPC-W. Thats why it "wins" these comparisons.

If Linux advocates claim the TPC-H score as a victory, theyll also have to acknowledge defeat in the other tests. The reality is that none of it matters.

This test is far more important to Silicon Graphics. It shows the companys commitment to Linux, and it shows that SGI can still produce great hardware. Unfortunately, the media and the generally inflamed public responding by the thousands to the message boards forgot about SGI, which very well could be saved by Linux.

There is a Linux "victory," however. Before Microsoft gets angry and starts pouring all its resources into retaking the performance lead, Linuxs amazing score shows that at least two companies with substantial resources (one far more than the other) are willing to go to bat for Linux and prove its value to the enterprise. It also shows that Linux wont be a bottleneck in large multiprocessor systems. It shows that DB2 is a useful decision-support database (although I question why Informix pulled its TPC-H scores in February). It doesnt, however, show that Linux is "faster" than Windows.

 
 
 
 
As the director of eWEEK Labs, John manages a staff that tests and analyzes a wide range of corporate technology products. He has been instrumental in expanding eWEEK Labs' analyses into actual user environments, and has continually engineered the Labs for accurate portrayal of true enterprise infrastructures. John also writes eWEEK's 'Wide Angle' column, which challenges readers interested in enterprise products and strategies to reconsider old assumptions and think about existing IT problems in new ways. Prior to his tenure at eWEEK, which started in 1994, Taschek headed up the performance testing lab at PC/Computing magazine (now called Smart Business). Taschek got his start in IT in Washington D.C., holding various technical positions at the National Alliance of Business and the Department of Housing and Urban Development. There, he and his colleagues assisted the government office with integrating the Windows desktop operating system with HUD's legacy mainframe and mid-range servers.
 
 
 
 
 
 
 

Submit a Comment

Loading Comments...
 
Manage your Newsletters: Login   Register My Newsletters























 
 
 
 
 
 
 
 
 
 
 
Rocket Fuel