Benchmarks Challenged

By Jim Louderback  |  Posted 2002-09-02 Print this article Print

AMD says tests were redesigned to favor Intel's Pentium 4.

So now CEOs have to sign off on their companies financial statements—on pain of imprisonment. I wish industry marketers were held to the same standard.

The latest "benchmarketing" flap concerns Intel and AMD. According to AMD, BAPCo, a nonprofit consortium, has unfairly stacked the deck in Intels favor.

BAPCo was founded by vendors and publishing companies, including Intel, but not AMD. BAPCos flagship Sysmark test is used by many companies to make buying decisions. Sysmark emulates how a computer is really used by timing common tasks. But the 2002 version, said AMD, has been unfairly redesigned to favor Intels Pentium 4 CPU.

AMD identified 14 tests that were removed from the latest version of the benchmark—each of which favored AMDs Athlon processor. AMD also highlighted a number of new tasks—all of which favored the Pentium 4. In Excel, according to AMD, the Pentium 4-friendly sort function now constitutes more than 90 percent of the test. That seems a tad high. Whether Intel unduly influenced BAPCo, or its benchmark toadies screwed up, I smell a rat. AMD has taken the high road and has joined BAPCo and will address the problem from within.

Its not just AMD vs. Intel. Rumors abound in the Valley about unethical behavior in the high-end 3-D graphics market, and database vendors are often charged with manipulating results so everyone ends up a winner.

Ethical lapses doomed Enron and WorldCom, and now corporate chieftains are only slightly more popular than cockroaches and Bud Selig. Buyers used to trust benchmarks, but these shenanigans may well taint the entire industry. Its too bad because I had thought wed be above the fray. Unfortunately, lying to potential customers isnt a crime. Yet.

Whom do you trust? Jim Louderback, editor in chief of Ziff Davis Internet, can be reached at

With more than 20 years experience in consulting, technology, computers and media, Jim Louderback has pioneered many significant new innovations.

While building computer systems for Fortune 100 companies in the '80s, Jim developed innovative client-server computing models, implementing some of the first successful LAN-based client-server systems. He also created a highly successful iterative development methodology uniquely suited to this new systems architecture.

As Lab Director at PC Week, Jim developed and refined the product review as an essential news story. He expanded the lab to California, and created significant competitive advantage for the leading IT weekly.

When he became editor-in-chief of Windows Sources in 1995, he inherited a magazine teetering on the brink of failure. In six short months, he turned the publication into a money-maker, by refocusing it entirely on the new Windows 95. Newsstand sales tripled, and his magazine won industry awards for excellence of design and content.

In 1997, Jim launched TechTV's content, creating and nurturing a highly successful mix of help, product information, news and entertainment. He appeared in numerous segments on the network, and hosted the enormously popular Fresh Gear show for three years.

In 1999, he developed the 'Best of CES' awards program in partnership with CEA, the parent company of the CES trade show. This innovative program, where new products were judged directly on the trade show floor, was a resounding success, and continues today.

In 2000, Jim began developing, a daily, live, 8 hour TechTV news program called TechLive. Called 'the CNBC of Technology,' TechLive delivered a daily day-long dose of market news, product information, technology reporting and CEO interviews. After its highly successful launch in April of 2001, Jim managed the entire organization, along with setting editorial direction for the balance of TechTV.

In the summer or 2002, Jim joined Ziff Davis Media to be Editor-In-Chief and Vice President of Media Properties, including, Microsoft Watch, and the websites for PC Magazine, eWeek and ZDM's gaming publications.


Submit a Comment

Loading Comments...
Manage your Newsletters: Login   Register My Newsletters

Rocket Fuel