At the Structure conference in San Francisco June 25, Facebook Vice President Jonathan Heiliger took chip makers and OEMs to task, saying the social networking site was not seeing the expected performance improvements from systems powered by the newest processors.
Heiliger, vice president of technical operations at Facebook, said Facebook engineers were not seeing the same performance results in tests that Advanced Micro Devices and Intel have been promoting for their latest processor releases.
AMD officials are now responding to the criticism. The officials, while noting Heiliger’s concerns, said the Opteron processors AMD has released in the past few months are among the highest-performing and most efficient on the market. They also said what is most needed are benchmarks that more accurately reflect real-world situations.
In a talk at the conference with GigaOM’s Om Malik, Heiliger said the results Facebook engineers were seeing were disappointing.
“The biggest thing that surprised us … is the less-than-anticipated performance gains from new microarchitectures,” Heiliger said. “The performance gains they are touting in the press, we are not seeing in our applications.”
In a blog post July 13, Nigel Dessau, senior vice president and chief marketing officer at AMD, defended the results that the new chips are hitting “in raw, classic benchmark terms,” but said Facebook may be getting thrown off because its programming model uses more PHP and Java than C++.
“Let’s face it: Synthetic benchmarks are essentially a useful evil,” Dessau wrote. “Everyone wants to know how a certain technology performs against a standardized test, but what happens when that test [bears] no real resemblance to the real work people do? You get a huge disconnect.”
That disconnect is amplified in highly scaled data centers where massive server farms run such environments as cloud computing, where small discrepancies between benchmarks and real results are magnified.
In a blog post July 7, AMD Product Marketing Director Margaret Lewis-who was in the audience at the show when Heiliger made his remarks-said AMD officials are talking a lot about how to make benchmarks better reflect real-world situations.
“Jonathan’s comments validate this approach,” Lewis said. “He has a very specialized software stack that he believes is not showing the performance of the industry-standard server benchmarks that AMD and Intel quote.”
That said, Lewis and Dessau defended what AMD is doing, pointing to the latest quad- and six-core Opterons as striking a good balance between raw performance and energy efficiency.
Dessau in his blog pointed to the quad-core Opteron EE chip-which consumes 40 watts of power-and the six-core Opteron HE, the latest in AMD’s “Istanbul” family that AMD launched July 12.
“This continues our tradition of bringing highly efficient, power-optimized solutions to market to meet the demands of customers small and large-from small businesses to massive server farms,” Dessau said.
AMD and Intel are locked in a tight competition to develop energy-efficient chips that improve performance. Where once the chip makers relied on cranking up the clock speed of chips to improve performance, they now are looking at other ways-such virtualization support and power management features-to improve performance while holding down power consumption.
Intel in March rolled out its quad-core Xeon 5500 Series “Nehalem EP” processors for servers with two sockets, which officials touted for their performance and efficiency gains. Intel is expected to release its “Nehalem EX” chips for four-socket servers later in 2009. AMD unveiled Istanbul in June.