IBM’s massive Roadrunner supercomputer, installed at the Los Alamos National Laboratory, is becoming a shining example of the extraordinary speed of development and innovation in the high-performance computing space.
Five years after becoming the first system to break the petaflop barrier and establishing itself as the fastest supercomputer in the world, the $125 million Roadrunner—which covered 6,000 square feet and held 6,563 dual-core Opteron processors from Advanced Micro Devices, which were coupled with special PowerXCell 8i graphics chips from IBM, all spread out over 296 server racks—has been decommissioned by the lab.
The lab shut down the massive system March 31, although researchers will have about a month to run experiments around operating system memory compression techniques and optimized data routing that will help in designing future cluster computers. After that, Roadrunner will be dismantled.
“Roadrunner was a truly pioneering idea,” Gary Grider, deputy division leader of the laboratory’s High Performance Computing Division, said in a statement March 29. “Roadrunner got everyone thinking in new ways about how to build and use a supercomputer. Specialized processors are being included in new ways on new systems, and being used in novel ways. Our demonstration with Roadrunner caused everyone to pay attention.”
The use of specialized processors was a key advancement for Roadrunner. The PowerXCell 8i—commonly referred to as a “Cell”—was a version of a specialized processor that had been used in Sony’s Playstation 3 gaming console. The PowerXCell 8i was specially optimized for use in scientific computing, and its use was a key part of Roadrunner’s speed, according to lab officials.
The Cell chips would take “the most computationally intense parts of the calculation—thus acting as a computational accelerator,” Los Alamos officials said in the statement. Not only did the use of these accelerators free up the AMD chips for other tasks and improve simulations run by the supercomputer, but it also helped drive greater energy efficiency, they said.
Before Roadrunner, such accelerators were used to create hybrid systems, but not in the supercomputer realm. That has changed over the past several years. Both Nvidia and AMD have aggressively pushed the use of their GPUs as accelerators in supercomputers to help ramp up both the performance and energy efficiency of the systems, and more supercomputers are leveraging the GPU accelerators in their designs.
Intel in November 2012 unveiled the first of its Xeon Phi coprocessors, x86-based chips that like the GPU accelerators work with CPUs to help supercomputers run compute-intensive and highly parallel workloads. Intel executives have argued that the Xeon Phi coprocessors enable programmers to work with familiar x86-based tools and that most workloads already are optimized for x86, giving them an advantage over the GPUs from Nvidia and AMD. However, officials with those vendors argued that Intel’s designs lag behind what Nvidia and AMD offer, including such areas as efficiency.
IBM’s Roadrunner Supercomputer Is Retired
What IBM engineers and the researchers at the Los Alamos National Lab were able to do with Roadrunner helped drive the use of coprocessors and accelerators and inform how new systems are being designed, particularly as the industry begins looking to exascale computing—1 quintillion calculations per second, which is 1,000 times faster than Roadrunner—by 2020, though Intel officials have said it could happen by 2018.
“Even in death, we are trying to learn from Roadrunner,” the lab’s Grider said.
Roadrunner, which hit a peak of 1.45 petaflops, was listed as the 22nd-fastest system on the November 2012 list of the world’s 500 fastest supercomputers. It was knocked from the top of the list in November 2009 by Cray’s Jaguar system.
However, compared with newer systems, it consumed a lot of energy. According to a report in VentureBeat, Roadrunner consumed 2.3 kilowatts to hit its top operating speeds, while the system that will replace it uses much less power and space. Los Alamos researchers said the newer supercomputers will have to become much more energy-efficient than Roadrunner and better at handling and storing large amounts of data.
The supercomputer was used as part of the National Nuclear Security Administration’s (NNSA) Advanced Simulation and Computer program, running computer simulations of weapons stockpiles for the Stockpile Stewardship Program. That program was part of the United States’ nuclear deterrent, according to Los Alamos.
“Roadrunner exemplified stockpile stewardship: an excellent team integrating complex codes with advanced computing architectures to ensure a safe, secure and effective deterrent,” Chris Deeney, NNSA’s assistant deputy administrator for stockpile stewardship, said in a statement. “Roadrunner and its successes have positioned us well to weather the technology changes on the HPC horizon as we implement stockpile modernization without recourse to underground testing.”
It also was used in other areas, including nanowire material behavior, laser backscatter and a simulation of the universe at a 70-billion-particle scale.