CERNs own network upgrade in support of the Large Hadron Collider project involved more than just connecting the 8,000 PC processors it is using at its data center.
Beyond those “batch worker nodes”—or PC processors—the Hewlett-Packard ProCurve switches will also link to high-end tape drives where the data will be stored and to a buffer layer between the processors and drives made up of PCs with integrated disks, according to Helge Meinhard, technical coordinator for server procurements for CERN in Geneva.
Some 2,000 tape and disk servers are connected via ProCurve 3400 and 3500 series switches at Gigabit Ethernet speeds, along with the PC processors also connected at Gigabit Ethernet speeds.
“For the disk servers and tapes, we have one switch connected [at 10 Gigabit Ethernet speeds] to a single router port,” Meinhard said.
Any switch connected directly to a Force10 TeraScale E-Series router uses a 10G-bps link. And the routers—about 16 in all—are connected to each other over 10G-bps links.
“For the CPU servers, we have lower bandwidth requirements, so we daisy chain four 48-port ProCurve switches and connect an aggregate of four switches to a single 10 Gigabit link into the router. We connect up to 188 CPU worker nodes to a single 10- Gig uplink,” Meinhard said.
The Force10 routers also connect CERN to the Tier 1 data centers in the grid at 10G bps. Overall, the CERN network upgrade includes some 600 ProCurve 3400cl, 400 ProCurve 3500yl and 20 ProCurve 5400-series switches.
The nonblocking router core can handle an aggregate bandwidth of about 2.4 terabits per second. Eight Force10 E1200 routers are configured with 40 10G bps ports to form the core mesh for the PC processor farm and link to two more E1200s that make up an aggregation layer.
Two Force10 E600s with 10G-bps WAN interfaces provide transatlantic links to sites in the United States, and other E1200s connect to European or Asian Tier 1 data centers in the grid.
Check out eWEEK.coms for the latest news, views and analysis on servers, switches and networking protocols for the enterprise and small businesses.