Facebook for several years has led the charge to rethink how data center resources are designed, going so far as launching the Open Compute Project in 2011 to encourage others to develop open-source models for designing servers, storage appliances and networking gear.
Now the social networking company, which runs massive data centers that process and store the huge amounts of data generated by its 1 billion members and its own internal operations, is unveiling a new homegrown networking module to link the hundreds of thousands of servers that populate their data centers, officials said.
The networking device, which Facebook engineers call the “6-pack” platform, leverages the top-of-rack switch—the “Wedge”—that the company designed and debuted in June 2014, along with the Linux-based operating system the company created for the Wedge, codenamed “FBoss.” The 6-pack is built with off-the-shelf technologies, and as with other data center devices developed by Facebook, is designed to enable Facebook to efficiently, cost-effectively and rapidly move data loads that are significantly larger than most other companies currently have to deal with.
It’s also an open design that is customizable and that can be leveraged by other organizations that run large data centers and need to manage huge amounts of data, presenting another challenge to traditional networking switch vendors likes Cisco Systems, Juniper Networks, Arista Networks and Extreme Networks. Facebook engineers, who have been wrestling with ways to become faster and more efficient over the years during the company’s rapid growth, have turned away from many of the systems offered by the major tech vendors and instead have developed designs in-house that fit their needs.
“As Facebook’s infrastructure has scaled, we’ve frequently run up against the limits of traditional networking technologies, which tend to be too closed, too monolithic, and too iterative for the scale at which we operate and the pace at which we move,” Facebook Yuval Bachar wrote in a post about the 6-pack on the company blog. “Over the last few years we’ve been building our own network, breaking down traditional network components and rebuilding them into modular disaggregated systems that provide us with the flexibility, efficiency, and scale we need.”
On the networking side, that initially led to the Wedge, a 1U (1.75-inch) switch that offered 16 ports of 40 Gigabit Ethernet. The 6-pack is a 6U (10.6-inch) design that can hold eight interface cards and includes 12 independent switching elements, with each element offering speeds of up to 1.28Tb/s. It can offer as many as 128 40GbE ports.
“Each element runs its own operating system on the local server and is completely independent, from the switching aspects to the low-level board control and cooling system,” Bachar wrote. “This means we can modify any part of the system with no system-level impact, software or hardware. We created a unique dual backplane solution that enabled us to create a non-blocking topology.”
Facebook Unveils Homegrown Networking Hardware
Facebook isn’t keeping the 6-pack to itself. It’s currently in production testing—along with the Wedge and FBoss—and the plan to offer the design to the Open Compute Project so that other organizations can build upon it for their own purposes.
“With ‘6-pack,’ we have created an architecture that enables us to build any size switch using a simple set of common building blocks,” Bachar wrote. “And because the design is so open and so modular—and so agnostic when it comes to switching technology and software—we hope this is a platform that the entire industry can build on.”
That creates another challenge for the likes of Cisco and Juniper, which have made billions of dollars selling their proprietary switches and which already are rapidly expanding their portfolios to address the burgeoning software-defined networking (SDN) and network-functions virtualization (NFV) trends. Cisco is pushing back with its Application Centric Infrastructure (ACI) and Nexus 9000 switches, and according to CEO John Chambers, the growth has been strong. During a conference call Feb. 11 regarding the company’s latest quarterly numbers, Chambers noted that Cisco now has 1,700 ACI and Nexus 9000 customers and that number of Nexus 9000 installed ports have passed 1 million in less than a year.
The CEO dismissed the threat of low-cost commodity systems to Cisco’s networking business, saying that most organizations are looking for a partner that can offer an architectural story that includes everything from security to collaboration.
“We are seeing no unusual competition in the market, no unusual competition with white label or white box, nor will we in the future,” Chambers said.
Some vendors, including Dell and Juniper, are offering open switches that can run third-party software from such companies as—in Dell’s case—Cumulus Networks, Big Switch Networks and Midokura. It’s a trend that Gartner analysts are calling “brite-box switching.”