How Facebook Is Handling All That Really Big Data

 
 
By Chris Preimesberger  |  Posted 2012-09-17 Email Print this article Print
 
 
 
 
 
 
 

Facebook designs its own servers and networking. It designs and builds its own data centers. Its staff writes most of its own applications and creates virtually all of its own middleware. Everything about its operational IT unites it in one extremely large system that is used by internal and external folks alike.

MENLO PARK, Calif. -- Facebook is much like the Starship Enterprise in that it likes to go to places no company has gone before.

This is probably because not too many IT companies, especially young ones, have had to serve upwards of 950 million registered users -- including a high percentage on a real-time basis -- daily. Not many have to sell advertising to about 1 million customers or have dozens of new products in the works, all at the same exact time.

Facebook, which has a clear do-it-yourself IT approach, also designs its own servers and networking. It designs and builds its own data centers. Its staff writes most of its own applications and creates virtually all of its own middleware. Everything about its operational IT unites it in one extremely large system that is used by internal and external folks alike.

For example, Facebook's human resources group, the accounting office, Mark Zuckerberg on email and even you at your laptop checking your status are all using exactly the same gigantic, amorphous data center system that circles the globe in its power and scope.

Everything Facebook Does Involves Big Data

"So just about everything we do turns out to be a big data problem," said Jay Parikh, vice president of Infrastructure Engineering at Facebook, who spoke recently to a small group of journalists at the company headquarters. "This affects every layer of our stack. We've talked with some of you about the servers, storage, networking and the data center, as well as all the software, the operations, the visibility, the tools -- it all comes together in this one application that we have to provide to all our users."

Big data simply is about having insight and using it to make impact on your business, Parikh said.

"It's really very simplistic. If you aren't taking advantage of the data you are collecting and being kept in your business, then you just have a pile of a lot of data," Parikh said. "We are getting more and more interested in doing things with the data we are collecting."

Facebook doesn't always know what it wants to do with the user lists, Web statistics, geographic information, photos, stories, messages, Web links, videos and everything else that the company collects, Parikh said. "But we want to collect everything, we want to instrument everything: cameras, when that door opens and closes, the temperature in this room, who walks in and out the lobby.

"We want to know who visits the site, what activities they do, where they do it on the site. So everything is interesting to us," he said.



 
 
 
 
Chris Preimesberger Chris Preimesberger was named Editor-in-Chief of Features & Analysis at eWEEK in November 2011. Previously he served eWEEK as Senior Writer, covering a range of IT sectors that include data center systems, cloud computing, storage, virtualization, green IT, e-discovery and IT governance. His blog, Storage Station, is considered a go-to information source. Chris won a national Folio Award for magazine writing in November 2011 for a cover story on Salesforce.com and CEO-founder Marc Benioff, and he has served as a judge for the SIIA Codie Awards since 2005. In previous IT journalism, Chris was a founding editor of both IT Manager's Journal and DevX.com and was managing editor of Software Development magazine. His diverse resume also includes: sportswriter for the Los Angeles Daily News, covering NCAA and NBA basketball, television critic for the Palo Alto Times Tribune, and Sports Information Director at Stanford University. He has served as a correspondent for The Associated Press, covering Stanford and NCAA tournament basketball, since 1983. He has covered a number of major events, including the 1984 Democratic National Convention, a Presidential press conference at the White House in 1993, the Emmy Awards (three times), two Rose Bowls, the Fiesta Bowl, several NCAA men's and women's basketball tournaments, a Formula One Grand Prix auto race, a heavyweight boxing championship bout (Ali vs. Spinks, 1978), and the 1985 Super Bowl. A 1975 graduate of Pepperdine University in Malibu, Calif., Chris has won more than a dozen regional and national awards for his work. He and his wife, Rebecca, have four children and reside in Redwood City, Calif.Follow on Twitter: editingwhiz
 
 
 
 
 
 
 

Submit a Comment

Loading Comments...

 
Manage your Newsletters: Login   Register My Newsletters























 
 
 
 
 
 
 
 
 
 
 
Rocket Fuel