Child of the Mother Ship

By Chris Preimesberger  |  Posted 2011-08-15 Print this article Print

In June 2011, Yahoo (which created Hadoop) and Benchmark Capital formed a separate company called Hortonworks, named after the elephant character in Dr. Seuss. Though run by several former Yahoo employees, Hortonworks will remain independent of its mother ship from a business perspective and will develop with its own commercial edition.

Yahoo CTO Raymie Stata, a key figure in this transition, is responsible for all IT development at the company. Even though Hadoop has moved to a new home, Stata told eWEEK that Yahoo doesn't consider the new company a "spinout."

"We will have more people within Yahoo working on Hadoop and related technologies than there will be at Hortonworks," Stata said. "We see this as increasing the investment that's being made in Hadoop.

"We're taking some of our key talent and using it to seed Hortonworks, so some employees will be moving from Yahoo to the new company. But this is not downsizing, and it's not a spinout. It's increasing the investment in Hadoop. Yahoo will continue to be a major contributor to all aspects of Hadoop going forward."

Yahoo's Vision for Hadoop

Stata explained that Yahoo has always had a vision of Hadoop becoming the industry standard in big data analytics software, but it also knew Hadoop would have to establish its own business entity.

One of the main reasons for creating Hortonworks, Stata said, is that Yahoo had already seen what the future holds for enterprise analytics (thanks to its six-year-long Hadoop development stage) and knew what would work. It saw that the need for big data analytics would soon become so widespread that a dedicated company would be necessary to focus solely on that-not on the advertising and Web services businesses that are Yahoo's meal ticket.

"We have been running a truly enterprise deployment of Hadoop, and I don't think anybody does that: It's a departmental solution today," Stata said. "But it's not going to be six years before other people are doing enterprise [analytics as Yahoo does]. That gap between Yahoo and the rest of the user base is shrinking.

"It's great to have an independent company that can have this relationship with Yahoo and see pain points that are on the road ahead. We now need to look at other customers, bring that input in and synthesize it with Yahoo's more futuristic view. Obviously, an independent company with a commercial mandate is going to do it a lot better than an open-source team inside Yahoo."

"What we do on Hadoop ultimately creates value for our shareholders," Stata concluded. "If Hadoop becomes the de facto industry standard for big data processing, that's goodness for us. That's been our mission in being so open in the development of Hadoop. We're getting to the last mile now, and it's all set up to reach that stature."

Chris Preimesberger Chris Preimesberger was named Editor-in-Chief of Features & Analysis at eWEEK in November 2011. Previously he served eWEEK as Senior Writer, covering a range of IT sectors that include data center systems, cloud computing, storage, virtualization, green IT, e-discovery and IT governance. His blog, Storage Station, is considered a go-to information source. Chris won a national Folio Award for magazine writing in November 2011 for a cover story on and CEO-founder Marc Benioff, and he has served as a judge for the SIIA Codie Awards since 2005. In previous IT journalism, Chris was a founding editor of both IT Manager's Journal and and was managing editor of Software Development magazine. His diverse resume also includes: sportswriter for the Los Angeles Daily News, covering NCAA and NBA basketball, television critic for the Palo Alto Times Tribune, and Sports Information Director at Stanford University. He has served as a correspondent for The Associated Press, covering Stanford and NCAA tournament basketball, since 1983. He has covered a number of major events, including the 1984 Democratic National Convention, a Presidential press conference at the White House in 1993, the Emmy Awards (three times), two Rose Bowls, the Fiesta Bowl, several NCAA men's and women's basketball tournaments, a Formula One Grand Prix auto race, a heavyweight boxing championship bout (Ali vs. Spinks, 1978), and the 1985 Super Bowl. A 1975 graduate of Pepperdine University in Malibu, Calif., Chris has won more than a dozen regional and national awards for his work. He and his wife, Rebecca, have four children and reside in Redwood City, Calif.Follow on Twitter: editingwhiz

Submit a Comment

Loading Comments...
Manage your Newsletters: Login   Register My Newsletters

Rocket Fuel