Child of the Mother Ship
In June 2011, Yahoo (which created Hadoop) and Benchmark Capital formed a separate company called Hortonworks, named after the elephant character in Dr. Seuss. Though run by several former Yahoo employees, Hortonworks will remain independent of its mother ship from a business perspective and will develop with its own commercial edition. Yahoo CTO Raymie Stata, a key figure in this transition, is responsible for all IT development at the company. Even though Hadoop has moved to a new home, Stata told eWEEK that Yahoo doesn't consider the new company a "spinout."
"We will have more people within Yahoo working on Hadoop and related technologies than there will be at Hortonworks," Stata said. "We see this as increasing the investment that's being made in Hadoop.
Yahoo's Vision for Hadoop
Stata explained that Yahoo has always had a vision of Hadoop becoming the industry standard in big data analytics software, but it also knew Hadoop would have to establish its own business entity. One of the main reasons for creating Hortonworks, Stata said, is that Yahoo had already seen what the future holds for enterprise analytics (thanks to its six-year-long Hadoop development stage) and knew what would work. It saw that the need for big data analytics would soon become so widespread that a dedicated company would be necessary to focus solely on that-not on the advertising and Web services businesses that are Yahoo's meal ticket. "We have been running a truly enterprise deployment of Hadoop, and I don't think anybody does that: It's a departmental solution today," Stata said. "But it's not going to be six years before other people are doing enterprise [analytics as Yahoo does]. That gap between Yahoo and the rest of the user base is shrinking. "It's great to have an independent company that can have this relationship with Yahoo and see pain points that are on the road ahead. We now need to look at other customers, bring that input in and synthesize it with Yahoo's more futuristic view. Obviously, an independent company with a commercial mandate is going to do it a lot better than an open-source team inside Yahoo." "What we do on Hadoop ultimately creates value for our shareholders," Stata concluded. "If Hadoop becomes the de facto industry standard for big data processing, that's goodness for us. That's been our mission in being so open in the development of Hadoop. We're getting to the last mile now, and it's all set up to reach that stature."