Hadoop Striving for Maturity, Credibility in the Enterprise
The discussion surrounding YARN was a big piece of the Hadoop summit as it should have been. The ability to add an enterprise-class resource manager allows for numerous applications to be created from the data (the "data lake" as one keynoter described it) contained in a corporation. The ultimate goal for companies is to merge their traditional row- and column-based data (which, while an older model, still has many of the security and privacy features still under development in Hadoop) with the massively scalable social and machine data within Hadoop. As one presenter explained, financial systems can track sales but Hadoop systems can uncover the customer sentiment surrounding those sales. The two systems in tandem would represent a major step forward in corporate computing. Hadoop is an open-source project overseen by the Apache Software Foundation, and in addition to YARN, there are ambitious projects, including the Knox security project. The Hadoop 2.0 version is still in beta format, but the nature of open-source development will see this project move, maybe not as fast as some would like, but transparently. Hortonworks, spun out of Yahoo in 2011 and the recent recipient of an additional $50 million in financing, is the most vocal advocate of Hadoop being implemented in its "purest" open-source form. The combination of new features in Hadoop and an increase in applications that can give customers a strategic advantage based on melding old and new systems could be just the thing to turn the intractable third into the "let's get going on the next big thing" category.
Eric Lundquist is a technology analyst at Ziff Brothers Investments, a private investment firm. Lundquist, who was editor in chief at eWEEK (previously PC WEEK) from 1996-2008, authored this article for eWEEK to share his thoughts on technology, products and services. No investment advice is offered in this article. All duties are disclaimed. Lundquist works separately for a private investment firm, which may at any time invest in companies whose products are discussed in this article and no disclosure of securities transactions will be made.