Organizations Should Assess Whether They Need Hadoop

By Darryl K. Taft  |  Posted 2012-06-25 Print this article Print


However, Hadoop is the big dog in the big data world. Attendance at Hadoop-related events continues to grow. Cloudera, a leading Hadoop distributor and service provider, has hosted annual Hadoop World conferences to ever-increasing sold out crowds in New York. The event has outgrown two different venues.

And at the last Hadoop World last November, Accel Partners announced a $100 million fund to invest in big data companies. At the show, Ping Li, a partner at Accel, announced its Big Data Fund, calling it "incredibly important given the explosion of big data." The new initiative's goal is to fund transformative early-stage and growth companies throughout the big data ecosystem, from next-generation storage and data management platforms to a wide range of revolutionary software applications and services€”i.e., data analytics, business intelligence, collaboration, mobile, vertical applications and many more, Accel said.

€œAs organizations increasingly struggle to extract value from an ever-expanding sea of data, more and more of them are turning to Hadoop,€ said Stephen O'Grady, an analyst with RedMonk.

Yet despite its popularity, implementers and analysts alike agree that Hadoop needs help to become more palatable for the enterprise.

€œWe€™ve been working with customers to help them use Hadoop to solve various problems,€ said Mike Olson, CEO of Cloudera. €œHadoop on its own is not enough to tackle the big data analysis problems and other problems they face.€

In other words, it takes a village, Olson said.

€œBig data will require a big ecosystem to make its way into the enterprise,€ said Tony Baer, principal analyst at Ovum. €œEnterprises demand a marketplace of tools, skills and services to take advantage of Hadoop. Cloudera is leveraging its early jumpstart in the Hadoop market with an effective partnering program that is showing true results.€

€œAs interest in Hadoop expands from early adopters to mainstream enterprise and government users, we are increasingly seeing the focus shift from development and testing to understanding potential use cases for the core distribution to the value-added tools and services that will enable and accelerate enterprise adoption,€ said Matt Aslett, senior analyst at 451 Research.

€œThere€™s a lot of talk in the industry about making Apache Hadoop deployments easier; however, Cloudera€™s approach encompasses the entire lifespan of systems, not just the initial setup,€ Olson said. €œWorkloads shift, teams change and the types of questions you want to ask change over time. You should be able to easily manage your systems while your usage of Hadoop evolves and grows.€

€œI also work for a company betting on Hadoop, and we are helping clients be successful with it,€ Wampler said. €œI believe it is a good, but not great, first-generation technology that is meeting needs today. It's fair to say Hadoop is the only game in town. However, I also believe it will be replaced over time with technologies that are more modular, scalable and flexible to address the growing variety of big data applications, just as Spring eventually replaced EJB.€

Wampler reiterated that Hadoop reminds him of EJB in almost every way. And just as the Spring Framework brought an essential rethinking to Enterprise Java, there is an essential €œrethink€ that needs to happen in big data, he said. Wampler, an aficionado of the Scala functional programming language, said the Scala community is well-positioned to create change in the big data world.

€œI think we should stop using Java,€ he said. €œThe object-oriented model is not really the right approach for data. We can take our existing Java code and we can start writing Clojure or Scala or JRuby. I€™m not slamming the JVM [Java Virtual Machine], I€™m slamming the language.€

Darryl K. Taft covers the development tools and developer-related issues beat from his office in Baltimore. He has more than 10 years of experience in the business and is always looking for the next scoop. Taft is a member of the Association for Computing Machinery (ACM) and was named 'one of the most active middleware reporters in the world' by The Middleware Co. He also has his own card in the 'Who's Who in Enterprise Java' deck.

Submit a Comment

Loading Comments...
Manage your Newsletters: Login   Register My Newsletters

Rocket Fuel