The worldwide ecosystem for Hadoop-MapReduce software is expected to grow at a compound annual rate of 60.2 percent, from $77 million in revenue in 2011 to $812.8 million in 2016, according to IDC.
IDC officials said they expect the Hadoop-MapReduce market to begin to grow rapidly as talent and tools for the wrangling of big data technology continue to improve. Hadoop MapReduce is a programming model and software framework for writing applications that rapidly process vast amounts of data in parallel on large clusters of compute nodes.
The Apache Hadoop software library is a framework that allows for the distributed processing of large data sets across clusters of computers using a simple programming model. It is designed to scale up from single servers to thousands of machines, each offering local computation and storage. MapReduce is a framework for processing highly distributable problems across huge datasets using a large number of computers. Hadoop is Apache’s free and open-source implementation of MapReduce.
“Hadoop and MapReduce are taking the software world by storm, inspiring a wide range of projects that collect both structured and unstructured data and producing output that may be used to answer a single question; serve as the foundation for a range of other questions, queries or searches; or be loaded into a data warehouse for more systematic and repeatable query,” Carl Olofson, program vice president for IDC’s Information Management software research, said in a statement.
“The Hadoop and MapReduce market will likely develop along the lines established by the development of the Linux ecosystem,” Dan Vesset, vice president of business analytics solutions at IDC, said in a statement. “Over the next decade, much of the revenue will be accrued by hardware, applications, and application development and deployment software vendorsboth established IT providers and startups, which in aggregate have raised more than $300 million in venture capital funding.”
IDC officials also said the firms research shows that one of the key drivers for the Hadoop-MapReduce market is the increasing availability of interactional, attitudinal and behavioral data from such sources as social media and other Web-based applications and a desire to mine such data for its full value.
In addition to the scarcity of tools and qualified staff, a factor expected to inhibit growth during the forecast period is the competition between open-source vendors and their closed-source counterparts, which may force lower license fees from the latter group, resulting in somewhat slower software revenue growth than would be the case if open-source projects did not represent so large a component of this market space, IDC said.
The study, “Worldwide Hadoop-MapReduce Ecosystem Software 2012-2016 Forecast,” examines the sales of software in support of, or based upon, MapReduce or Hadoop functions. The Hadoop-MapReduce ecosystem includes that subset of big data software that operates with or under Hadoop or MapReduce environments.