Hadoop to Shine in Big Data's Next Phase, Predictive Analytics

Future uses and the adoption of Apache Hadoop were the focus of several keynotes kicking off the Hadoop Summit June 9.

Hadoop Futures 2

SAN JOSE, Calif.-- Over the next few years, Hadoop will be adopted by virtually all enterprises and become just as much a part of the core IT infrastructure as database management systems, according to Mike Gualtieri, principal analyst at Forrester Research.

“It’s a data operating system and a fundamental data platform that in the next couple of years 100 percent of large companies will adopt,” Gualtieri predicted Tuesday at the Hadoop Summit here.

Gualtieri and other speakers said Hadoop adoption will continue to grow rapidly because it “democratizes big data problem solving” and is scaling to offer new capabilities and insights into increasingly large data stores.

Enterprises already are reaping benefits from big data analysis with its ability to extract insights from historical data, such as what customers buy, where they buy it and when. But new tools, powered by Hadoop, are giving enterprises the ability to also predict what customers will buy, greatly enhancing the value of the data to anticipate customer needs and business trends.

Several speakers here emphasized the power of predictive analytics in a variety of applications, including commercial and industrial.

Vince Campisi, CIO of GE Software, said Hadoop is going to help unlock a new ‘Industrial Internet’ that will help manufacturers anticipate when, for example, components are likely to break, so they can replace them ahead of time to save millions of dollars in downtime. GE already offers some of this capability with its own Predictivity software, he noted.

“Hadoop breaks down the data silos. We see the walls coming down,” said Campisi. He said that current systems aren’t designed to handle what he estimates will be 50 billion devices connected to the Internet of Things, with each one generating data. But Hadoop can scale to meet such immense data volumes and offer insights into relationships that “we didn’t even know mattered,” Campisi said.”

One of the architects of Hadoop, Arun Murthy said the rise of big data has led to the same concerns companies had back in the days when mainframes controlled most computer operations, such as management, reliability, security and quality of service. Now Hadoop-powered systems, such as the Hortonworks Enterprise Data Platform, are designed to address those same concerns in the cloud, noted Murthy, who is also a co-founder of Hortonworks.

Murthy said that computing has to speed up for predictive analytics to be effective. “We see speeds and feeds going faster with the need to take snapshots of what’s happening now in micro-seconds,” he said.

Proactive Data Analysis

Typically companies have used a big data system to discern things like when and where someone bought an airplane ticket. In this new era of predictive analytics, Murthy says, companies are becoming more proactive. Today, credit card companies will call when a customer makes a purchase that the system deduces is outside their normal buying pattern and thus, might be fraudulent.

“Hadoop is part of what makes that possible,” he said. Other predictive analysis applications will take hold in the near future, he said. For example, a system might proactively notify a truck driver of a traffic jam that might slow his route. Such a system could also show alternative routes to keep deliveries on schedule.

David Needle

David Needle

Based in Silicon Valley, veteran technology reporter David Needle covers mobile, bi g data, and social media among other topics. He was formerly News Editor at Infoworld, Editor of Computer Currents...