SAN JOSE, Calif.—With any new technology there are likely concerns about whether it’s worth the investment in time, expense and deviation from the tools and infrastructure any organization is already familiar with.
But a panel of enterprise IT executives speaking at the Hadoop Summit here said they have no regrets about diving in with Hadoop to help manage their increasingly large pools of big data.
“There are always going to be the ‘it will never work people.’ I would say kill the fear. Haters to the left. Just go for it and get started,” said David T. Lin, leader and evangelist at Symantec.
Chris Dingle, director of customer intelligence at Canadian cable giant Rogers, said his company had a Hortonworks implementation of Hadoop going in five business days. “We started off by making a business use case, which really brings people’s time and energy and passion to the table,” said Dingle.
As a leading vendor of consumer and enterprise security products, Symantec has been dealing with big data for years with an urgent need to detect security threats as quickly as possible from among the hundreds of millions of endpoints it monitors. Lin says Symantec’s adoption of the Hadoop-based Hortonworks Data Platform has allowed it to scale its detection and remediation systems faster.
With its earlier big data analysis system, Lin said there were instances following a large-scale attack when its security system would get backed up and detection would take three to four hours. “That’s unacceptable. With Hadoop, the latency is down to seconds,” said Lin. “That means instead of customers waiting, protection comes in seconds. That’s an order of magnitude of change. And we’re confident as we scale out the response time will still be seconds.”
Telecommunications carrier Verizon is another company that’s using Hadoop to get a better handle on its big data needs. Rob Smith, executive director of IT at Verizon, said its long-established data warehouse system was too static in terms of not giving the company the latest customer insights.
For example, customer churn, where users sign up for a service, but quickly drop it, is a big deal for carriers. “When you have over 100 million customers, any small change is significant,” said Smith. With Hadoop, Smith says Verizon was able to do a click-stream analysis of social media and searches related to Verizon.
“All those data points have a connection to customer behavior and intention,” he said. With its Hadoop system in place, Verizon was able to create a more “holistic platform” of different data feeds that let the company identify complaints and other issues more quickly.
“We do iconic [product] launches, and now we can see sentiment and where there’s confusion and we’re able to make adjustments in real time and see the call center volumes go down,” Smith said.
Another company with well-established big data credentials is $50 billion oil and gas services giant Schlumberger. Anil Varma, vice president of data and analytics at Schlumberger, says it is still “early days” toward his ultimate goal of having all of the company’s business and operational data together in one place.
“We want to get a consistent view of everything—that’s the long-term vision,” said Varma. With its Hadoop implementation though, Varma feels things are headed in the right direction.
“We like Hadoop because it’s an integrated ecosystem that’s consistent no matter who queries it. You look at processes the same way as you look at business lines,” he said. “As you connect the activities of the company horizontally, your notion of a process expands beyond the business lines.”
In a later session, well-known business author and consultant Geoffrey Moore said Hadoop has grown a lot since he spoke at the conference a few years ago.
“I remember in 2012 everyone was like, ‘All we are saying is give Hadoop a chance'”—a twist of a Beatles lyric that drew a laugh. “It was a gathering of the faithful. Now we’re at the use case phase. We’re across the chasm,” he said, in reference to his best-selling tech marketing book Crossing the Chasm. “Perhaps not in the Tornado [another book reference, Inside the Tornado], but there are compelling use cases.”