MOUNTAIN VIEW, Calif.—With the growth of big data comes the challenge of analyzing and accessing the right information from among the huge stores of data today’s systems can collect. Companies typically employ data scientists to help in the effort, but a panelist here at the AlwaysOn Silicon Valley Innovation Summit said future growth is tied to smarter systems and less human involvement.
“The problem I see with Hadoop [systems that manage big data] is that it’s like bragging on an episode of [the TV show] ‘Hoarders’ how much you’ve saved,” said Arijit Sengupta, CEO of BeyondCore, which offers an automated system (BeyondCore for Office) for analyzing big data. “The speed of hoarding is getting faster, but the problem is that a human being is asking the questions and that’s the slow part.”
Sengupta compared the current state of big data systems to the early days of Google when the search giant had people categorizing Web search results. “It didn’t scale; the same thing is happening now with big data,” he said.
Stefan Groschupf, CEO of Datameer, which offers a big data analytics solution, agreed that advances need to be made, but he emphasized the need to give greater access to big data. “What we see with our banking and retail customers is that they are trying to get away from having data scientists locked in the basement, but instead to democratize the decisions,” he said. “The human brain is still the best at decision-making.”
Sengupta disagreed: “If humans make the decisions, we’ve lost the battle.”
When BeyondCore pitched Sears as a client, the retailer wanted to know how long it would take before they could expect to get any insights and asked for a demo using their own data. “We used their data, and they found customer insights in 10 minutes that they hadn’t been able to find in five months of analyzing their data,” said Sengupta.
The idea that broader and easier access to big data is needed was seconded by Sandhya Venkatachalam, a partner at venture firm Centerview Capital. “When we talk to customers about big data, we hear that IT is getting pressure from users on sales automation,” she said. “They want to get the next generation of BI [business intelligence] into hands of users quickly.”
If there was one area of agreement among panel members, it was that this is very much the early days of big data. “The market and use case for big data is immense, but the market is still very, very early,” said Mike Maciag, chief operating officer of Altiscale, which offers access to Hadoop big data systems as a cloud-based service.
Maciag disagreed with Sengupta that computer systems are ultimately best equipped to analyze big data and extract the value hidden within. “While we see machine learning driving a lot of this, human decision on top of that is absolutely critical,” Maciag said in a later interview with eWEEK.
“Machines are going to be human assistants for a long time,” he added.