Whole Sets, Instead of Subsets

By Chris Preimesberger  |  Posted 2011-08-15 Print this article Print

Historically, data analytics software hasn't had the capability to take a large data set and use all of it-or at least most of it-to compile a complete analysis for a query. Instead, it has relied on representative samplings, or subsets, of the information to render these reports, even though analyzing more information produces more accurate results.

That approach is changing with the emergence of new big data analytics engines, such as Apache Hadoop, LexisNexis' HPCC Systems and 1010data's cloud-based analytics service. These new platforms are causing "the disappearing role of summarization," said Tim Negris, senior vice president of 1010data, a cloud-based data analytics provider. "With regard to big data, it's one thing to just suck [data] in and put it somewhere, but it's quite another thing to actually make use of it.

"One of the barriers to this is that most of the database makers, like Oracle and others, require a good deal of work [to prepare the data] prior to actually doing anything with it. We eliminate that and put the data directly in the hands of the analysts."

Hadoop and HPCC Systems do that, as well. All three platforms provide complete looks at big data sets. Instead of a team of analysts spending days or weeks preparing the parameters for data subsets, and then taking 1, 2 or 10 percent samplings, all the data can be analyzed at one time, in real time.

Why bother? Because data sitting in storage arrays and cloud accounts represents unrefined value in its most basic form. If interpreted properly, the stories, guidelines and essential information buried in storage and databases can open the eyes of business executives as they make strategic decisions for their company.

Management consultant Management consultant and venture capitalist Peter Cohan, president of Peter S. Cohan & Associates and a faculty member at Babson College in Wellesley, Mass., recently gave a cogent example of this in a Forbes article: Walmart wanted to find out the biggest-selling items people bought before a hurricane hits.

The No. 1 answer-batteries- was not a surprise. But the unexpected No. 2 item was Kellogg's Pop-Tarts. It turns out that those sugar-boost pastries are great for emergencies: They last a long time, don't require refrigeration or preparation, and are easy to carry and store.

As a result of this intelligence, Walmart can now stock up on Pop-Tarts in its Gulf Coast stores ahead of storm season. This is where the reach of new-generation business analytics tools shine: by directly helping enterprises make smart decisions.

Chris Preimesberger Chris Preimesberger was named Editor-in-Chief of Features & Analysis at eWEEK in November 2011. Previously he served eWEEK as Senior Writer, covering a range of IT sectors that include data center systems, cloud computing, storage, virtualization, green IT, e-discovery and IT governance. His blog, Storage Station, is considered a go-to information source. Chris won a national Folio Award for magazine writing in November 2011 for a cover story on Salesforce.com and CEO-founder Marc Benioff, and he has served as a judge for the SIIA Codie Awards since 2005. In previous IT journalism, Chris was a founding editor of both IT Manager's Journal and DevX.com and was managing editor of Software Development magazine. His diverse resume also includes: sportswriter for the Los Angeles Daily News, covering NCAA and NBA basketball, television critic for the Palo Alto Times Tribune, and Sports Information Director at Stanford University. He has served as a correspondent for The Associated Press, covering Stanford and NCAA tournament basketball, since 1983. He has covered a number of major events, including the 1984 Democratic National Convention, a Presidential press conference at the White House in 1993, the Emmy Awards (three times), two Rose Bowls, the Fiesta Bowl, several NCAA men's and women's basketball tournaments, a Formula One Grand Prix auto race, a heavyweight boxing championship bout (Ali vs. Spinks, 1978), and the 1985 Super Bowl. A 1975 graduate of Pepperdine University in Malibu, Calif., Chris has won more than a dozen regional and national awards for his work. He and his wife, Rebecca, have four children and reside in Redwood City, Calif.Follow on Twitter: editingwhiz

Submit a Comment

Loading Comments...
Manage your Newsletters: Login   Register My Newsletters

Rocket Fuel