IBM Big Data Platform Adds Hadoop, Analytics Advancements

IBM announced two new technologies to help users better wrangle big data: BLU Accelerator and the IBM PureData System for Hadoop.

IBM announced two new technologies to help enterprises tackle big data by making it simpler, faster and more economical to analyze massive amounts of data.

At an event at its Almaden Research Center in San Jose, Calif., on April 3, IBM announced BLU Acceleration and the IBM PureData System for Hadoop.

BLU Acceleration represents the work of hundreds of IBM developers and researchers in labs around the world and combines a number of techniques to dramatically improve analytical performance and simplify administration, IBM said. BLU Acceleration enables users to have much faster access to key information, leading to better decision making.

The software extends the capabilities of traditional in-memory systems—which enables data to be loaded into random access memory (RAM) instead of hard disks for faster performance—by providing in-memory performance even when data sets exceed the size of the memory. During testing, some queries in a typical analytics workload were more than 1,000 times faster when using the combined innovations of BLU Acceleration, IBM said.

Innovations in BLU Acceleration include "data skipping," which allows the ability to skip over data that doesn't need to be analyzed (for example, if there is duplicate information); the ability to analyze data in parallel across different processors; and greater ability to analyze data transparently to the application, without the need to develop a separate layer of data modeling. Another innovation in BLU Acceleration is called "actionable compression," where data no longer has to be decompressed to be analyzed.

BNSF Railway Company, one of the largest freight rail transportation networks in North America, is using IBM BLU Acceleration to more quickly understand the vast amounts of data from the organization's 1,700 servers that tracks maintenance, weather, scheduling, inventory, safety, deliveries and more. BNSF operates more than 1,400 trains a day on 32,500 route miles of track in 28 states and two Canadian provinces.

"BNSF transports many of the products and materials that we use every day in America and around the world, so tracking of these shipments is critical to our organization," Kent Collins, a database solutions architect at BNSF, said in a statement. "Working with IBM we are now embracing our organization's big data with the power of analytics. Thanks to the new technology, we're performing tasks more quickly than ever before, for example, one of the queries improved over hundredfold, and our storage consumption went down by about 10 times. One of the things that impressed us the most about BLU Acceleration is its simplicity. We just load the data and run queries."

"Big data is about using all data in context at the point of impact," Bob Picciano, general manager of IBM Information Management, said in a statement. "With the innovations we are delivering, now every organization can realize value quickly by leveraging existing skills as well as adopt new capabilities for speed and exploration to improve business outcomes."