eWEEK content and product recommendations are editorially independent. We may make money when you click on links to our partners. Learn More.
IBM has upgraded its BigInsights big data portfolio with new support for machine learning, the R programming language and more.
With the release of IBM BigInsights for Apache Hadoop, Big Blue provides new In-Hadoop analytics technologies to accelerate the conversion of data into valuable insight for businesses.
As the size and complexity of Hadoop applications continue to grow, data science has become a key function for unlocking meaningful insights and identifying actions that could optimize outcomes across an entire business. IBM BigInsights for Apache Hadoop features a broad data science toolset to query data, visualize, explore and conduct distributed machine learning at scale.
For example, a business analyst who needs to quickly find relevant information or the data scientist who needs to make sense of the data with statistical modeling needs the whole environment to be easy for IT to manage and deploy for everyone to use.
“There’s so much data out there that it’s often difficult for companies to find the information that really matters,” said Anand Mahurkar, CEO of Findability Sciences, an IBM BigInsights customer. “IBM BigInsights helps connect data elements that often go undiscovered to bring context to the customer relationship. Some of our clients have improved customer retention by up to 25 percent with this enhanced understanding. They can now focus efforts and resources on where they’ll have the greatest effect, helping win and retain more business and drive more profitable operations.”
IBM BigInsights for Apache Hadoop will introduce three new modules, including the IBM BigInsights Analyst, which will include IBM’s SQL engine and the company’s intuitive spreadsheet and visualizations to find data quickly and easily. On average, millions of SQL queries are run each year. With BigInsights Analyst, the efficiency of these queries has been shown in some cases to improve by up to four times on Apache Hadoop depending on the shuffle size. And the use of ANSI-compliant SQL means queries can run unchanged against Hive, HBase and relational databases.
The software also features IBM BigInsights Data Scientist, which will deliver a new machine-learning engine that automatically tunes its performance over large-scale data to find interesting patterns—plus over a dozen industry-specific algorithms such as Decision Trees, PageRank and Clustering to help tackle complex problems out of the box. It will also provide native support for open-source R statistical computing, helping clients leverage their existing R algorithms or gain from the more than 4,500 freely available statistics packages from the R community.
Finally, IBM BigInsights for Apache Hadoop features IBM BigInsights Enterprise Management, which will introduce new management tools designed to help allocate resources and optimize workflows. These tools will allow deployments that can scale to large numbers of users and clusters, and will help satisfy high workload demand. The tools also will provide multi-tenancy and multi-instance support in a cluster.
IBM also announced the IBM Open Platform with Apache Hadoop, a new platform based on open-source software that will provide the necessary data access controls and authentication for an enterprise. Plus, IBM added support for Apache Spark to enable new computing engines to facilitate interactive analytic applications.
Big Data University
IBM is sponsoring Big Data University and its new courses for programming for analytics and data science that cover machine learning. Big Data University is an effort to enhance big data skills by delivering free online courses to a community of more than 230,000 registered participants around the world.
IBM also is a founding member of the Open Data Platform (ODP) Initiative, a new industry association to help drive collaboration, innovation and standardization across Hadoop and big data technologies.
“In our fast paced world, the ability to turn data into insight is the difference between success and failure,” said Beth Smith, general manager of the IBM Analytics Platform, in a statement. “With these announcements, our ability to address the sophisticated needs of data scientists, to improve access for a broader community of analysts, and the surety of production scale will set in motion the next wave of value our clients expect from big data.”
IBM’s Hadoop solutions fit into the company’s broader analytics platform to help deliver insight from data when, where and how it is needed. In particular, the predictive capabilities of IBM’s SPSS offering can build the predictive models, exercise machine learning and R in Hadoop, apply business optimization and deploy these models into real-time business processes.