Informatica, Hortonworks Team Up to Release Free Data Parser for Hadoop

By Chris Preimesberger  |  Posted 2011-11-02 Print this article Print

The parser turns unstructured complex data, such as Web logs, social media data, call detail records and other data formats, into a structured or semi-structured format in Hadoop.

Data integrator Informatica and fledgling Apache Hadoop developer Hortonworks jointly announced Nov. 2 that Informatica's HParser is now available free of charge to Hortonworks Data Platform users and available for download from the Hortonworks Website.

The data parser is also included as part of Informatica's freely downloadable Community Edition. Hortonworks is a Hadoop development spin-out from Yahoo that was launched last July at the annual Hadoop conference.

Big data workloads often contain numerous file formats, not all of which can be processed-or processed efficiently-by the Hadoop engine, which is designed to handle large batches of data. The HParser enables users "to extract meaning from data in any encoding and format, then process it to look for analytic patterns, or store it, and so on," Informatica Senior Vice President and General Manager Juan Carlos Soto told eWEEK.

"That step is typically considered parsing. It is then put into a format that is easier for you to process going forward."

Runs on Most Distributions of Hadoop

Informatica's HParser runs on virtually any distribution of Apache Hadoop. It uses the parallelism of the MapReduce framework to efficiently turn unstructured complex data, such as Web logs, social media data, call detail records and other data formats, into a structured or semi-structured format in Hadoop.

Once transformed into a more structured format, the data can be more rapidly used and validated to drive business insights and improve operations, Soto said.

HParser Community Edition provides users with an interactive tool to simplify and speed the data parsing and analytics process for some of the most often-used data types in Apache Hadoop, such as logs, Omniture, XML and JSON.

"Informatica is taking a critical step toward making Hadoop easier to use together with Hortonworks, a team that has contributed more than 80 percent of all the code in Apache Hadoop," Soto said. "We are empowering enterprises to turn big data into competitive advantage and ultimately maximize their 'return on data.'"

The HParser Community Edition includes a visual, drag-and-drop utility for quickly and easily creating parsing definitions.

Background on Hadoop

Apache Hadoop, open-source software built in Java that works with distributed data-intensive applications, enables applications to scale securely in order to handle thousands of nodes and petabytes of data. A number of companies now are using Hadoop daily to predict business patterns, find tendencies in scientific data and predict the weather, among many other functions. More and more businesses are finding out that they need to analyze their stored data and use those metrics to help them make better business decisions. Hadoop has certainly created the most buzz among new-generation big-data analytics packages.

Hadoop has gone from creator Doug Cutting's science project to mainstream business in a short time. Hortonworks was created during the last four months as an independent, privately held, VC-funded company to lead the Hadoop community and market the open-source product into the future. Officially, Mothership Yahoo is now one of its customers.

Hortonworks is an appropriate name for the new company because it is congruent with Hadoop itself-which is named after the stuffed toy elephant that belongs to Cutting's young son.


Chris Preimesberger Chris Preimesberger was named Editor-in-Chief of Features & Analysis at eWEEK in November 2011. Previously he served eWEEK as Senior Writer, covering a range of IT sectors that include data center systems, cloud computing, storage, virtualization, green IT, e-discovery and IT governance. His blog, Storage Station, is considered a go-to information source. Chris won a national Folio Award for magazine writing in November 2011 for a cover story on and CEO-founder Marc Benioff, and he has served as a judge for the SIIA Codie Awards since 2005. In previous IT journalism, Chris was a founding editor of both IT Manager's Journal and and was managing editor of Software Development magazine. His diverse resume also includes: sportswriter for the Los Angeles Daily News, covering NCAA and NBA basketball, television critic for the Palo Alto Times Tribune, and Sports Information Director at Stanford University. He has served as a correspondent for The Associated Press, covering Stanford and NCAA tournament basketball, since 1983. He has covered a number of major events, including the 1984 Democratic National Convention, a Presidential press conference at the White House in 1993, the Emmy Awards (three times), two Rose Bowls, the Fiesta Bowl, several NCAA men's and women's basketball tournaments, a Formula One Grand Prix auto race, a heavyweight boxing championship bout (Ali vs. Spinks, 1978), and the 1985 Super Bowl. A 1975 graduate of Pepperdine University in Malibu, Calif., Chris has won more than a dozen regional and national awards for his work. He and his wife, Rebecca, have four children and reside in Redwood City, Calif.Follow on Twitter: editingwhiz

Submit a Comment

Loading Comments...
Manage your Newsletters: Login   Register My Newsletters

Rocket Fuel