Hadoop Will Be Used More in Real-Time Applications

Hadoop’s capabilities now make it possible to stream data into the cluster and analyze it in an interactive fashion—both in real time. Hadoop was purpose-built for cost-effective analysis of data sets as enormous as the World Wide Web.

Hadoop has always been a good fit for applications that process massive amounts of data for predictive modeling and other analytics. More and more, organizations will use these applications to generate revenue by more accurately targeting—and in some cases adapting—products and services.

Hadoop will distance itself from MongoDB, Cassandra, Couchbase and the numerous NoSQL options to become the safe choice. In stark contrast to the fractured and niche-oriented nature of the alternatives, Hadoop offers a uniform approach to a broad set of APIs for big data analytics (including MapReduce, query languages and database access, with easy integration of leading analytic and search platforms) along with an expanding ecosystem to deliver a wide range of services.

The need for data scientists and operations personnel is growing fast, but it is not yet keeping up with the demand. A quick look at sites such as Dice.com, Monster, Glassdoor, Careerbuilder and others show a high number of data-scientist-type jobs open. Indeed.com noted that every year since 2008, the need for Hadoop administrators has eclipsed similar jobs by a very high percentage.

The talent pool with structured query language skills is well-established and will drive Hadoop’s support of SQL. SQL-like languages, such as HiveQL and DrQL, are examples of tools that are making Hadoop accessible to the large SQL-fluent community.

One application that is particularly well-suited for HBase is binary large objects (BLOB) stores. HBase is Hadoop’s open-source, nonrelational, distributed database modeled after Google’s BigTable and written in Java. These BLOBs require large databases with rapid retrieval. BLOBs typically are images, audio clips or other multimedia objects, and storing BLOBs in a database enables a variety of innovative applications. One example is a digital wallet—which enables users to upload their credit card images, checks and receipts for online processing; the technology eases banking, purchasing and lost-wallet recovery.

Harnessing not just the power of Hadoop but making its provisioning and integration in the corporate data center much more seamless will drive changes in the data center.

HBase is a large-scale, distributed database built on top of the Hadoop Distributed File System (HDFS). Facebook Messages, which combines messages, chat and email into a real-time conversation, is the first application in Facebook to use HBase in production. As a result, we will see more Hadoop deployments involved in lightweight online transaction processing (OLTP).

With the increased popularity and success of Hadoop, organizations will expand the applications and groups leveraging Hadoop clusters. Increased attention will be placed on multi-tenant features and the ability to share a cluster successfully across users and administrators.

Hadoop will become one of the killer apps for cloud adoption. The number of Hadoop clusters offered by cloud vendors is going through an intense uptick as organizations tap Hadoop as a killer application.