Hadoop, Java Combo Top of Mind for Hiring Managers

by Darryl K. Taft

Java is the leading skill hiring managers look for in combination with Hadoop, according to Dice’s Open Web. Java remains immensely popular and is the second most popular language in the TIOBE Programming Community Index for May 2013. Hadoop and the Hadoop Distributed File System (HDFS) are written in Java.

Dice’s Open Web identifies “developer” as the second most-sought-after skill in addition to Hadoop. Hadoop developer training also is in high demand. For instance, Cloudera offers a popular training course on Hadoop development.

Dice’s Open Web ranks NoSQL skills as the third most-sought-after skill set in addition to Hadoop. Professionals with Hadoop and NoSQL experience pulled in more than $100,000 a year on average.

MapReduce skills are the fourth most-sought-after in addition to Hadoop by hiring managers, according to Dice. MapReduce, initially developed at Google, is a programming model for processing large data sets with a parallel, distributed algorithm on a cluster. Hadoop is a popular free implementation of the MapReduce model.

The catchall term of big data skills came in fifth as the most popular among hiring managers in addition to Hadoop. Big data is a collection of data sets so large and complex that it becomes difficult to process using on-hand database management tools or traditional data processing applications. The challenges include capture, curation, storage, search, sharing, transfer, analysis and visualization.

Demand for professionals experienced with the Pig platform ranked sixth among hiring managers looking for Hadoop experts. Apache Pig is a platform for analyzing large data sets that consists of a high-level language for expressing data analysis programs, coupled with the infrastructure for evaluating these programs. Pig’s structure is amenable to substantial parallelization, which helps in handling very large data sets. Pig’s infrastructure layer consists of a compiler that produces sequences of Map-Reduce programs.

The seventh most-sought-after skill in combination with Hadoop was Linux. Linux is the foundation of many big data efforts, including those at SAP HANA, Hortonworks, WANDisco, Teradata and IBM’s Watson to name a few.

The combination of Python programming skills and Hadoop is the eighth most popular among hiring managers, according to the Dice report. Python is a programming language that lets IT professionals work more quickly and integrate systems more effectively. Python supports multiple programming paradigms, including object-oriented, imperative and functional programming styles.

Coming in ninth among hiring managers was the combination of Hadoop and Hive. Hive is a data warehouse system for Hadoop that facilitates easy data summarization, ad hoc queries and the analysis of large datasets stored in Hadoop-compatible file systems. Hive provides a mechanism to project structure onto this data and query the data using a SQL-like language called HiveQL. At the same time, this language also allows traditional map/reduce programmers to plug in their custom mappers and reducers when it is inconvenient or inefficient to express this logic in HiveQL.

The combination of Scala programming skills and Hadoop rounded out the list of the top 10 most-sought-after skill sets, according to Dice. Scala is a general-purpose programming language designed to express common programming patterns in a concise, elegant and type-safe way. It smoothly integrates features of object-oriented and functional languages, enabling Java and other programmers to be more productive.