Big Data Analysis Takes a Big Bite Out of HPC Workloads: IDC

A high proportion of high-performance computing sites are analyzing big data, while demand for co-processors and accelerators has doubled.

New research from IDC reveals that two-thirds of high-performance computing sites are conducting big data analysis as part of their HPC workloads. The research also finds that HPC sites using co-processors and accelerators more than doubled during the past two years.

One of the biggest findings of the 2013 Worldwide Study of HPC End-User Sites shows that the proportion of sites (representing 905 HPC systems) using co-processors or accelerators in their HPC systems climbed from 28.2 percent in the 2011 to 76.9 percent in 2013.

IDC said co-processors/accelerators grew slightly more than 1 percent of all processor types in 2011 to 3.4 percent in 2013. Intel Xeon Phi co-processors and Nvidia graphics processing units (GPUs) led the pack, followed by field-programmable gate arrays (FPGAs) in the No. 3 position. However, IDC noted that the use of these newer devices is often for exploratory purposes.

"The most surprising finding of the 2013 study is the substantially increased penetration of co-processors and accelerators at HPC sites around the world, along with the large proportion of sites that are applying big data technologies and methods to their problems," Earl Joseph, IDC's program vice president for Technical Computing, said in a statement.

An unexpected finding reveals that 67 percent of the HPC sites perform big data analysis on their HPC systems, with 30 percent of the available computing cycles devoted on average to big data analysis. Since IDC did not ask about big data analysis in the 2011 study, a comparison could not be made.

The report also indicates that the HPC sites using cloud computing to handle part of the HPC workloads climbed from 13.8 percent in 2011 to 23.5 percent in 2013. Public and private cloud use was split among the sites in 2013.

IDC said the study confirmed its supply-side research finding that storage is the fastest-growing technology area at HPC sites.

The HPC end-user study consists of six reports that cover processors and co-processors/accelerators, storage and interconnects, applications software, systems software and operating systems, big data analysis, and cloud computing.