Yes, the phenomenon of so-called "big data" has clearly made it to the buzzword level. Sometimes truly important technology trends, in fact, do become buzzwords.
To begin to qualify this, we can look at the projected numbers from the analysts—which, of course, are themselves computed by using big data sets. Big data storage (and the analytics processing that uses all that raw data) is a burgeoning market that IDC, for one, has forecast to surge at a 40 percent compound annual growth rate, from $3.2 billion in 2010 to $17 billion in 2015. The hockey-stick curve will certainly continue to zoom up-and-to-the-right after that.
Microsoft recently conducted a big data survey of its enterprise customers which found that almost two-thirds (62 percent) of respondents said they currently store at least 100 terabytes of data. Take a look at how much data that you, as a single consumer, store; it's a safe bet that it is way more than you might have imagined. Then think of how much data enterprises have to manage on a daily basis or even an annual basis.
Big data presents a formidable new frontier in which data sets can grow so large that they become awkward to work with using traditional database management tools. The need for new tools, frameworks, hardware, software and services to handle this emerging issue represents a huge market opportunity.
One big factor that is driving the relentless growth of big data volumes is the incredible explosion of connected devices in the marketplace.
For example, Apple sold 51 million new iPhones in the last quarter of 2013 alone, and that represents only 15 percent of the worldwide market. Samsung owns 31 percent, thanks to its sale of 103 million Android phones. Sales of video cameras and sensors for machine data generation in enterprises are also off the charts. All these devices are generating huge volumes of data that have to be stored and managed.
Here are some good stats to consider: There are upwards of 6 billion people living on Earth. There are now an estimated 4 billion connected devices in the world. The market saturation of all those devices is continually producing data at nearly unfathomable rates.
There are immense new business opportunities in big data. There is demand for effective big data toolsets that provide scalable, high-performance analytics at the lowest cost and in near-real time as business users increasingly demand continuous access to data. By analyzing this data, companies gain greater intelligence as well as a competitive advantage.
eWEEK covers this beat nearly every day in some form or fashion, whether it's from the storage hardware and software angle—by far the healthiest overall sector in all of IT—or from cloud services, or from analytics, or from innovation.