Big data, and the ability to sift through and manage it, has become a core competitive advantage in nearly every industry, as successful companies are mining their data for information that might turn a series of seemingly random numbers and information into solid-gold profits.
Still, what does the term "big data" really mean? Is it something beyond a buzzword? Simply stated, big data is just that--all the data that is generated and comes into the realm of companies today. And it's growing.
IBM, one of the leading players in this emerging space, estimates that 2.5 quintillion bytes of data are created every day from a variety of sources, including sensors, social media and billions of mobile devices around the world, making it difficult for businesses to navigate and analyze it to improve competitiveness, efficiency and profitability.
IDC estimates the market for big data technology and services will grow at an annual rate of nearly 40 percent to reach $16.9 billion by 2015.
Moreover, every month people send 1 billion Tweets and post 30 billion messages on Facebook. Meanwhile, more than 1 trillion mobile devices are in use today and mobile commerce is expected to reach $31 billion by 2016.
Big data and analytics go hand-in-hand. Analytics is the discovery and communication of meaningful patterns in data. One of the most common uses of analytics is to mine business data to describe, predict and improve business performance.
Specific areas of analytics include enterprise decision management, retail analytics, marketing and Web analytics, predictive science, credit risk analysis and fraud analytics. Retail, marketing and customer management have been key areas of focus for leading vendors. For instance, IBM and Adobe have made major investments in marketing analytics. And SAS, another leader in the space, has focused on analytics of all types, including predictive analytics.
However, the field is still very much open to newcomers, with startups like Quant5 entering the market with their brand of analytics solutions and services.
"Industries furthest ahead in big data are Web/Internet, financial services, retail, some manufacturing and marketing departments across all industries," said Andrew Brust, a big data expert, analyst and consultant who founded Blue Badge Insights in New York City. "Web/Internet, retail and marketing are analyzing Weblog and social media data to determine browsing and buying patterns and sentiment. Financial services analyze market data, ostensibly for trading strategies and design of financial products. Manufacturing companies use so-called 'historian' data to monitor assembly line equipment and develop predictive models for when such equipment might fail."
Mark Pitts, director of science, solutions and strategy at United Healthcare (UHC), told eWEEK that the company uses analytics for a variety of purposes to serve its millions of customers, including quarterly forecasts, preventing fraud and customer service opportunities. In addition, his team uses the SAS High-Performance Analytics platform, which enables them to analyze more data and work in a massively parallel environment. As such, Pitts said his team was able to bring one particular process that used to take more than four hours down to 10 seconds.