Data Storage Demands Complicated by Data Variety
A survey of data scientists reveals it is not only the volume of data but the different types of data collected that is causing storage headaches.Nearly three-quarters (71 percent) of data scientists said big data had made their analytics more difficult and data variety, not volume, was to blame, according to a survey of 111 data scientists from computational database management system designer Paradigm4. The survey results indicated that it is the diverse types of data, not the volume, that's a bigger challenge to data scientists and is causing them to leave data on the table. The report also revealed 39 percent said their job had become more stressful with the growth of big data, while nearly half of data scientists (49 percent) said they're finding it more difficult to fit their data into relational database tables. "The increasing variety of data sources is forcing data scientists into shortcuts that leave data and money on the table," Marilyn Matz, CEO of Paradigm4, said in a statement. "The focus on the volume of data hides the real challenge of data analytics today. Only by addressing the challenge of utilizing diverse types of data will we be able to unlock the enormous potential of analytics."
The vast majority (91 percent) said they're using complex analytics on their big data now or plan to within the next two years, although the survey also showed that 36 percent of data scientists say it takes too long to get insights because the data is too big to move to their analytics software.