eWEEK content and product recommendations are editorially independent. We may make money when you click on links to our partners. Learn More.
While state and local agencies express interest in leveraging data to improve decision making and meet mission objectives, they are struggling to process and analyze the data, according to the “State and Local Big Data Gap” study by MeriTalk, the government IT network, which was underwritten by NetApp.
State and local IT professionals also report a gap between big data’s promise and big data reality. State and local agencies estimate that they have just 46 percent of the data storage or access, 42 percent of the computational power and 35 percent of the personnel they need to successfully leverage big data. In addition, 57 percent say their current enterprise architecture is not able to support big data initiatives.
Despite the challenges, the survey results indicated that some state and local agencies are working to close the technology gap. Thirty-nine percent of survey respondents said they are investing in IT systems and solutions to improve data processing, 39 percent are improving the security of stored data, and 37 percent are investing in IT infrastructure to improve data storage.
However, more than three-quarters (79 percent) of state and local IT professionals say they are just somewhat or not very familiar with the term “big data,” and only 2 percent say they have a complete big data strategy. Big data isn’t even on the radar screen for 44 percent of state and local agencies—they are not even discussing it.
While they understand the promise of big data, just 59 percent of state and local agencies are analyzing the data they collect and less than half are using it to make strategic decisions. On average, state and local IT professionals report that it will take their agencies at least three years to take full advantage of big data.
“State and local agencies have made great strides in consolidating applications and data into fewer physical resources,” Regina Kunkle, vice president of state and local government for NetApp, said in a statement. “Storage efficiencies like deduplication and compression help to manage the explosive storage growth by reducing the amount of storage required and simplifying data management. However, agencies still have data silos, and they are just beginning to explore how to effectively analyze this disparate data. To help them unlock this valuable wealth of information, agencies should look toward big data solutions.”
The current average state and local agency stores 499 terabytes of data, and those IT professionals indicated they expect that amount of data to continue to grow. The majority (87 percent) of state and local agencies say the size of their stored data has grown in the last two years, and 97 percent expect data to grow by an average of 53 percent in the next two years.
The top challenge when it comes to managing large amounts of data is storage capacity (46 percent) followed by speed of analysis and processing (34 percent), and analysis (32 percent). To further complicate data management, agencies are unclear about who owns the data. Nearly half (47 percent) believe that IT owns the data, and 31 percent believe ownership belongs to the department that generated it.