Need for Speed: How Rapid Data Processing Benefits Enterprises

While virtually all organizations would benefit from processing and acting upon data quickly, few are successful in doing so, according to a recent survey from Streamlio. The resulting report, titled “The Speed of Data,” reveals that only a minority of businesses formally measure how fast they process data. However, by improving this speed, they’d greatly streamline operations, improve real-time customer interactions and more effectively respond to business/market changes. They’d also boost the performance of their business intelligence (BI) and security solutions. More than 300 data decision-makers took part in the research, which was conducted by Dimensional Research. The following slide show presents survey highlights, with charts provided courtesy of Streamlio.

Virtually all respondents to the Streamlio survey said it’s important to their organization to process and act upon data as quickly as possible. Two-thirds said this is “critically important.”

Only 17 percent of respondents said their current speed in processing data was “good.” But 94 percent said their organization is making tech investments to do this faster.

Despite the need to quickly process data, only 24 percent of respondents said their company has specific metrics in place to measure whether they’re doing this. More than three-quarters said they measure this, but not in a formal way—or they don’t measure this at all.

When asked which initiatives are driving the need to process data immediately, 47 percent of respondents cited new analytics, machine learning and artificial intelligence (AI) projects. The integration of new, real-time and streaming data sources ranked second, as cited by 44 percent of respondents.

In identifying the biggest barriers to immediate data processing, 53 percent of respondents cited difficulties in making changes to existing solutions and infrastructure. Nearly one-half said they struggle to find employees with the right data engineering, operational or analytical skills.

About three-quarters of respondents said more than one-half of their company’s data is processed periodically in a “batch run,” instead of immediately as it arrives. Nearly two of five said more than 90 percent of their organization’s data is processed in batch.

Three of five respondents said that their company’s business intelligence and reporting products would greatly benefit by processing data immediately, rather than “in batch.” Other top types of technologies that would benefit by faster data processing are app/system monitoring and alerting solutions (as cited by 54 percent of respondents), security/fraud detection products (52 percent) and customer interaction tools (51 percent).

Two-thirds of respondents said their organization would increase efficiencies as part of streamlined operations by processing data immediately. Three of five said they’d enable real-time customer interactions by doing so, and one-half said they’d respond to business/market changes more effectively.

When asked which additional tech capabilities they’d find valuable, 64 percent of respondents cited the real-time processing of analytics. Nearly three of five cited the ability to query data immediately, as it is received.

With greater tech capabilities such as the real-time processing of analytics, 58 percent of respondents said they’d increase customer satisfaction. Nearly the same percentage said they’d enable new, data-driven applications that were impractical or impossible before, and more than one-half said they’d expand access to real-time data to more users.