10 Big Data Myths Enterprises Must Come to Terms With

by Chris Preimesberger

Having searchable, accessible data is definitely critical, but it is only part of the equation. Determining what to do with the data and knowing the right questions to ask to derive actionable insights are also essential parts of the solutions that drive ROI.

When architecting big data initiatives, it’s easy to assume that connecting the data will be a straightforward, simple process. However, there are organizational and technical challenges. Data managers from different business units often disagree on the best methods for sharing data and what to standardize on (e.g., unique identifiers or primary keys). Typically, each business unit focuses on summary data to answer specific questions—excluding data important to other units—and frequently lose sight of the entire picture.

The value of data is that it is dynamic and evolves with additional context. Data snapshots representing a fixed moment in time are valid today but may not be tomorrow, and they definitely lack a view into changing data contexts.

Simply tracking and measuring everything without tools to filter the noise results in unnecessary technical overhead, processing and access control issues, or worse—misleading conclusions. With the right tools in place, the least-suspected data element can be the one that provides the “Ah-ha!” moment.

Every big data technology solution has its strengths and its weaknesses. It’s important to find out which data types can be loaded and easily queried and which require custom development in your technology platform.

The impact of big data is likely to be incremental and subtle. We don’t need volumes of data to identify the biggest trends—they’re obvious. It’s getting to the nuggets of critical information hidden in the exabytes of stored data that will help decision-makers make the best choices at the best time in context.

Since it’s a relatively new career path, there are only so many data scientists available. There’s been a good deal of concern generated that without data scientists on staff, it’s impossible to succeed. Having the right tools with the business acumen and context is far more critical.

More data may be collected than ever before, but there has been too much data for organizations to consume and act on for decades. What have changed are the tools to manage, secure and access the data.

Big data initiatives are often extremely costly, but they don’t have to be. Everything doesn’t have to be built in-house from the ground up. Often a hybrid solution combining in-house expertise and third-party tools offers a less expensive, more effective long-term solution.

Big data is more likely to be messy, unstructured, “dirty” data that requires a lot of scrubbing—sometimes manual, but more often automated—to perform meaningful analytics.