Unstructured Data Troves: 10 Points for Taking Control of Them

By Chris Preimesberger

The term unstructured data describes information that is not organized into a well-defined schema. Nearly all that lives outside relational databases is unstructured and even includes images, videos and log files produced by computers, machines and sensors. Even this list is unstructured data.

The sheer amount of unstructured data is staggering — it makes up 80 percent or more of all digitally stored data. As the data-driven economy grows, the amount of unstructured data being produced grows as well, due to society’s constant use of social media, emails, documents, photos, videos and many other types of data.

Unstructured data is so voluminous because it is being created everywhere, all the time. Every minute, there are more than 6,000 pictures shared on social media sites and more than 200 million emails sent. People aren’t the only ones creating this data—servers generate log files, and machine-based sensors produce mountains of information.

Your company strategy, vision and goals are locked in unstructured data—in the Websites, presentations or Microsoft Word documents where they are held. Even your medical record is unstructured; the notes and test values are often stored as entirely unstructured data.

Rework for recovering lost data is expensive. Employees spend a significant portion of their time creating presentations, proposals and business plans. Businesses lose nearly 20 percent of their productivity looking for previously created unstructured data and laboriously re-creating it if it can’t be found.

Unstructured data isn’t well-organized or easy to access, but companies that analyze this data and integrate it into their information management landscape can significantly improve employee productivity. Analysis allows organizations to quickly identify experts and authoritative information, which saves time and prevents rework. It can also help businesses capture important decisions and the supporting evidence for those decisions.

Analyzing social content—tweets, Facebook posts and transcripts from support calls, for example—gives a clear view of how customers perceive your products. Proactively understanding the issues and responding dramatically improves your customers’ satisfaction, while feature requests can be captured, grouped and prioritized in ways never before possible.

Before a salesperson contacts an existing customer to up-sell or renew a contract, he or she needs to know about the existing deal with the customer, how happy that customer is with the product or service, and who has been working with them in the past. Some of this data—customer contact details, deal size and structure—are often stored in structured databases, while communications, support cases and prior emails are locked in different systems.

There are many laws that cover the management of unstructured data, such as how long data must be retained, or how data should be managed if your company is involved in a lawsuit. Failing to comply can lead to fines or even lost court cases.

The value of unstructured data is clear, but so are the costs of effectively analyzing and managing it. Complex text, audio and video analysis software often requires customization for your organization, and the individuals using the data management systems may need training to ensure that they are getting the most from the investment. Have specific goals in mind when you start an unstructured data analysis or management project.