Industry Experts Predict Progress in Integrating AI, Big Data in 2018

The most important big data trend in 2018 will involve greater integration with artificial intelligence (AI), data science and machine learning, but metadata management and global data fabrics will also play a key role.

2018 Big Data Predictions

Big changes are coming in the big data and analytics arena, and IT decision-makers should prepare for major advances in 2018. The most significant big data trend is the increased integration of artificial intelligence (AI) and machine learning, but metadata management and global data fabrics will also make an impact next year. Following are predictions from some prominent industry experts.

“Artificial intelligence—especially data science and machine learning (DS&ML)—will change the way we acquire, manage and analyze data,” said Jorgen Heizenberg, Gartner data and analytics research director. “Today, due to its complexity, this is mostly done by humans—often developers hired from an external service provider, he said.

“However, DS&ML are the engines of future data and analytics (D&A) services. At first, it will be more about automating simple and routine tasks like data extraction," Heizenberg said. "In time, more complex and non-routine tasks will follow, leading to ‘intelligent’ automation. This will potentially scale the enterprise insights, as it allows more room for human-based analytics.”  

“We will witness a shift from labor-based D&A services toward machine-based, often as part of a converged analytical solution of services and software or ‘servware.’ Companies should introduce testing and review boards on all models, algorithms and data used in order to build trust,” Heizenberg recommended.

Getting Better Outcomes From Big Data and AI

“There is no AI without IA—information architecture,” stated Rob Thomas, general manager, IBM Analytics. “If companies don’t have the right infrastructure, it’s hard to do AI right. To get meaningful results, data needs to be in an organized state, and the right technology has to be put into action.”

Thomas pointed out that the internet of things (IoT) has added significantly to data challenges by forcing companies to ingest massive amounts of data at incredibly high speeds in short periods of time. This is putting a strain on many organizations.

Most executives Thomas has spoken with understand that data is a source of competitive advantage, but they want to get better outcomes than the ones they’ve gotten to date. One solution he suggested is to create data catalogs in a consumable form, which can help by making it easier to find data and gain insights from it.

Metadata management plays a role here. “Companies need a single source of truth,” Thomas said. “Data catalogs provide that, and metadata management details what’s in the catalog.”

This type of data governance is essential for both regulatory compliance and self-service access to data, according to Thomas. He added that the European Union’s General Data Protection Regulation (GDPR) “will give a big boost to data governance in 2018.”

Another point Thomas stressed is the importance of democratizing data analytics so that even non-technical workers—not just data scientists and business analysts—can use these products.

“Analytics systems should be simple, elegant and have a great design,” he said, adding that this would make these products easier to install and use, thus making them available to a growing number of employees. “At, for example, a non-tech person can download DB2 in less than 10 minutes,” he said.

Boosting Analytics With AI

“The world is moving from big data to all data—structured, unstructured and contextual—from sources such as sensors, social, video and chat,” said Jean-Luc Chatelain, chief technology officer at Accenture Applied Intelligence. “This makes it much harder to separate the signal from the noise, but you can’t get good insights from bad data. That’s why, in 2018, artificial intelligence will have a much bigger role to play in data preparation.