eWEEK content and product recommendations are editorially independent. We may make money when you click on links to our partners. Learn More.
With the explosion of big data and companies mining it for analytics and insights, developers are working on more big data projects than ever.
In fact, more than a third—36 percent—of all developers who are working on big data or advanced analytics projects are using elements of machine learning, according to a recent Evans Data study on big data and advanced analytics.
Huge amounts of data are being generated by sources such as social media, video, audio, wearables and the Internet of things (IoT), leading companies to look for new ways to handle and process all of that data to make it more meaningful and to derive insights from it. Enter machine learning.
Machine learning is one of a number of technologies being adapted to help make sense of all this new data being generated. Machine learning is a field of computer science involving creating and continuously improving algorithms that automatically analyze data to identify patterns or predict outcomes, said Mike Gualtieri, principal analyst at Forrester Research.
Moreover, machine learning is the study of computer algorithms that provide computer programs with the ability to learn, discover, predict and improve automatically using large amounts of data without explicit programming, said Dave Schubmehl, research director for Cognitive Systems and Content Analytics at IDC.
Evans Data said that the market for machine learning is fragmented, yet developers are using machine learning to empower applications in the financial sector, the IoT and manufacturing.
“Machine learning includes many techniques that are rapidly being adopted at this time and the developers who already work with Big Data and advanced analytics are in an excellent position to lead the way,” said Janel Garvin, CEO of Evans Data, in a statement. “We are seeing more and more interest from developers in all forms of cognitive computing, including pattern recognition, natural language recognition, and neural networks and we fully expect that the programs of tomorrow are going to be based on these nascent technologies of today.”
IBM CEO Ginni Rometty has dubbed this the “cognitive era,” and IBM is leading the way with its Watson cognitive computing system. Several other companies have prominent cognitive technology offerings or projects in the works, including Amazon, Apple, Facebook, Google, Hewlett Packard Enterprise, Microsoft and SAS, among others.
In May, Evans Data surveyed more than 500 developers working with big data. The results indicated that decision trees are the most used analytical model that links in closely with artificial intelligence and machine learning development. The next most cited analytical models were linear regression and logistics regression, the survey said. And company departments most likely to be using big data or advanced analytics are logistics, distribution and operations, according to the survey.
Other survey findings include that developers working with big data spend at least some of their time instrumenting processes. Indeed, 42 percent are analyzing data in real time, and 38 percent said they analyze unstructured data. Meanwhile, developers working with big data noted that the primary improvement they would like to see is improved security of off-site data stores.
The 200-page Evans Data “Big Data and Advanced Analytics 2016” report covers areas such as the big data landscape, barriers and challenges to big data, advanced analytics tools and services, data warehousing, big data and the Internet of things, Hadoop, artificial intelligence and machine learning, real-time events and time series processing, parallelism and big data, and cloud computing.