If you believe what some of the big-data experts are predicting, 2019 could well become a year that will bring renewed interest in data lakes, of all things. Private clouds, too, may be making a comeback.
Other big-data-related predictions include hybrid becoming the de facto general environment, big data being consolidated into “little data” and transparency becoming a key AI requirement.
Another expert in this segment believes “there will be a resurgence of developers looking at what their apps do with data, as they will have to make those deployments get data out to the rest of the business for them to use. For enterprises with a big mix of hybrid cloud services, traditional and modern apps, and big business problems to solve, that data issue is going to be top of mind.”
Here is the latest in a series of eWEEK prediction articles ahead of the new year 2019.
Alex Gorelik, founder/CTO of Waterline Data:
Now proven, AI and ML will dig deeper into enterprise: “In 2018, we watched as the time, cost, and labor-intensive manual processes that have been holding up the big data initiatives within organizations began to melt away. Automation, AI and ML—proven now not just in terms of speed but also accuracy—is now being applied to more and more business functions. This fits into a general trend of moving away from hard-coding business process and operations into software–and adjusting people and physical operations to match the predefined and rigid business processes–and toward dynamically adapting business processes and operations to the physical realities and historical learnings.
“For example, universities are measuring historical admission and acceptance trends to determine who is likely to accept admission and how much would scholarships affect their decision. Alternative credit risk analysis is being performed to determine creditworthiness of first-time or low-income borrowers. Customer churn predictions are being gleaned from sentiment analysis of social media. Key to all these applications is the ability to create good stable models and the key to building good stable models is being able to find the right data and create the right features. In 2019, AI and ML will play a big role in finding and understanding the data needed to build those models.”
Say hello to hybrid environments: “Last year, I predicted broad adoption of the cloud will finally force object stores to be hardened and properly governed, and that the new standards would require data governance that’s cloud, location and platform agnostic. In 2019, you will see more organizations that are now comfortable with the cloud rowing a hybrid, heterogeneous data estate that includes multiple fit-for-purpose big data, relational and NoSQL data stores both on-premise and in the cloud. With a hybrid model in place, applications that work best on the public cloud can reside there. Those that need to remain on-premises can do so. While this seems like it would create greater complexity, in 2019, you will see more and more solutions that abstract this complexity through location and compute transparency. From file systems like MapR’s data fabric that create a single name space to AIOps, which addresses complexity in virtual data centers, end users will be increasingly shielded from the complexity of hybrid architectures while getting full benefits of fit-for-purpose, elastic solutions that it offers.”
It’s the data lake’s great return: “While organizations have been traditionally focused on the mechanics of creating and hydrating the data lakes, but frequently creating data swamps instead, 2019 will see a renewed focus on data lake adoption. This is very similar to what we experienced with data warehousing where the initial generation of data warehouses were often misguided and lacked adoption, but they taught the organizations what was really required to create value and achieve broad adoption. I believe we are at the same stage with data lakes and in 2019 the focus will turn from the mechanics of the data lake to making the data in the lakes findable, usable and governed at scale and in automated manner, powered by the new spate of AI-driven data catalogs and governance solutions. Even new data lakes will get rolled out in a much more deliberate manner with clear initial use cases and usage and governance policies. We will also see more data lakes being built or migrated to the cloud to take advantage of managed infrastructure, elastic storage and compute and rich ecosystems as more organizations begin adopting Virtual Data Lakes that span multiple systems.”
Big data becomes little data: “No, organizations won’t be dumping all the stockpiles of their data, but well, they will in limited scope. With greater visibility into the data they have will come opportunities to rationalize and consolidate for significant savings in storage costs and even more accurate analytics now that organizations know which data is corrupted and can be jettisoned. But “becoming little” also speaks to large volumes of data that used to choke the organization now becoming manageable enough to put to use, thanks to the automation of key processes like cataloging.”
Explainability will emerge as key AI requirement: “As more and more business (and government) is run using AI and ML algorithms, there will be more focus on transparency and explainability. Why was a mortgage denied? Can a bank prove that none of the illegal demographics (like race, gender and so forth) were used to make the decision or train the model that made the decision. Finding the appropriate data sets and documenting their lineage and quality is the first step to such transparency and explainability. If we do not know where data came from or what it means, we will not be able to explain the model or insure it’s proper and legal operations.”
Jerry Melnick, President and CEO of SIOS Technology:
Advances in technology will make the cloud substantially more suitable for critical applications: “With IT staff now becoming more comfortable with the cloud for critical applications, their concerns about security and reliability, especially for five-9’s of uptime, have diminished substantially. Initially, organizations will prefer to use whatever high availability failover clustering technology they currently use in their data centers to protect critical applications being migrated to the cloud. This clustering technology will also be adapted and optimized for enhanced operations in the cloud. At the same time, cloud service providers will continue to advance their ability to provide higher service levels, leading to the cloud ultimately becoming the preferred platform for all enterprise applications.”
Dynamic utilization will make HA and DR more cost-effective for more applications, further driving migration to the cloud: “With its virtually unlimited resources spread around the globe, the cloud is the ideal platform for delivering high uptime. But provisioning standby resources that sit idle most of the time has been cost-prohibitive for many applications. The increasing sophistication of fluid cloud resources deployed across multiple zones and regions, all connected via high-quality internetworking, now enables standby resources to be allocated dynamically only when needed, which will dramatically lower the cost of provisioning high availability and disaster recovery protections.”
The cloud will become a preferred platform for SAP deployments. “As the platforms offered by cloud service providers continue to mature, their ability to host SAP applications will become commercially viable and, therefore, strategically important. For CSPs, SAP hosting will be a way to secure long-term engagements with enterprise customers. For the enterprise, “SAP-as-a-Service” will be a way to take full advantage of the enormous economies of scale in the cloud without sacrificing performance or availability.”
Cloud “quick-start” templates will become the standard for complex software and service deployments. “Quick-start templates are wizard-based interfaces that employ automated scripts to dynamically provision, configure and orchestrate the resources and services needed to run specific applications. Among their key benefits are reduced training requirements, improved speed and accuracy, and the ability to minimize or even eliminate human error as a major source of problems. By making deployments more turnkey, quick-start templates will substantially decrease the time and effort it takes for DevOps staff to setup, test and roll out dependable configurations.”
Advanced analytics and artificial intelligence will be everywhere and in everything, including infrastructure operations: “Advanced analytics and artificial intelligence will simplify IT operations, improve infrastructure and application robustness, and lower overall costs. Along with this trend, AI and analytics will become embedded in high availability and disaster recovery solutions, as well as cloud service provider offerings to improve service levels. With the ability to quickly, automatically and accurately understand issues and diagnose problems across complex configurations, the reliability, and thus the availability, of critical services delivered from the cloud will vastly improve.”
Justin Yunag, Chief Digital Officer, Financial Services, Atos North America:
Public cloud users will shift back to private due to cost and control: “The appeals of migrating to the public cloud used to be that companies could pay less and control data more. However, both have since turned into disincentives for technology decision-makers. 2019 will be the year of taking back what is owned by the company. In the public cloud, there are many additional costs. For example, ingress of data may be free, but cloud providers charge for every megabyte that leaves the network. This means the slow drip of egress adds up quickly. In terms of private clouds, cloud providers offer a flat rate every month which benefits cost budgets. In addition, long-term retention of data has exposed public cloud to frequent SEC auditing, resulting in businesses losing control of data. Overall, design, functionality, and cost can be at a higher degree of difficulty in the public cloud, higher than originally thought in prior years.”
Patrick McFadin, VP of Developer Relations, DataStax:
The needs of tomorrow’s enterprise in a hybrid/multi-cloud world: “More companies want to roll out hybrid and multi-cloud applications, but they do also have to think about hybrid cloud data, too. This is a harder task to get right. I think 2019 will be the year when people move from looking solely at application components and instead look at the wider app infrastructure problems. The data side is harder to solve, particularly if you want to avoid getting locked into public cloud provider options.
“There will be a resurgence of developers looking at what their apps do with data, as they will have to make those deployments get data out to the rest of the business for them to use. For enterprises with a big mix of hybrid cloud services, traditional and modern apps, and big business problems to solve, that data issue is going to be top of mind.”