Google Updates Cloud Platform, Delivers Dataflow Beta
Kershaw also noted that Google’s goal with Cloud Dataflow was to create an environment where any programmer or any analyst could take the power of big data and be able to transform their business quickly and easily. “The intent is if you can do basic Java programming you can now write big data applications,” he said. “A few years ago that was not possible. It was impossible for the average programmer to be able to deal with the complexity of stringing together map reductions.” Making big data easier along with the natural advantages of the cloud will drive a transformation in how people approach these problems. In that regard, big data and the cloud are natural bedfellows. Doing big data in the cloud helps organizations be more productive when building applications, with faster and better insights without having to worry about the underlying infrastructure. “It’s very difficult to do an on-prem model where you have to buy, set up and run machines to suit the growing needs of your big data environment,” Kershaw said. “So the on-demand compute model and big data just go together hand- in- hand. There’s the operations piece, there’s ability to scale and run different workloads and there’s the issue of security and collaboration and how you can take information and share it across the organization. We think the cloud collaboration model and the new tools we’re delivering to make big data easy are going to change the game in how organizations use big data. So it’s no longer just the realm of the data scientist; it’s really going to be accessible to any developer anywhere at any time.”BigQuery is a large-scale analytics engine that allows you to run through massive volumes of data and do it with a SQL front end. It is Google’s flagship product for being able to integrate large scale analytics with off –the- shelf business tools. Google also announced the availability of BigQuery in Europe so users can store their data in Google Cloud Platform European data centers and support for data residency so users can specify which continent they want their data to be stored in and Google will make sure it stays there. Google also enhanced BigQuery’s ingestion capability, so it can now ingest 100,000 rows per second per table. And the company introduced row-level permissions, a new security feature that helps with how you store information. “BigQuery is the ideal platform for storing, analyzing, and sharing structured data,” Vambenepe said. “It also supports repeated records and querying inside JSON objects for loosely structured data.” Meanwhile, Google Cloud Pub/Sub is designed to provide scalable, reliable, and fast event delivery as a fully managed service, he said. “Along with BigQuery streaming ingestion and Dataflow stream processing, it completes the platform’s end-to-end support for low-latency data processing,” Vambenepe added. “Whether you’re processing customer actions, application logs, or IoT events, Google Cloud Platform allows you to process them in real time, the cloud way. Leave Google in charge of all the scaling and administration tasks so you can focus on what needs to happen, not how.”
Meanwhile, Google also updated BigQuery.