Confluent, which makes a streaming-data platform based on open-source Apache Kafka, on May 22 announced a partnership with Google to make that cloud-based service available on Google Cloud Platform for enterprise production use cases.
These projects could include event-driven applications, machine learning for advanced analytics and hybrid-cloud data pipelines, the Palo Alto, Calif.-based company said.
By using a fully-managed streaming data service on GCP, users can offload the operating burden of Apache Kafka and stream data at scale in real time to Google Cloud’s big data services, which including BigQuery, Cloud Machine Learning Engine and TensorFlow. This data streaming enables enterprises and developers to focus on generating faster and better business insights instead of managing infrastructure.
Built by the original creators of Apache Kafka, Confluent Cloud enacts the company’s expertise gained from managing large streaming environments. This is a fully-managed service that enables enterprises to deploy their customer experience projects, fleet management upgrades, fraud detection and other real-time, large-scale initiatives in one place.
Open-Source Streaming Data Platform is a Workhorse
Apache Kafka is an open-source stream processing platform written in Scala and Java by Jay Kreps, Neha Narkhede and Jun Rao while they were working at LinkedIn. It was then open-sourced through the Apache Software Foundation. Kafka provides a unified, high-throughput, low-latency platform for handling real-time data feeds.
Unlike conventional enterprise messaging software, Kafka is able to handle all the data flowing through a company and do it in near real time. Kafka is a high-performance publish/subscribe message bus designed for high availability and data durability, with minimal latency. Kafka acts as a central data backbone and enables loose coupling between applications (Akka), data processing (Spark) and data persistence (Cassandra) services.
Using Confluent Cloud on Google Cloud Platform, enterprises can operationalize streaming data by:
- Building event-driven applications: They can productize Confluent Platform for event-driven applications by combining Confluent Cloud streaming services with Google Cloud solutions, including Cloud Functions, App Engine and Kubernetes.
- Unleashing developer velocity: This enables Kafka developers to start building applications on Google Cloud in minutes, without the operational burden of managing Kafka clusters.
- Providing a persistent bridge to cloud: Users can build a real-time data pipeline between data centers and Google Cloud and accelerate multi-cloud adoption by making data reliably available across any cloud.
- Analyzing data at massive scale: Confluent Cloud can be used to stream data at scale and in real time to Google Cloud’s big data solutions, including BigQuery, Cloud Machine Learning Engine and TensorFlow.
With Confluent Cloud, enterprises and developers have the option to build and run applications in the cloud while knowing that the team that created Kafka is maintaining and managing the Kafka clusters.
In addition to the partnership with Google Cloud, Confluent now has two subscription options available:
- Confluent Cloud Enterprise: A subscription to the streaming data service for all applications, from development to production. With full enterprise capabilities such as SLAs, security and VPC peering to connect to infrastructure while running highly-scaled, real-time applications and pipelines.
- Confluent Cloud Professional: A subscription to the streaming data service to meet the needs of projects under development as well as smaller organizations and applications. It offers the low latency of Kafka and automatic upgrades, among other features, to speed up Kafka application development.
Confluent Cloud Professional is in limited availability on Google Cloud and Amazon Web Services. Pricing is based on hours of provisioned capacity and billed on usage at the end of the month. Confluent Cloud Enterprise also is in limited availability on Google Cloud and is generally available on Amazon Web Services. Go here for pricing details.
For more information on Confluent Cloud, go here.