eWEEK content and product recommendations are editorially independent. We may make money when you click on links to our partners. Learn More.
As time and IT innovation move on, we’re seeing more and more clouds for specific purposes.
These are clouds for verticals that include retail, health care, financial services and science projects. But they also are aimed at specific jobs, such as video processing, genome tracking, oil and gas exploration and others.
As of Nov. 28, there’s a new cloud in town, one that is purpose-built for streaming data. Confluent, which developed its own streaming-data platform based on Apache Kafka, announced at Amazon re:Invent 2017 in Las Vegas that its Confluent Cloud is now ready for prime time and is available for enterprise production workloads.
Built by the original creators of Apache Kafka, Confluent Cloud puts into action the company’s expertise gained from managing large streaming environments. This is a fully-managed service that enables enterprises to deploy their customer experience projects, fleet management upgrades, fraud detection and other real-time, large-scale initiatives in one place.
Open-Source Streaming Data Platform is a Workhorse
Apache Kafka is an open-source stream processing platform written in Scala and Java by Jay Kreps, Neha Narkhede and Jun Rao while they were working at LinkedIn. It was then open-sourced through the Apache Software Foundation. Kafka provides a unified, high-throughput, low-latency platform for handling real-time data feeds.
Unlike conventional enterprise messaging software, Kafka is able to handle all the data flowing through a company and do it in near real time. Kafka is a high-performance publish/subscribe message bus designed for high availability and data durability, with minimal latency. Kafka acts as a central data backbone and enables loose coupling between applications (Akka), data processing (Spark) and data persistence (Cassandra) services.
As the overall volume of data from machines and human-operated devices–mostly from machines–continues to zoom up, IT systems sometimes find themselves straining to keep that data moving from place to place at internet speed.
All those moving bits are becoming heavy loads for systems old and new. Server and storage software needs to refreshed all the time in order to keep up, or else an enterprise can fall behind its competitors in delivery of services to customers.
Nutanix, OpenStack, Mesosphere All Use Kafka
At the outset of the IoT and IIoT (industrial internet of things) age, data center operating systems like Nutanix, OpenStack and Mesosphere are three of the engines that are able to scale up in real time as needed by workload demands.
One thing in common they all have is–you guessed it: Apache Kafka.
Confluent’s platform includes Confluent Cloud, Confluent Enterprise and Confluent Open Source, to create a more complete distribution of Kafka intended for large-scale production across a variety of environments. It improves Kafka by expanding its integration capabilities, adding tools to optimize and manage Kafka clusters, and ensuring the streams are secure.
With Confluent Cloud, enterprises and developers have the option to build and run applications in the cloud, knowing that the team who created Kafka are maintaining and managing the Kafka clusters.
Key features include:
- Latest Kafka capabilities: Always uses the latest Apache Kafka version and so developers never have to worry about upgrades again.
- Performant: Highest throughput rate of any streaming data service. Standard plan is 5 MB/s and custom plans are available for any scale.
- Reliable: Optional support for multiple availability zones and 99.95 percent service level agreement.
- Client support: Full support for the range of clients offered by Confluent, including Java, Python, C/C++, Go and .NET, as well as the Kafka Streams API.
- Flexible: Configurable retention period, storage and throughput rate to suit any workload.
- Kafka expertise: Backed by the team that created Kafka that has the most extensive experience operating it at scale.
“Confluent Cloud is now production ready and capable of handling enterprise workloads, ideal for companies that are looking for experts to run their Apache Kafka as a Service for them,” Narkhede, co-founder and CTO at Confluent, said. “Enterprises and developers can now focus on building applications that provide value to the business, rather than worrying about managing and running Kafka.”
Availability and Pricing
The service is currently available on AWS US-West-2 and US-East-1 Regions, with support for Microsoft Azure and Google Cloud to be added in the future. Confluent Cloud is elastically priced based on usage. Pricing is determined based on throughput, retention and availability.
For more information, go here.
Palo Alto, Calif.-based Confluent is at AWS re:Invent 2017 through Nov. 30 at booth No. 1300.