Microsoft Azure IoT Connects to Apache Kafka to Manage Device Links

A new open-source connector enables businesses to build IoT applications that gather and make sense of mountain of data from disparate sources.

Kafka Azure IoT

Building an Azure-backed internet of things (IoT) application that uses the popular open-source Apache Kafka stream ingestion technology just got easier.

Like RabbitMQ, Kafka is considered an IoT application building block for enterprises. It acts as a queuing system for large IoT deployments, enabling storage and analytics systems to catch up to the high volumes of data produced by disparate sensors and other connected devices.

Microsoft today released Kafka Connect for Azure IoT Hub, an open-source connector that enables customers to feed telemetry data into the company's could-based IoT device connection and management service.

"Azure IoT Hub provides secure two-way communication with devices, device identity and device management at extreme scale and performance," said Sam George, partner director at Microsoft Azure Internet of Things, in a Dec. 5 announcement. "Kafka Connect for Azure IoT Hub enables developers to connect IoT Hub to open source systems using Kafka for a powerful, secure and performant IoT solution. Kafka Connect for IoT Hub can also be used with the new managed Kafka solution available in Azure HDInsight."

The connector arrives two weeks after the company released new Azure IoT Hub device management features, including a new Device Twin capability that digitally portrays a customer's physical devices and allows users to sync settings between devices and the cloud service. New device management libraries enable customers to apply firmware updates or reboot to their IoT devices. Users can also now roll back devices to their original factory settings or configure devices using the parameters of their virtual twin, whether they are currently online or offline.

While Microsoft is focused on improving interoperability and enhancing Azure IoT's management capabilities as the platform's reach expands, the company is also keeping an eye on security.

In October, Microsoft launched its new Security Program for Azure IoT that matches customers with IoT security specialists. Participating partners can provide customers with end-to-end security audits, ensuring that every link in the IoT chain, including devices, gateways and cloud connectivity components, works together to prevent hacks and keep data secure. Casaba Security, CyberX, Praetorian and Tech Mahindra are among the first auditors to offer up their services under the program.

IoT security has become a priority for vendors seeking to capitalize on one of the biggest bright spots in the IT solutions market.

While companies rush to IoT-enable their offerings, cyber-attackers are following close behind, causing some to call into question the wisdom of forging ahead with large-scale IoT deployments. In October, Domain Name System (DNS) services provider DynDNS was hit with a massive distributed denial-of-service attack (DDoS) that snarled internet traffic and slowed access to many popular online services like Spotify and Twitter. The culprit was a botnet called Mirai that largely consists of compromised IoT devices.

"If broken devices or devices with poorly configured defaults in any autonomous system can be aggregated into a force that can attack any individual, company or network, then it would seem logical to conclude a susceptible device anywhere poses a risk to everyone," wrote Chris Baker, Monitoring and Analytics manager at Dyn, in his Oct. 20 analysis of the incident.

Pedro Hernandez

Pedro Hernandez

Pedro Hernandez is a contributor to eWEEK and the IT Business Edge Network, the network for technology professionals. Previously, he served as a managing editor for the Internet.com network of...