Close
  • Latest News
  • Artificial Intelligence
  • Big Data and Analytics
  • Cloud
  • Networking
  • Cybersecurity
  • Applications
  • IT Management
  • Storage
  • Sponsored
  • Mobile
  • Small Business
  • Development
  • Database
  • Servers
  • Android
  • Apple
  • Innovation
  • Blogs
  • PC Hardware
  • Reviews
  • Search Engines
  • Virtualization
Read Down
Sign in
Close
Welcome!Log into your account
Forgot your password?
Read Down
Password recovery
Recover your password
Close
Search
Logo
Logo
  • Latest News
  • Artificial Intelligence
  • Big Data and Analytics
  • Cloud
  • Networking
  • Cybersecurity
  • Applications
  • IT Management
  • Storage
  • Sponsored
  • Mobile
  • Small Business
  • Development
  • Database
  • Servers
  • Android
  • Apple
  • Innovation
  • Blogs
  • PC Hardware
  • Reviews
  • Search Engines
  • Virtualization
More
    Home Big Data and Analytics
    • Big Data and Analytics

    Apache Kafka Lands on Microsoft’s Cloud-Based Big Data Platform

    By
    Pedro Hernandez
    -
    December 19, 2017
    Share
    Facebook
    Twitter
    Linkedin
      Kafka

      eWEEK content and product recommendations are editorially independent. We may make money when you click on links to our partners. Learn More.

      After a yearlong beta period, Apache Kafka on Azure HDInsight is now ready to take on production workloads with Microsoft’s rollout of the solution.

      Kafka is an open-source stream processing platform that is becoming increasingly pivotal to monetizing the wealth of information generated by the internet of things (IoT) for many enterprises. Azure HDInsight is Microsoft’s cloud-based big data platform.

      By bringing them together and offering Kafka as a managed service on Azure, Microsoft claims it can help remove some of the barriers that are preventing organizations from seizing business opportunities in a hyperconnected world.

      “HDInsight is a managed platform with a 99.9 percent SLA [service-level agreement] on open source workloads,” wrote Raghav Mohan, a program manager at Microsoft Azure Big Data, in a Dec. 18 announcement. “With this addition, our enterprise customers no longer worry about managing Kafka clusters, as HDInsight manages and fixes the issues involved with running Kafka at an enterprise scale.”

      Naturally, Microsoft was among the first to use its own offering in a production setting.

      At Microsoft, Apache Kafka on Azure HDInsight powers Siphon, a distributed system that the company uses to ingest, distribute and consume streaming data for subsequent processing and analytics. According to Microsoft, Siphon ingests more than a trillion messages each day and helps deliver some of the company’s largest cloud services, including Office 365, Skype and Bing Ads.

      Outside of Redmond, Wash., GE Healthcare is using Apache Kafka on Azure HDInsight to power a new generation of intelligent health care services that use machine learning and big data to solve problems faced by medical facilities and their patients. Adobe, one of Microsoft’s most high-profile cloud customers, is using the service to process a billion Adobe Experience Cloud transactions a day.

      Microsoft also announced the general availability of an HDInsight integration with Azure Log Analytics, allowing customers to monitor their various HDInsight clusters from a single interface. Users can view metrics produced by various open-source frameworks and view the performance characteristics of clusters on the virtual machine level, including memory and CPU utilization. More information on enabling Log Analytics is available here.

      Finally, Microsoft is reducing the cost of running big data workloads on Azure HDInsight.

      Beginning Jan. 5, 2018, customers may see up to a 52 percent reduction in the price of their HDInsight subscriptions. “Customers will get even more value from their batch processing, interactive querying, machine learning, streaming analytics, and real-time analytics workloads on Azure HDInsight at a much lower price,” said Rimma Nehme, group program manager of Azure Cosmos DB and Open Source Software Analytics at Microsoft, in a separate announcement.

      The savings are more drastic for users running the R Server analytics platform on Azure HDInsight, added Nehme. Microsoft slashed prices by 80 percent on R Server workloads to $0.016 per core hour, she noted.

      Pedro Hernandez
      Pedro Hernandez is a contributor to eWEEK and the IT Business Edge Network, the network for technology professionals. Previously, he served as a managing editor for the Internet.com network of IT-related websites and as the Green IT curator for GigaOM Pro.
      Get the Free Newsletter!
      Subscribe to Daily Tech Insider for top news, trends & analysis
      This email address is invalid.
      Get the Free Newsletter!
      Subscribe to Daily Tech Insider for top news, trends & analysis
      This email address is invalid.

      MOST POPULAR ARTICLES

      Latest News

      Zeus Kerravala on Networking: Multicloud, 5G, and...

      James Maguire - December 16, 2022 0
      I spoke with Zeus Kerravala, industry analyst at ZK Research, about the rapid changes in enterprise networking, as tech advances and digital transformation prompt...
      Read more
      Applications

      Datadog President Amit Agarwal on Trends in...

      James Maguire - November 11, 2022 0
      I spoke with Amit Agarwal, President of Datadog, about infrastructure observability, from current trends to key challenges to the future of this rapidly growing...
      Read more
      Cloud

      IGEL CEO Jed Ayres on Edge and...

      James Maguire - June 14, 2022 0
      I spoke with Jed Ayres, CEO of IGEL, about the endpoint sector, and an open source OS for the cloud; we also spoke about...
      Read more
      Applications

      Kyndryl’s Nicolas Sekkaki on Handling AI and...

      James Maguire - November 9, 2022 0
      I spoke with Nicolas Sekkaki, Group Practice Leader for Applications, Data and AI at Kyndryl, about how companies can boost both their AI and...
      Read more
      IT Management

      Intuit’s Nhung Ho on AI for the...

      James Maguire - May 13, 2022 0
      I spoke with Nhung Ho, Vice President of AI at Intuit, about adoption of AI in the small and medium-sized business market, and how...
      Read more
      Logo

      eWeek has the latest technology news and analysis, buying guides, and product reviews for IT professionals and technology buyers. The site’s focus is on innovative solutions and covering in-depth technical content. eWeek stays on the cutting edge of technology news and IT trends through interviews and expert analysis. Gain insight from top innovators and thought leaders in the fields of IT, business, enterprise software, startups, and more.

      Facebook
      Linkedin
      RSS
      Twitter
      Youtube

      Advertisers

      Advertise with TechnologyAdvice on eWeek and our other IT-focused platforms.

      Advertise with Us

      Menu

      • About eWeek
      • Subscribe to our Newsletter
      • Latest News

      Our Brands

      • Privacy Policy
      • Terms
      • About
      • Contact
      • Advertise
      • Sitemap
      • California – Do Not Sell My Information

      Property of TechnologyAdvice.
      © 2022 TechnologyAdvice. All Rights Reserved

      Advertiser Disclosure: Some of the products that appear on this site are from companies from which TechnologyAdvice receives compensation. This compensation may impact how and where products appear on this site including, for example, the order in which they appear. TechnologyAdvice does not include all companies or all types of products available in the marketplace.

      ×