Close
  • Latest News
  • Big Data and Analytics
  • Cloud
  • Networking
  • Cybersecurity
  • Applications
  • IT Management
  • Storage
  • Sponsored
  • Mobile
  • Small Business
  • Development
  • Database
  • Servers
  • Android
  • Apple
  • Innovation
  • Blogs
  • PC Hardware
  • Reviews
  • Search Engines
  • Virtualization
Read Down
Sign in
Close
Welcome!Log into your account
Forgot your password?
Read Down
Password recovery
Recover your password
Close
Search
Logo
Logo
  • Latest News
  • Big Data and Analytics
  • Cloud
  • Networking
  • Cybersecurity
  • Applications
  • IT Management
  • Storage
  • Sponsored
  • Mobile
  • Small Business
  • Development
  • Database
  • Servers
  • Android
  • Apple
  • Innovation
  • Blogs
  • PC Hardware
  • Reviews
  • Search Engines
  • Virtualization
More
    Home Big Data and Analytics
    • Big Data and Analytics

    Microsoft Embraces Spark for Cloud-Based Big Data Analytics

    By
    Pedro Hernandez
    -
    June 7, 2016
    Share
    Facebook
    Twitter
    Linkedin
      Spark for Azure HDInsight

      Using this week’s Spark Summit in San Francisco as a backdrop, Microsoft announced the general availability of Spark for Azure HDInsight, the company’s cloud-based big data analytics platform. As its name suggests, Apache Spark is taking the big data world by storm with an open-source data processing engine that prioritizes speed and ease of use.

      This week, Microsoft said it is more fully embracing Spark by bringing Spark for Azure HDInsight out of beta. Tiffany Wissner, senior director of Microsoft Data Platform Marketing, described the product as “a fully managed Spark service from Hortonworks that has been hardened for the enterprise and made simpler for you to use,” in a June 6 announcement. “You can also rely on the industry’s highest availability service level agreement for Spark at 99.9 percent.”

      In a bid to speed up deployments, Microsoft has embarked on several product integrations, including working with Hortonworks—the lead commercial sponsor of Hadoop, on which HDInsight is based—to enhance the YARN resource manager. The company also teamed with Cloudera to co-lead Livy, a REST-based Spark service (currently in alpha) that is intended to provide an easy way for applications to interface with Spark setups.

      “Spark gives you fast big data processing with a general purpose flexible API,” said Anand Iyer, senior product manager at Cloudera, in a statement. “We see a natural tendency among our customers and partners to want to leverage Spark’s capabilities from client applications that can easily interface with Spark, and Livy makes that possible.”

      Microsoft announced that the company plans to make R Server for HDInsight, complete with Spark integration, available on the cloud this summer. R is a statistical programming language that is generally favored by data scientists.

      “This makes it easy to move code and projects to the cloud with a few clicks and within a few minutes without buying hardware or hiring specialized operations teams typically associated with big data infrastructure,” explained Wissner.

      In June, the on-premises version of R Server for Hadoop will support native Spark execution frameworks, she added. Microsoft estimates that by combining R Server with Spark, customers will be able to train their statistical models on larger data sets and 100 times faster than open-source R and nearly two times faster than Spark’s own machine learning library (MLLib) by running R functions over thousands of nodes in a Spark cluster.

      Also new is Microsoft R Client, a free R client for data scientists. In addition to enabling analytics on locally stored data, the software can be used on production instances of HD Insight with Spark, along with SQL Server R Services and R Server for Hadoop, said Wissner.

      Finally, on the data visualization front, Microsoft announced that Power BI now supports Spark Streaming. By incorporating Spark’s stream processing technology, customers can now publish real-time events to Power BI’s dashboards for an up-to-the-second updates on key business metrics.

      Pedro Hernandez
      Pedro Hernandez is a contributor to eWEEK and the IT Business Edge Network, the network for technology professionals. Previously, he served as a managing editor for the Internet.com network of IT-related websites and as the Green IT curator for GigaOM Pro.

      MOST POPULAR ARTICLES

      Big Data and Analytics

      Alteryx’s Suresh Vittal on the Democratization of...

      James Maguire - May 31, 2022 0
      I spoke with Suresh Vittal, Chief Product Officer at Alteryx, about the industry mega-shift toward making data analytics tools accessible to a company’s complete...
      Read more
      Cybersecurity

      Visa’s Michael Jabbara on Cybersecurity and Digital...

      James Maguire - May 17, 2022 0
      I spoke with Michael Jabbara, VP and Global Head of Fraud Services at Visa, about the cybersecurity technology used to ensure the safe transfer...
      Read more
      Applications

      Cisco’s Thimaya Subaiya on Customer Experience in...

      James Maguire - May 10, 2022 0
      I spoke with Thimaya Subaiya, SVP and GM of Global Customer Experience at Cisco, about the factors that create good customer experience – and...
      Read more
      Cloud

      IGEL CEO Jed Ayres on Edge and...

      James Maguire - June 14, 2022 0
      I spoke with Jed Ayres, CEO of IGEL, about the endpoint sector, and an open source OS for the cloud; we also spoke about...
      Read more
      Cloud

      Yotascale CEO Asim Razzaq on Controlling Multicloud...

      James Maguire - May 5, 2022 0
      Asim Razzaq, CEO of Yotascale, provides guidance on understanding—and containing—the complex cost structure of multicloud computing. Among the topics we covered:  As you survey the...
      Read more
      Logo

      eWeek has the latest technology news and analysis, buying guides, and product reviews for IT professionals and technology buyers. The site’s focus is on innovative solutions and covering in-depth technical content. eWeek stays on the cutting edge of technology news and IT trends through interviews and expert analysis. Gain insight from top innovators and thought leaders in the fields of IT, business, enterprise software, startups, and more.

      Facebook
      Linkedin
      RSS
      Twitter
      Youtube

      Advertisers

      Advertise with TechnologyAdvice on eWeek and our other IT-focused platforms.

      Advertise with Us

      Menu

      • About eWeek
      • Subscribe to our Newsletter
      • Latest News

      Our Brands

      • Privacy Policy
      • Terms
      • About
      • Contact
      • Advertise
      • Sitemap
      • California – Do Not Sell My Information

      Property of TechnologyAdvice.
      © 2022 TechnologyAdvice. All Rights Reserved

      Advertiser Disclosure: Some of the products that appear on this site are from companies from which TechnologyAdvice receives compensation. This compensation may impact how and where products appear on this site including, for example, the order in which they appear. TechnologyAdvice does not include all companies or all types of products available in the marketplace.

      ×