Close
  • Latest News
  • Big Data and Analytics
  • Cloud
  • Networking
  • Cybersecurity
  • Applications
  • IT Management
  • Storage
  • Sponsored
  • Mobile
  • Small Business
  • Development
  • Database
  • Servers
  • Android
  • Apple
  • Innovation
  • Blogs
  • PC Hardware
  • Reviews
  • Search Engines
  • Virtualization
Read Down
Sign in
Close
Welcome!Log into your account
Forgot your password?
Read Down
Password recovery
Recover your password
Close
Search
Logo
Logo
  • Latest News
  • Big Data and Analytics
  • Cloud
  • Networking
  • Cybersecurity
  • Applications
  • IT Management
  • Storage
  • Sponsored
  • Mobile
  • Small Business
  • Development
  • Database
  • Servers
  • Android
  • Apple
  • Innovation
  • Blogs
  • PC Hardware
  • Reviews
  • Search Engines
  • Virtualization
More
    Home Applications
    • Applications
    • Networking
    • Servers
    • Storage

    Amazon Web Services Extends Hadoop Support

    By
    Darryl K. Taft
    -
    October 2, 2009
    Share
    Facebook
    Twitter
    Linkedin

      NEW YORK-Amazon Web Services has moved to make the Hadoop framework for handling large data sets easier to use, more manageable and more productive with new support in the Amazon Elastic MapReduce technology.

      At the Hadoop World: NYC conference here on Oct. 2, Peter Sirota, Amazon.com‘s general manager for Elastic MapReduce, announced that Elastic MapReduce now supports Apache Hive.

      According to an AWS blog post on the news, “Hive builds on Hadoop to provide tools for data summarization, ad hoc querying and analysis of large data sets stored in Amazon S3 [Simple Storage Service]. Hive uses a SQL-based language called Hive QL with support for map/reduce functions and complex extensible user defined data types such as JSON and Thrift. You can use Hive to process structured or unstructured data sources such as log files or text files. Hive is great for data warehousing applications such as data mining and click-stream analysis.”

      Sirota said AWS’ motivation for adding new support for Hadoop was, “We want to enable customers to cost effectively manage vast amounts of data.” Elastic MapReduce uses Hadoop.

      The Apache Hadoop project develops open-source software for reliable, scalable, distributed computing. Hadoop enables applications to work with thousands of nodes and petabytes of data. It was inspired by Google’s MapReduce and GFS (Google File System) papers.

      “Large-scale data processing has a lot of muck and we want to remove it for our customers,” Sirota said. He noted that it is hard to manage compute clusters and also hard to tune Hadoop. “So we decided to make Hadoop simple and easy,” he said.

      One new feature in Elastic MapReduce is support for Apache Pig, an open-source platform for analyzing large data sets. Other features include support for batch and interactive modes, and concurrent access to multiple file systems.

      Sirota said he would like to see the Hadoop ecosystem continue to grow. With that in mind, he announced, “Amazon Elastic MapReduce is now supported by Karmasphere Studio for Hadoop, a NetBeans-based integrated development environment (IDE) that makes it easy to develop, debug and deploy job flows from your desktop directly to Amazon Elastic MapReduce.”

      Sirota “also announced … a private beta release of Amazon Elastic MapReduce support for Cloudera’s Hadoop distribution. Cloudera customers can obtain a support contract to gain access to custom Hadoop patches and help with the development and optimization of processing pipelines,” according to the AWS blog post. It continued:

      “These new tools give you the power to process gigantic data sets while keeping you at arm’s length from some of the more complex aspects of parallel programming. You don’t have to find a server cluster, install a bunch of software, coordinate and synchronize processes, copy data between servers, or negotiate with your colleagues for access to shared resources. These tools reduce the distance between the problem and the solution and allow you to spend your time on the more interesting aspects of your work.”

      Darryl K. Taft
      Darryl K. Taft covers the development tools and developer-related issues beat from his office in Baltimore. He has more than 10 years of experience in the business and is always looking for the next scoop. Taft is a member of the Association for Computing Machinery (ACM) and was named 'one of the most active middleware reporters in the world' by The Middleware Co. He also has his own card in the 'Who's Who in Enterprise Java' deck.

      MOST POPULAR ARTICLES

      Cybersecurity

      Visa’s Michael Jabbara on Cybersecurity and Digital...

      James Maguire - May 17, 2022 0
      I spoke with Michael Jabbara, VP and Global Head of Fraud Services at Visa, about the cybersecurity technology used to ensure the safe transfer...
      Read more
      Big Data and Analytics

      Alteryx’s Suresh Vittal on the Democratization of...

      James Maguire - May 31, 2022 0
      I spoke with Suresh Vittal, Chief Product Officer at Alteryx, about the industry mega-shift toward making data analytics tools accessible to a company’s complete...
      Read more
      Big Data and Analytics

      GoodData CEO Roman Stanek on Business Intelligence...

      James Maguire - May 4, 2022 0
      I spoke with Roman Stanek, CEO of GoodData, about business intelligence, data as a service, and the frustration that many executives have with data...
      Read more
      Applications

      Cisco’s Thimaya Subaiya on Customer Experience in...

      James Maguire - May 10, 2022 0
      I spoke with Thimaya Subaiya, SVP and GM of Global Customer Experience at Cisco, about the factors that create good customer experience – and...
      Read more
      Cloud

      Yotascale CEO Asim Razzaq on Controlling Multicloud...

      James Maguire - May 5, 2022 0
      Asim Razzaq, CEO of Yotascale, provides guidance on understanding—and containing—the complex cost structure of multicloud computing. Among the topics we covered:  As you survey the...
      Read more
      Logo

      eWeek has the latest technology news and analysis, buying guides, and product reviews for IT professionals and technology buyers. The site’s focus is on innovative solutions and covering in-depth technical content. eWeek stays on the cutting edge of technology news and IT trends through interviews and expert analysis. Gain insight from top innovators and thought leaders in the fields of IT, business, enterprise software, startups, and more.

      Facebook
      Linkedin
      RSS
      Twitter
      Youtube

      Advertisers

      Advertise with TechnologyAdvice on eWeek and our other IT-focused platforms.

      Advertise with Us

      Menu

      • About eWeek
      • Subscribe to our Newsletter
      • Latest News

      Our Brands

      • Privacy Policy
      • Terms
      • About
      • Contact
      • Advertise
      • Sitemap
      • California – Do Not Sell My Information

      Property of TechnologyAdvice.
      © 2021 TechnologyAdvice. All Rights Reserved

      Advertiser Disclosure: Some of the products that appear on this site are from companies from which TechnologyAdvice receives compensation. This compensation may impact how and where products appear on this site including, for example, the order in which they appear. TechnologyAdvice does not include all companies or all types of products available in the marketplace.

      ×