Close
  • Latest News
  • Big Data and Analytics
  • Cloud
  • Networking
  • Cybersecurity
  • Applications
  • IT Management
  • Storage
  • Sponsored
  • Mobile
  • Small Business
  • Development
  • Database
  • Servers
  • Android
  • Apple
  • Innovation
  • Blogs
  • PC Hardware
  • Reviews
  • Search Engines
  • Virtualization
Read Down
Sign in
Close
Welcome!Log into your account
Forgot your password?
Read Down
Password recovery
Recover your password
Close
Search
Logo
Logo
  • Latest News
  • Big Data and Analytics
  • Cloud
  • Networking
  • Cybersecurity
  • Applications
  • IT Management
  • Storage
  • Sponsored
  • Mobile
  • Small Business
  • Development
  • Database
  • Servers
  • Android
  • Apple
  • Innovation
  • Blogs
  • PC Hardware
  • Reviews
  • Search Engines
  • Virtualization
More
    Home Database
    • Database

    Hortonworks Acquires Onyara, Launches DataFlow Product

    By
    Darryl K. Taft
    -
    August 25, 2015
    Share
    Facebook
    Twitter
    Linkedin
      big data

      Hortonworks, a provider of enterprise Hadoop solutions and services, announced it has signed an agreement to acquire Onyara, the creator of and key contributor to Apache NiFi.

      A top-level open-source project at the Apache Software Foundation (ASF), Apache NiFi is an easy-to-use system to process and distribute data. The acquisition will make it easy for users to automate and secure data flows and to collect, conduct and curate real-time business insights and actions derived from data in motion.

      “The NiFi user interface and ease of extension have made it extremely easy to get up and running and even customize,” said Craig Connell, CTO at Leverege, which develops software for users to manage and visualize large networks of diverse sensors, including Internet of things (IoT) environments. “It is great that NiFi also easily integrates with other parts of the Apache big data world like Spark, Kafka and Hadoop.”

      As a result of the acquisition, Hortonworks is introducing Hortonworks DataFlow powered by Apache NiFi, which is complementary to the Hortonworks Data Platform (HDP). The transaction is expected to close in the third quarter of 2015.

      Explaining the impetus behind NiFi, Joe Witt, CTO at Onyara told eWEEK: “We saw that it was difficult to see and control how data flows from access points as small as a sensor back to the data center. And there was disjoint logging and no real bidirectional intelligence as data was in motion. So we created ‘Niagara Files’ to address this, which through the NSA Technology Transfer was donated to the Apache Software Foundation and became Apache NiFi. NiFi makes it easy to automate and secure IoAT [Internet of anything] data flows that collect, conduct and curate real-time business insights and actions derived from sensors, machines, geo-location devices, clicks, logs, social feeds, etc.—the IoAT.”

      In addition, Witt and his team recognized the need for more comprehensive security and governance when it comes to IoAT. As more and more data is generated from IoAT, including data from sensors, geo-location devices, server logs, clicks, machines, social feeds, as well as any other data source at the edge, securely ingesting and processing the data from the “jagged edge” is an issue, he said. Customers and developers have had no choice but to create custom, disjointed and loosely integrated solutions to solve the problem of analyzing data and providing insights. However, NiFi provides fine-grained provenance metadata that supports compliance and governance as well as secure end-to-end data routing, including encryption and compression.

      “Hortonworks is focused on doing everything possible to enable our customers to transform their business through data-driven insights and actions,” Rob Bearden, CEO of Hortonworks, said in a statement. “Onyara’s impressive work on security and simplicity in NiFi, combined with their commitment to open-source makes for a perfect addition to our technology team.”

      Hortonworks officials said a new data paradigm that includes data from machines, sensors, geo-location devices, social feeds, click streams, server logs and more is fueling the Internet of anything and driving the need for trusted insights from data at the very edge to the data lake in real time with full fidelity. Many IoAT applications need two-way connections and security from the edge to the data center. This results in a “jagged edge” that increases the need for security but also data protection, governance and provenance. These applications also need access to both data in-motion and data at-rest.

      Hortonworks Acquires Onyara, Launches DataFlow Product

      Moreover, while many of today’s systems are custom-built, loosely secured, difficult to manage and not integrated, Hortonworks DataFlow powered by Apache NiFi will simplify and accelerate the flow of data in motion into HDP for full fidelity analytics. Combined with HDP, these complementary offerings will give customers a holistic set of secure solutions to manage and find value in the increasing volume of streaming IoAT data, Hortonworks said.

      “Simply put, dataflow is the next generation of big data analysis—and could entirely disrupt the now-decade-old Hadoop ecosystem,” said Apurva Dave, vice president of marketing at Jut, which provides an operations data hub unifying and correlating logs, metrics and events for DevOps, developer and IT operations. “Dataflow deals with streaming data—like Internet of things and real-time usage information—as well as data at rest. This technology is an effort for Hortonworks to make data more useful for their base of users and their respective businesses. Hortonworks will be competing both with dataflow cloud services, such as Google, as well as pure-play dataflow technologies like Jut.”

      Witt argues that the new Hortonworks solution stands alone. “The new Hortonworks DataFlow powered by Apache NiFi is unique in the market today,” he said. “There really isn’t one solution that can address IoT data streams at scale, securely, with governance and bidirectional intelligence. IoAT gets really complex due to sheer volume of data, and manageability is tough. Through the combined use of Hortonworks Data Platform and Hortonworks DataFlow, data at rest as well as real-time data in motion can now be blended to provide historical and perishable insights for predictive analytics.”

      Released through a National Security Agency (NSA) technology transfer program in the fall of 2014, Apache NiFi then entered into incubation at ASF and became an Apache Top-Level Project. Over the past eight years, Onyara’s engineers were the key contributors to the U.S. government software project that evolved into Apache NiFi.

      Witt worked at the NSA, and the rest of the team came from the defense industry and have worked closely together for several years. “Back in 2006, I created the software and then with support from very talented developers and senior leadership we were able to drive the technology further to solve unique problems seen at NSA,” he said. The technical co-founders of Onyara are all committers to the Apache NiFi project and remain active within the growing community.

      “Nearly a decade ago, when IoAT began to emerge, we saw an opportunity to harness the massive new data types from people, places and things, and deliver it to businesses in a uniquely secure and simple way,” Witt said in a statement. “We look forward to joining the Hortonworks team and continuing to work with the Apache community to advance NiFi.”

      Alexandar Ryabov, senior director of data engineering at Wargaming.Net, said, “NiFi addresses dataflow challenges we have right now and provides upside for where we’re heading. It is also a big win for us that it is designed for the global enterprise.”

      Added Mike Bishop, chief systems architect at Prescient Edge: “NiFi’s well designed, mature API has made our integration process remarkably straightforward. With it, we’re able to track the origin, transformation, and persistence of data throughout our analytic processes.”

      Darryl K. Taft
      Darryl K. Taft covers the development tools and developer-related issues beat from his office in Baltimore. He has more than 10 years of experience in the business and is always looking for the next scoop. Taft is a member of the Association for Computing Machinery (ACM) and was named 'one of the most active middleware reporters in the world' by The Middleware Co. He also has his own card in the 'Who's Who in Enterprise Java' deck.

      MOST POPULAR ARTICLES

      Android

      Samsung Galaxy XCover Pro: Durability for Tough...

      Chris Preimesberger - December 5, 2020 0
      Have you ever dropped your phone, winced and felt the pain as it hit the sidewalk? Either the screen splintered like a windshield being...
      Read more
      Cloud

      Why Data Security Will Face Even Harsher...

      Chris Preimesberger - December 1, 2020 0
      Who would know more about details of the hacking process than an actual former career hacker? And who wants to understand all they can...
      Read more
      Cybersecurity

      How Veritas Is Shining a Light Into...

      eWEEK EDITORS - September 25, 2020 0
      Protecting data has always been one of the most important tasks in all of IT, yet as more companies become data companies at the...
      Read more
      Big Data and Analytics

      How NVIDIA A100 Station Brings Data Center...

      Zeus Kerravala - November 18, 2020 0
      There’s little debate that graphics processor unit manufacturer NVIDIA is the de facto standard when it comes to providing silicon to power machine learning...
      Read more
      Apple

      Why iPhone 12 Pro Makes Sense for...

      Wayne Rash - November 26, 2020 0
      If you’ve been watching the Apple commercials for the past three weeks, you already know what the company thinks will happen if you buy...
      Read more
      Logo

      eWeek has the latest technology news and analysis, buying guides, and product reviews for IT professionals and technology buyers. The site’s focus is on innovative solutions and covering in-depth technical content. eWeek stays on the cutting edge of technology news and IT trends through interviews and expert analysis. Gain insight from top innovators and thought leaders in the fields of IT, business, enterprise software, startups, and more.

      Facebook
      Linkedin
      RSS
      Twitter
      Youtube

      Advertisers

      Advertise with TechnologyAdvice on eWeek and our other IT-focused platforms.

      Advertise with Us

      Menu

      • About eWeek
      • Subscribe to our Newsletter
      • Latest News

      Our Brands

      • Privacy Policy
      • Terms
      • About
      • Contact
      • Advertise
      • Sitemap
      • California – Do Not Sell My Information

      Property of TechnologyAdvice.
      © 2021 TechnologyAdvice. All Rights Reserved

      Advertiser Disclosure: Some of the products that appear on this site are from companies from which TechnologyAdvice receives compensation. This compensation may impact how and where products appear on this site including, for example, the order in which they appear. TechnologyAdvice does not include all companies or all types of products available in the marketplace.

      ×