eWEEK content and product recommendations are editorially independent. We may make money when you click on links to our partners. Learn More.
Appistry on June 29 “announced strategic alliances with leading Hadoop ecosystem vendors Concurrent, Datameer and Kitenga,” the company said in a news release.
With the announcement, made at the Hadoop Summit on June 29, “Without any changes to their applications, users of the Concurrent, Datameer and Kitenga products can choose Appistry CloudIQ Storage as a more robust file system for their enterprise-grade Hadoop deployments. CloudIQ Storage is fully decentralized and offers higher availability and greater reliability than HDFS [Hadoop Distributed File System], whose ‘NameNode’ is a single point of failure and network bottleneck,” said Sam Charrington, vice president of products for Appistry.
Moreover, “Each of the three vendors’ products has been validated against Appistry CloudIQ Storage Hadoop Edition, which offers plug-and-play compatibility with (HDFS), and each will work with Appistry to support joint customers using CloudIQ Storage in place of HDFS.”
As Microsoft CEO Steve Ballmer said, “It’s all about ‘Developers, Developers, Developers,’ right,” Charrington said in an interview with eWEEK. “Storage issues, and the single point of failure inherent in HDFS, are a huge pain point for Hadoop users.”
He added that a key question that arose at the Hadoop Summit was, “Are there any plans to build reliability into HDFS?” According to Charrington, “Yahoo’s answer was no, but you can try what they do internally: Rewrite your applications to post critical data to multiple Hadoop clusters.”
However, that represents a level of complexity for developers, he said.
“Our approach with CloudIQ Storage is to offer a Hadoop Adapter, which allows users to transparently replace HDFS with CloudIQ Storage, gaining a more reliable file system designed from the ground up to provide mission-critical levels of reliability,” Charrington said.
As the Hadoop Summit indicates, and as Appistry said, “A rich ecosystem of vendors has emerged with tools to support enterprise adoption and use of Apache Hadoop. These vendors each add unique value by simplifying the development of analytical applications on Hadoop and hiding the complexity of the underlying MapReduce framework.” Yet, although “these new tools simplify Hadoop’s ‘front end,’ the reliability of the HDFS ‘back end’ and the single point of failure in its NameNode [remain] a barrier for enterprises seeking to deploy Hadoop for mission-critical applications,” the company said.
“Cloud computing and ‘big data’ are driving a convergence of application and storage in order to meet the scalability and performance requirements of a new generation of data-intensive applications,” Charrington said. “Appistry’s CloudIQ Platform delivers this through what we call ‘computational storage,’ in which we dynamically migrate application workloads across a cloud based on the data requirements of a particular application-level operation.”
Indeed, “Hadoop is a good example of this convergence as well, being both an application platform and a storage platform,” Charrington added. “With our Hadoop Edition we cater to developers and users that want to take advantage of the Hadoop application platform (i.e. MapReduce) but who also need the high levels of reliability we offer with CloudIQ Storage. Hadoop Edition provides an adapter that allows them to swap out HDFS for our stuff without changing any of their tools or applications, and the partner support is strong validation of this.”
“Hadoop is an ecosystem, not a company, and Appistry CloudIQ Storage makes that ecosystem more powerful,” Kevin Haar, CEO of Appistry, said in a statement. “By providing an enterprise-grade storage solution for Hadoop with plug-and-play compatibility with existing tools and applications, Appistry is helping to expand the reach of Hadoop in the enterprise by addressing the HDFS pain points of reliability and robustness.”
According to Appistry, “Concurrent develops software to help enterprises realize the full benefits of parallel computing clusters. Concurrent has released the Cascading open-source project, a layer over MapReduce that allows developers to rapidly create complex data processing and computing applications through a straightforward Java API. Cascading provides simpler abstractions to define applications and glue them together, as well as interfaces to work with external systems.”
“CloudIQ Storage offers an important option for customers that count on Hadoop, and we’re excited to see Cascading perform quite well on this alternative storage platform,” Chris Wensel, founder and CTO of Concurrent, said in a statement. “Our testing showed full compatibility with Cascading and Hadoop MapReduce, making adoption quick and easy for users.”
Appistry said, “Datameer offers … a big data analytics solution built on Hadoop that helps business users access, analyze and use massive amounts of data. Founded by Hadoop veterans in 2009, the company’s breakthrough product, Datameer Analytics Solution (DAS), provides unparalleled access to data with minimal IT resources. DAS scales to 4,000 servers and petabytes of data and is available for all major Hadoop distributions including Apache, Cloudera, Yahoo, IBM and Amazon.”
“The powerful combination of Datameer’s big data analytics and Appistry’s cloud application platform provides the Hadoop user community with an easy-to-use solution that supports even the most rigorous mission-critical application requirements,” Ajay Anand, CEO of Datameer, said in a statement. “Our partnership will open up more opportunities for new and existing customers to use Hadoop to tackle their big data challenges.”
Finally, Appistry said, “Kitenga provides a multimedia content mining solution for non-programming professionals, designed to reduce or eliminate the complexity, risk and cost typically associated with implementing massively scaled-out multimedia content mining solutions. Kitenga’s ZettaVox application combines ready-to-run content processing elements with a content mining/analytics solution that distributes processing over cloud and cluster-based computing assets by leveraging the Hadoop framework. The entire process can be monitored, batched, and the results analyzed using visualization and analytics tools.”
“Kitenga is pleased to announce support for Appistry’s CloudIQ platform. CloudIQ makes it easy to scale out and manage Hadoop instances. Moreover, Appistry has developed a product called CloudIQ Storage Hadoop Edition that provides a high-availability distributed file system replacement for Hadoop’s native system and eliminates the weaknesses of the standard Hadoop file system,” Mark Davis, CTO of Kitenga, said in a statement. “The combination of CloudIQ and ZettaVox means that organizations can create enterprise-grade content mining solutions that are managed using a robust cloud management platform.”