Scality Targets Big Data with the First Hadoop Implementation on the Cloud Data Management Interface standard, CDMI*

Share Article

Removes the name node single point of failure and delivers a compute-in-place solution for Hadoop

scality logo
Our engineering team did a fantastic job leveraging CDMI and SOFS as key components to boost the Hadoop experience.

Scality, the revolutionary innovator in scale-out storage infrastructure, today announced a new milestone for the storage industry, with the first production-grade implementation of Apache Hadoop on CDMI.

Scality’s storage implementation for Hadoop delivers benefits long awaited by the Hadoop community, resulting in a more cost effective, easier-to-use, more resilient and higher performing Hadoop infrastructure.

  •     Eliminates the single-point-of-failure in Hadoop’s architecture by replacing Hadoop’s NameNode server with Scality’s own metadata architecture.
  •     Enables compute on the storage node itself, significantly reducing the need for data transfer by enabling in-place processing and sharing data location with the Job Tracker.
  •     Delivers high levels of data protection without the excessive hardware overhead of standard replication by making available its implementation of erasure coding technology (ARC).
  •     Eliminates the need to load files through HDFS by utilizing Scality’s Open Cloud Access (OCA): users can read and write files through a standard file system and at the same time process the content with Hadoop. In addition they benefit from the scalability of Scality’s SOFS (Scale-Out File System)

“Hadoop is an amazing set of tools to analyze large amounts of data relatively simply. Over the past 2 years, we have seen a growing demand from our customers for help solving the inefficiencies of the existing open source distributions. We have contributed our Hadoop solution to the CDMI community, thus ensuring that it can be used with any CDMI-compatible storage. However when it is deployed on Scality, users get the added benefits of RING’s self-healing, efficient data protection, and unsurpassed performance,” says Jerome Lecat, CEO of Scality. “Furthermore, our OCA strategy makes it even easier for service providers to use Hadoop without changing their application, since our CDMI framework can read data directly from the file system.”

Scality does not intend to replace or compete with existing Hadoop distributions. Instead, Scality expects to enhance existing open-source projects by delivering better storage solutions. Scality’s storage for Hadoop is compatible and has been tested with Hortonworks HDP 1.0 and Cloudera CDH4.

“Our engineering team did a fantastic job leveraging CDMI and SOFS as key components to boost the Hadoop experience,” said Philippe Nicolas, Director of Product Strategy at Scality. “Scality’s RING storage software is a very comprehensive solution. Its tremendous degree of concurrency delivers unbeatable throughput and resilience in this context,” concludes Nicolas.

About Scality

Scality redefines storage at scale to make petabyte scale storage easier, less expensive, more reliable and higher performance.

Scality’s RING Organic software turns generic x86 servers into a rock solid storage platform. The servers provide the storage media, whether it is performance oriented SSD or capacity oriented SATA, and Scality award-winning software takes care of the rest: Centralized management, data protection, self-healing, high availability and automated-tiering. At the core, Scality leverages a completely scale-out, software-based, design, based on modern object storage technology with no single point-of-failure which delivers outstanding scalability while the end to end parallel architecture provides unsurpassed performance.

With its fully standards based design, Scality RING supports POSIX file systems, REST standards such as CDMI, Hadoop and block storage to extend virtual machines within the OpenStack framework. The software solution delivers the promise of the cloud, without requiring massive changes to your application. Scality is used by cloud providers, video broadcasters, Web 2.0 portals and HPC labs to deliver service to millions of users worldwide since 2010. For more information, visit: http://www.scality.com or follow @Scality on Twitter.

*.CDMI is a cloud-oriented data management standard developed and promoted by SNIA. For further information, please refer to http://snia.org/cdmi.

Share article on social media or email:

View article via:

Pdf Print

Contact Author

Monique Shefer
Scality
4157209691
Email >
Visit website