Data optimization provides benefits to Hadoop users that go well beyond simple data compression.
Eatontown, New Jersey (PRWEB) October 17, 2012
Altior Inc. announces availability of AltraSTAR - Hadoop Storage Accelerator and Optimizer, based on its Altraflex HW acceleration platform and CeDeFS Filter Layer software. AltraSTAR can significantly reduce Map/Reduce job execution time, increase Data Node storage capacity up to 6x, while simultaneously delivering increased I/O performance.
In a Terasort test using an eight (8) Data Node Hadoop cluster with dual six-core CPUs per node, AltraSTAR was able to reduce storage consumption by half, and improve Map/Reduce Job execution time by 15%. As a result, end-users can see a drop in the 3 Year Hadoop Cluster by as much as 30%. This TCO benefit is derived from:
- Lowest Capital Expenditure: Since storage is the most expensive component of the system reducing storage requirements by half in Data Nodes will significantly reduce initial system cost.
- Highest Rack Analytics Density: By optimizing storage capacity, Altior’s Hadoop Acceleration Platform provides the highest processing power and analytics density in a given rack. Altior’s Hadoop Acceleration Platform provides the greatest data analytics density per square foot of any Hadoop cluster - about 1 Petabyte per rack for fully populated Data Nodes.
- Faster Data Analytics: Most Hadoop Jobs are Disk IO bound, and AltraSTAR considerably improves Disk IO performance. This increased Disk IO facilitates reduces Job Execution time leading to faster Data Analytics
- Reduced Power Consumption: Altior’s FPGA based HW Acceleration consume up to 90% less power than CPU based solutions, resulting in significant savings in the expense of operating the cluster.
- Improved Reliability: Storage devices, because of moving parts, hinder reliability. By halving the Storage disks per Node, Altior’s Hadoop Acceleration solutions increase the Nodes MTBF for a given amount of Storage.
Hadoop jobs must process huge amounts of data. Efficient processing requires that as much data as possible be loaded into the Hadoop data nodes. Altior’s CeDeFS data optimization system ensures the efficient use of all available storage. Hadoop data is usually quite compressible consisting of text files, server logs, and click streams. CeDeFS’s high speed data compression can optimize available storage on the Data Nodes by as much as six-fold. In addition to enlarged storage capacity, CeDeFS also provides an important I/O acceleration service to the Hadoop cluster. Since data is compressed on disk it can be delivered more quickly to I/O bound processing tasks. The result for the Hadoop user is faster Map/Reduce execution times since the cluster’s processors will spend less time waiting idle for I/O completion and more time on application job execution. The I/O acceleration effect of CeDeFS data optimization will generally exceed that possible from solid state disks at a small fraction of the cost and over many times the capacity of an SSD.
AltraSTAR consists of the CeDeFS filter layer software for Linux and the AltraFlex data compression accelerator hardware. The AltraFlex data compression accelerator delivers over 10 Gbps of compression throughput while consuming less than 10W of power. Additional acceleration cards can be added to scale throughput even further. Compared to a multi core server class CPU, the AltraFlex data compression accelerator delivers as much as 4.25 times more compression throughput while consuming less than 1/8th the power.
AltraSTAR can be transparently integrated with any standard Map/Reduce solutions such as Cloudera’s CDH3 and CDH4 - without requiring any change in the Hadoop application or workflow. The AltraFlex data compression accelerator operates asynchronously in parallel with the host CPU. The combination of asynchronous operations and high speed compression means that CeDeFS will operate without adding any latency to the disk I/O subsystem.
“Data optimization provides benefits to Hadoop users that go well beyond simple data compression” said Ramana Jampala, Altior CEO. “With greater storage capacity larger jobs can be run on fewer nodes. Furthermore, since AltraSTAR accelerates I/O, jobs will run faster while consuming less power.”
AltraSTAR for Hadoop is available immediately. Accelerator pricing depends on the exact configuration of the accelerator card and acceleration cores and must be determined in consultation with Altior sales. For more information please contact Altior Sales at 1-732-440-1280 ext. 242 or email sales(at)altior(dot)com.
COME SEE US AT Storage Networking World in Santa Clara, October 17-19.
Founded in 2004 as CebaTech, Altior™ offers a broad range of industry-leading hardware and software solutions for the networking and storage markets. Altior products deliver high performance for increased throughput, significant energy savings, and effective network and storage optimization. Altior’s platform solutions deliver realizable value to customers by enhancing system performance, reducing development time, and achieving faster time to market. For more information, please visit http://altior.com.