Apache Hadoop 2.0.2-alpha Release Features Enhancements To HDFS HA & YARN

The Apache Hadoop community of developers recently announced the release of the second alpha release of Apache Hadoop known as Apache Hadoop 2.0.2-alpha. The release features enhancements such as the following:

•HDFS HA enhancements including support for automated failover using ZooKeeper and support for security
•YARN testing and stabilization

YARN, which stands for “Yet Another Resource Negotiator,” provides a framework for creating distributed processing applications and infrastructures. YARN additionally provides an apparatus for scheduling requests of resources such as CPUs and manages the execution of such requests. Harsh Chouraria’s excellent blog post explains how YARN differs from MapReduce 2.0 by noting that “YARN is a generic platform for any form of distributed application to run on, while MR2 is one such distributed application that runs the MapReduce framework on top of YARN.” YARN has already been deployed on massive clusters totaling almost 6000 nodes at Yahoo.

The Hadoop community is now close to the release of Hadoop-2.x sometime early in 2013, which will feature final tweaks on functionality such as:

•HDFS without shared storage
•YARN ResourceManager
•YARN scheduling enhancements

Developers can download the latest Hadoop release from the Apache Hadoop page or Hortonworks Data Platform 2.0 Alpha, the latter of which integrates with additional frameworks such as Apache Pig and Apache Hive.


Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: