The Apache Hadoop community of developers recently announced the release of the second alpha release of Apache Hadoop known as Apache Hadoop 2.0.2-alpha. The release features enhancements such as the following:
•HDFS HA enhancements including support for automated failover using ZooKeeper and support for security
•YARN testing and stabilization
YARN, which stands for “Yet Another Resource Negotiator,” provides a framework for creating distributed processing applications and infrastructures. YARN additionally provides an apparatus for scheduling requests of resources such as CPUs and manages the execution of such requests. Harsh Chouraria’s excellent blog post explains how YARN differs from MapReduce 2.0 by noting that “YARN is a generic platform for any form of distributed application to run on, while MR2 is one such distributed application that runs the MapReduce framework on top of YARN.” YARN has already been deployed on massive clusters totaling almost 6000 nodes at Yahoo.
The Hadoop community is now close to the release of Hadoop-2.x sometime early in 2013, which will feature final tweaks on functionality such as:
•HDFS without shared storage
•YARN scheduling enhancements
Developers can download the latest Hadoop release from the Apache Hadoop page or Hortonworks Data Platform 2.0 Alpha, the latter of which integrates with additional frameworks such as Apache Pig and Apache Hive.