This week, EMC and VMware spinoff Pivotal announced the availability of Pivotal HD 2.0, a commercial distribution of Apache Hadoop that now features support for Apache Hadoop 2.2. Moreover, Pivotal also revealed the general availability of Pivotal GemFire XD, a SQL compliant, in-memory database designed for real-time analytics for Big Data processing. In its initial release, Pivotal GemFire XD represents an in-memory distributed data store that “provides a low-latency SQL interface to in-memory table data, while seamlessly integrating data that is persisted in HDFS.” Because GemFire brings the power of real-time analytics to Hadoop, it empowers mobile providers to run complex algorithms on incoming calls to route the call appropriately, or geospatial navigation systems to alter suggested routes based on incoming data about traffic and weather conditions. Like Apache Spark, a parallel data processing framework that facilitates real-time analytics on Hadoop, GemFire enables real-time Big Data analytics but is explicitly designed for data environments with high demands for scalability and availability. Michael Cucchi, Pivotal’s senior director of product marketing, commented on Pivotal’s interest in Spark and GemFire XD in an interview with InformationWeek as follows:
We’re excited about Spark and will support it, but it’s generally used for [data] ingest or caching,” GemFire XD is an ANSI-compliant SQL database with high-availability features, and it can run over wide-area networks, so you can have an instance in Europe and another in North America with replication.
Built on the vFabric SQLFire product that belongs to the category of NewSQL databases noted for high performance and scalability, GemFire XD is adds features such as HDFS-persistence and off-heap memory storage for table data. In addition to GemFire XD, Pivotal 2.0 also features an integration with GraphLab for graphing analytics as well as enhancements to HAWQ such as support for MADlib, R, Python, Java, and Parquet. Overall, Pivotal 2.0 represents a notable advancement over Pivotal 1.1 that brings the power of YARN, real-time analytics via GemFire XD and graphing technology to Hadoop and Big Data processing and analytics. With Pivotal HD 2.0 released less than 6 months after the November 1, 2013 release of Pivotal HD 1.1, Pivotal promises to innovate in the Big Data space at the same dizzying rate with which Amazon Web Services innovates with regard to cloud computing technologies and platforms. Expect to hear more about the conjunction of real-time analytics and graphing technologies on Hadoop via Pivotal 2.0 as customer use cases proliferate and circulate throughout the Big Data space.
Cloud Computing Today 