Pivotal Embraces Hortonworks Hadoop Distribution While Hortonworks Commits To Supporting Pivotal’s Hadoop Native SQL Platform

On April 12, Hortonworks and Pivotal announced a deepening of their strategic relationship as major players in the space for commercial Hadoop distributions and Big Data analytics. Pivotal will standardize its Hadoop distribution on Pivotal HDP, a platform that is identical to the Hortonworks Data Platform. Meanwhile, Hortonworks will offer Pivotal’s SQL on Hadoop platform, Pivotal HDB, as an offering within its own portfolio under the branding Hortonworks HDB. Powered by Apache HAWQ, Hortonworks HDB will be identical to Pivotal HBD and conversely illustrates Hortonwork’s standardization on a Pivotal technology as a counterpoint to Pivotal’s embrace of the Hortonworks Data Platform as its core Hadoop distribution. The expanded collaboration between Pivotal and Hortonworks means that Pivotal aligns with one of the industry’s most widely used Hadoop distributions in the form of Hortonworks. Hortonworks, on the other hand, now can brand and offer professional services for Pivotal’s Hadoop Native SQL platform in ways that give it parity with Cloudera’s Impala. The deepening of the partnership between Pivotal and Hortonworks promises to complicate the battle for Hadoop distribution supremacy by giving Hortonworks enhanced access to Pivotal’s renowned big data application development and analytics capabilities. As such, the real winner here is Hortonworks, although Pivotal stands to gain from standardizing on HDP and thereby enabling it to focus on its core strengths in analytics and application development.

Pivotal Open Sources Its Big Data Suite And Announces Partnership With Hortonworks

Pivotal recently announced the open sourcing of key components of its Pivotal Big Data Suite. Parts of the Pivotal Big Data Suite that will be outsourced include the MPP Pivotal Greenplum Database, Pivotal HAWQ and Pivotal GemFire, the NoSQL in-memory database. Pivotal’s decision to open source the core of its Big Data suite builds upon its success monetizing the Cloud Foundry platform and intends to accelerate the development of analytics applications that leverage big data and real-time streaming big data sets. The open sourcing of Greenplum, Pivotal’s SQL on Hadoop platform HAWQ and GemFire renders Pivotal’s principal analytics and database platforms more readily accessible to the developer community and encourages enterprises to experiment with Pivotal’s solutions. Sundeep Madra, VP of the Data Product Group, Pivotal, remarked on Pivotal’s decision to open source its Big Data suite as follows:

Pivotal Big Data Suite is a major milestone in the path to making big data truly accessible to the enterprise. By sharing Pivotal HD, HAWQ, Greenplum Database and GemFire capabilities with the open source community, we are contributing to the market as a whole the necessary components to build solutions that make up a next generation data infrastructure. Releasing these technologies as open source projects will only help accelerate adoption and innovation for our customers.

Pivotal’s announcement of the open sourcing of its Big Data suite comes in tandem with a strategic alliance aimed at synergistically maximizing the competencies of both companies to deliver best-in-class Hadoop capabilities for the enterprise. The partnership with Hortonworks includes product roadmap alignment, integration and the implementation of a unified vision with respect to leveraging the power of Apache Hadoop to facilitate the capability to derive actionable business intelligence on a scale rarely performed within the contemporary enterprise. In conjunction with the collaboration with Hortonworks, Pivotal revealed its participation in the Open Data Platform, an organization dedicated toward promoting the use of Big Data technologies centered around Apache Hadoop whose Platinum members include GE, Hortonworks, IBM, Infosys, Pivotal and SAS. The Open Data Platform intends to ensure components of the Hadoop ecosystem such as Apache Storm, Apache Spark and Hadoop-analytics applications integrate with and optimally support one another.

All told, Pivotal’s decision to open source its Big Data suite represents a huge coup for the Big Data analytics community at large insofar as organizations now have access to some of the most sophisticated Hadoop-analytics tools in the industry at no charge. More striking, however, is the significance of Pivotal’s alignment with Hortonworks, which stands to tilt the balance of the struggle for Hadoop market share toward Hortonworks and away from competitors Cloudera and MapR, at least for the time being. Thus far, Cloudera has enjoyed notable traction in the financial services sector and within the enterprise more generally, but the enriched analytics available to the Hortonworks Data Platform by means of the partnership with Pivotal promise to render Hortonworks a more attractive solution, particularly for analytics-intensive use cases and scenarios. Regardless, Pivotal’s strategic evolution as represented by its open source move, its collaboration with Hortonworks and leadership position in the Open Data Platform constitute a seismic moment in Big Data history wherein the big data world shakes as the world’s most sophisticated big data analytics firm qua Pivotal unites with Hortonworks, the company responsible for the first publicly traded Hadoop distribution. The obvious question now is how Cloudera and MapR will respond to the Open Data Platform and the extent to which Pivotal’s partnership with Hadoop distributions remains exclusive to, or focused around Hortonworks in the near future.

Hortonworks and New Relic Post Impressive Gains In First Day Of IPO Trading

On Friday December 12, Hortonworks finished its first day of trading with a share price of $26.48, roughly 65% more than the IPO price of $16 per share. Hortonworks plans to raise $100M by means of 6,250,000 publicly available shares. Friday’s impressive showing bodes well for the Hadoop infrastructure and analytics market in 2015, particularly given that Hortonworks competitors are gearing up to execute IPOs in 2015 or shortly thereafter. Cloud monitoring and analytics vendor New Relic similarly gained in its first day of trading by rising 48% from $23 per share to $33.02 by the end of the day. The results represented a huge coup for venture capitalist Peter Fenton of Benchmark Capital, who serves on the board of directors of both companies. Whereas Hortonworks raised $100M in its IPO, New Relic raised $115M. The real winner in both of these IPOs, however, is Yahoo given that Yahoo owns roughly 20% of the shares of its spin-off Hortonworks and 16.8% of shares of New Relic.

Hortonworks Files For IPO

As reported by Arik Hesseldahl in Recode, Apache Hadoop vendor Hortonworks has filed for an IPO. The decision by Hortonworks to offer public shares represents the first IPO from a major Hadoop vendor. In fiscal year 2013, Hortonworks reported a loss of $36.6M relative to $11M in revenue. Meanwhile, for the first 9 months of 2014, Hortonworks increased its revenue to $33.3M but posted a loss of $86.7M. The decision by Hortonworks to go public comes after two major capital raises in 2014. In July, HP invested $50M in Hortonworks, following upon the $100M raised by Hortonworks in March. Given the gargantuan capital raises specific to Hortonworks competitors Cloudera and MapR as well, the Big Data landscape should also expect IPOs from Cloudera and MapR in the near future. Meanwhile, more detailed analysis regarding the prospects of Hortonworks executing a successful IPO will emerge in coming weeks in anticipation of the launch of the IPO either in late 2014 or early 2015. In 2011, Hortonworks was spun out of Yahoo, its principal investor. Hortonworks plans to raise up to $100M by means of its IPO.

HP Invests $50M In Hortonworks As Big Data Revolution Accelerates With Gusto

On Thursday, HP announced an agreement to invest $50M in Hortonworks. HP’s investment builds on the $100M Hortonworks raised in March in a financing red led by funds managed by Blackrock and Passport Capital as well as existing investors. The investment illustrates HP’s commitment to its reseller relationship with Hortonworks that allows it to resell the Hortonworks Data Platform. Moreover, HP plans to continue refining the engineering of its products such that they integrate with YARN, the resource management component of version 2.x of Hadoop. In addition to preparing its products to operate in conjunction with YARN, HP will be integrating its product architecture to optimally perform in conjunction with the Hortonworks Data Platform more generally. Key HP products targeted for integration with the Hortonworks Data Platform include the HP HAVEn platform, one component of which is HP Vertica. As a result of the $50M equity investment, HP’s Executive Vice President and Chief Technology Officer Martin Fink will join the board of directors of Hortonworks. HP’s investment in Hortonworks underscores how the Big Data revolution lies poised to accelerate as technology companies deepen their relationships with Hadoop vendors in anticipation of delivering turnkey big data analytics solutions that simplify and streamline the operationalization of Big Data.

Hortonworks Announces Readiness Of YARN For Apache Spark

On Thursday, Hortonworks announced that Apache Spark is “YARN Ready” and compatible with the multiple workloads and additional CPU processing-demands specific to Spark applications. As a result of the compatibility of Apache Spark with YARN, Hadoop users can now use one Hadoop cluster with a single repository of data for a variety of purposes rather than having to segment workloads such that some data is dedicated to Apache Spark. More specifically, Hadoop users can now rest assured that YARN-based applications work collaboratively with applications that leverage Spark’s capabilities to facilitate real-time analytics, interactive analytics, machine learning and stream processing. Hortonworks introduced Apache Spark to the Hortonworks Data Platform as a technology preview download in May but today announces the integration of Spark with YARN, its recent acquisition, XA Secure, for authentication and data security purposes, as well as Ambari toward the larger goal of delivering an integrated, turnkey, enterprise-grade Hadoop platform. Thursday’s announcement by Hortonworks responds to similar statements by competitors MapR regarding the integration of Spark into its Hadoop distribution, and Cloudera’s announcement of its enterprise-grade support for Apache Spark.

The following graphic illustrating the integration of Spark into YARN originated from the Hortonworks blog post Making Apache Spark YARN Ready.