Pepperdata Announces Cluster And Job Optimization Product For Cloud-based Hadoop Clusters On Amazon EMR

Pepperdata today announces a new product that helps Amazon EMR customers optimize the performance of their cloud-based, Hadoop jobs. Pepperdata with Amazon EMR delivers enhanced analytics related to the performance of jobs running on EMR data in addition to optimizing the performance of jobs in collaboration with instruction and feedback from users. The product gives Amazon EMR users granular visibility into cluster performance in conjunction with analytics on individual jobs that leverage metrics related to CPU, memory and unused capacity as illustrated by the graphic below:

pepperdata

Because Pepperdata translates its analytics into enhanced performance optimization on Amazon EMR, customers benefit from decreased cloud utilization as well as enhanced job performance. Sean Suchter, CTO of Pepperdata, remarked on the significance of PepperData’s product for Amazon EMR as follows:

Amazon EMR is designed to help companies process huge amounts of data easily and cost-effectively without having to commit unnecessary resources. As customers embrace Hadoop in the cloud they need to be able to manage cost and performance without any big surprises. Pepperdata eliminates those blind spots with very granular insight into the performance of current and historical EMR runs.

Here, Suchter comments on the ability of Pepperdata’s EMR product to enable customers to manage costs for Hadoop-related cloud resources while optimizing performance. Whereas Amazon Web Services EMR clusters terminate upon the completion of a run and subsequently make it difficult for users to access performance-related data, Pepperdata’s product for Amazon EMR allows users to analyze the performance of clusters and their constituent jobs even after the cluster has terminated. As a result, teams can analyze historical data to progressively improve cluster performance by determining the optimal amount of computing resources for cloud-based Hadoop jobs. Today, Pepperdata also announces the availability of Adaptive Scaling for EMR, a product that purchases Amazon EMR instances in accordance with budget and time constraints specified by clients. All told, today’s announcements from Pepperdata represent a notable addition to the space of products specializing in both infrastructure and application optimization for cloud-based Hadoop workloads. Expect to hear more from Pepperdata as big data adoption expands and companies increasingly turn their attention from deploying Hadoop clusters and their related applications toward the task of optimizing performance both at the level of clusters as well their associated jobs and applications.

Advertisement

Guest Blog Post: “The Art of the Cloud Wars” By Scott Jeschonek, Director of Cloud Solutions, Avere Systems

Editorial note: This article was authored by Scott Jeschonek, Director of Cloud Solutions, Avere Systems. The opinions expressed below are those of the author, Scott Jeschonek.

Chinese military strategist Sun Tzu once wrote that battles are won or lost before they are ever fought, but can the same be said for the cloud wars? Though many industry thought leaders have made projections, the future of how the battle between public cloud providers will unfold remains hazy. Despite projections that global IT spending will fall in 2016, investment in public cloud services is expected to grow 16% this year, fueling the fire of the on-going war. Most industry experts would agree that AWS, Google Cloud Platform, Microsoft Azure and IBM Cloud Services are the key players to watch, however many CIOs still struggle with determining which cloud service provider is right for them.

The table stakes for becoming the top public cloud provider are only getting higher with each passing quarter, and at the same time, the rise of a multi-cloud strategy is shaking things up. So, how will the cloud wars play out and how can enterprises that want to embrace the cloud choose the right CSP (or combination of CSPs) for their organizations?

When the technical features of the four hottest CSPs put them on relatively even playing field, it is the brand personalities and customer experience that can help organizations evaluate which cloud provider is most suitable for their needs. Below is a breakdown of how each CSP’s culture and decades-long experience with their respective specialties has influenced their approach to building and selling cloud offerings.

AWS

Launched officially in 2006, AWS is the oldest among the major CSPs and draws strongly from its foundations as an online marketplace to provide ease-of-use and a seamless on-demand experience. In the cloud wars landscape, AWS is like the U.S. military of the public cloud – bigger than all the others combined. AWS is very streamlined in how it offers its products, how they can be purchased and how fulfillment is handled. AWS’s ability to provide a user-friendly e-commerce experience isn’t a surprise, because, after all, AWS is Amazon. Just like you can buy a book or a suitcase from Amazon.com, you can buy compute time or storage. AWS also boasts sophisticated large-scale data base products, APIs and control structures, including AWS Lambda, which lets customers run code without provisioning or managing servers. Given Amazon’s core value of customer service, the rapid growth of AWS and its ease of use make sense.

Google Cloud Platform

While Google’s cloud offering is newer on the scene, its recent enterprise investment makes it one to watch. The company’s history of being forward-thinking and cutting edge is carried throughout its cloud platform value prop. At its NEXT conference in March, Google focused its keynotes on the sophistication of its technology, and how savvy developers could take advantage of Google Cloud Platform’s dynamic point-to-point networking, artificial intelligence and machine learning offerings. Google arguably has the most sophisticated developer cloud on the market, which makes sense as one of the large-scale inventors of web scale, or scale-out computing.

IBM

IBM, which combined its July 2013 acquisition of SoftLayer Technologies with its IBM SmartCloud to form the IBM Cloud Services Division, is building on its historical strength as a provider of consulting and computer services. Professional services and consulting is not necessarily top of mind for AWS or GCP, so IBM’s global cloud platform appeals to those customers looking for a stronger engagement model. Professional services is part and parcel to IBM, and it can combine its cloud offerings and consulting capabilities with other technologies such as Watson.

Azure

Rounding out the four, Microsoft’s Azure has a strong and steady history in the CSP market. Though it offers many of the same features as the other three “big guns,” it has stood out by intelligently leveraging its long-standing strength in the enterprise and bridging its existing enterprise capabilities with its newer cloud offerings. Customers can run other Microsoft software offerings such as Exchange, SQL Server or Active Directory both in the cloud or on premises. Microsoft also offers licensing incentives for existing enterprises customers as an option to help them embrace Azure.

If your organization is, for example, a next-generation application outfit and has been in the cloud from the get-go, then AWS or GCP will suit you just fine. If you’re generating an Internet-of-Things-based application with a mobile front end, for instance, that will run on Android and iOS and you’re storing a lot of data, any one of the four will handle the job admirably.

But here’s where it gets a little tricky: if you’re an enterprise customer with your own (or leased) data center(s) and Microsoft applications and backoffice applications processing lots of data (a bank, for example), it’s a much bigger proposition to move to the public cloud. This type of customer is currently a challenge for all four of the major CSPs. For organizations with a more traditional IT infrastructure, it’s not a matter of simply copying and pasting their applications and technology into the cloud and calling it a day.

A Demilitarized Zone: The Rise of the Multi-cloud Strategy

Partly in response to these complexities, we’ve seen the rise of hybrid cloud and multi-cloud architectures and approaches. Multicloud, in particular, has been an unexpected twist in enterprise cloud adoption. CSPs catalyzed enterprise cloud adoption by driving prices lower and enhancing the sophistication of their offerings, yet now this very same competitive dynamic is allowing businesses to choose different clouds for different workloads based on the strength of each CSP.

There’s also the age-old concern about having a single supplier and being subject to vendor lock-in. By adopting a multi-cloud approach, enterprises can avoid the “data gravity” problem: as data accumulates there is a greater likelihood that more and more additional services and applications will be attracted to this data. By keeping data in different clouds, enterprises can avoid arduous and difficult data migration while taking advantage of differing pricing structures among the CSPs.

In the end, yes, we’re going to see an intensification of the cloud wars among the four big public cloud providers – Amazon, Google, IBM and Microsoft – and that’s a good thing for enterprises moving to the cloud. This competition is at once driving down prices, increasing buyer options and inspiring innovations in IT architecture that will ultimately lead to more freedom of choice and the ability to purpose-build cloud environments for enterprises who are looking to the cloud(s).

AWS Acquires Cloud9, Startup That Delivers An Integrated Development Environment For The Cloud

Amazon Web Services recently purchased Cloud9, a San Francisco-based startup that delivers an integrated development environment (IDE) that empowers developers to build and collaborate with respect to the development of applications. Cloud9 allows developers to create and edit code from a multitude of development frameworks in the cloud. Cloud9’s integrated development environment (IDE) empowers developers to collaborate using an Ubuntu workspace that supports programming languages that include C++, C#, HTML, Java, Javascript, Node.js, PHP, Python and Ruby. In addition, the platform enables developers to perform compatibility testing of applications across a variety of browsers. In a blog post, Cloud9 CEO and co-founder Ruben Daniels remarked on Cloud9’s transition to AWS as follows:

In the meantime, you’ll still be able to depend on and continue to invest safely in Cloud9. It’s still business as usual—we’ll continue to work with our Ace Open Source community and to provide our innovative services to you and our hundreds of thousands of customers worldwide. Over time, we’ll work with AWS to do even more on your behalf.

Daniels, here, notes how Cloud9’s “hundreds of thousands of customers worldwide” can continue to safely use Cloud9 for the immediate future although he expects the collaboration with AWS will enable the delivery of “even more” features and functionality, going forward. Given its focus on empowering developers to write software in the cloud, Cloud9 stands to enhance the ability of AWS customers to develop applications within cloud-based environments as opposed to writing code on-premise and then transporting that code to the cloud. Cloud9’s customers include Atlassian, MailChimp, Mozilla and Salesforce.com. The acquisition of Cloud9 by AWS enriches its portfolio of options for writing code directly within the Amazon Web Services infrastructure and, as such, gives it a competitive advantage over the likes of Microsoft Azure and the Google Cloud Platform. Terms of the acquisition were not disclosed.

AWS Launches Asia Pacific (Mumbai) Region

Amazon Web Services today announced the launch of the Asia Pacific (Mumbai) Region. As a result of the launch, AWS delivers its cloud products and services across 13 regions and a total of 35 availability zones, globally. Today’s launch of the Asia Pacific (Mumbai) Region taps into a market that can serve more than a billion Indians and promises to significantly boost its market presence in India from its existing customer base of roughly 75,000 India-based customers. The AWS Asia Pacific (Mumbai) Region will initially consist of two distinct availability zones.

InfoReliance Secures Contracts With The Justice Department And The Government Accountability Office To Deliver Managed AWS IaaS And PaaS Cloud Services

InfoReliance, an AWS reseller and technology consulting firm, has been awarded contracts to deliver Amazon Web Services to The Justice Department and the Government Accountability Office. InfoReliance will deliver managed Amazon Web Services IaaS and PaaS services that helps customers operationalize AWS public clouds in collaboration with InfoReliance’s consultative support, guidance and professional services. InfoReliance will also provide Amazon Web Services training and guidance on IT architecture as well as technical support. Headquartered in Fairfax, Virginia, InfoReliance has delivered IT and cloud computing consulting services to over 30 government and defense agencies since its inception in 2000. The contract awarded to InfoReliance underscores the potential for public cloud vendors to increase market share within the lucrative government sector space as well as the opportunity for managed service providers to help government agencies navigate the process of deploying and managing a public cloud.

Salesforce Designates Amazon Web Services As Preferred Public Cloud Vendor

On May 25, Salesforce announced its selection of Amazon Web Services as its preferred public cloud infrastructure provider. The designation of AWS as its preferred public cloud vendor consolidates its long-standing relationship with Amazon Web Services and facilitates the global expansion of “core services” such as the Sales Cloud, Service Cloud, App Cloud, Community Cloud and Analytics Cloud. Additionally, credible reports indicate that Salesforce plans to spend roughly $400M on Amazon Web Services. Currently, Heroku, Marketing Cloud Social Studio, SalesforceIQ, and the Salesforce IoT Cloud run on the Amazon Web Services public cloud. The deepening of the relationship between Salesforce and Amazon Web Services marks a huge coup for AWS insofar as it underscores its ability to court the enterprise for computing services for critical and everyday business operations. The partnership between Salesforce and AWS also indicates decreased reliance on Oracle hardware on the part of Salesforce and presages a more profound infrastructure transformation away from Oracle-based datacenters toward the AWS cloud.