On May 16, Crate.io announced the availability of CrateDB 2.0, an open source SQL database that specializes in IoT and machine data. The innovation of CrateDB consists in leveraging SQL to aggregate and perform real-time analytics on IoT and machine data instead of the NoSQL databases commonly used in the industry for related use cases. CrateDB’s ability to accommodate the ingestion of high velocity streams of data and to perform queries on rapidly changing datasets, with impressive levels of scalability and latency, allows developers to combine their familiarity with SQL alongside a solution specially designed for the unique needs of IoT and machine data applications. CrateDB 2.0 features clustering upgrades that deliver improved query performance by means of faster aggregations and new index structures. In addition, CrateDB 2.0 contains a bevy of SQL enhancements that give developers a greater range of options regarding joins, sub-selects and the renaming and re-indexing of tables. The Enterprise Edition of CrateDB 2.0 offers performance monitoring, enhanced security as well as the ability for end users to create user-defined functions. CrateDB 2.0’s clustering upgrades, SQL enhancements and enterprise-grade security and performance monitoring mark a new milestone in the platform’s evolution that testifies to its readiness to embrace enterprise-grade workloads that include sensor data, GPS data and the industrial internet more generally. Subsequent to news of its general availability in December 2016, Crate.io’s release of open source and enterprise-grade versions of CrateDB underscores the early traction the platform has received, with over 1.3 million downloads and 50 customers using Crate.io in production. With the IoT and machine data space gearing up for a rampant proliferation of devices and corresponding datasets in forthcoming years, expect Crate.io to continue building on its recent momentum, particularly as organizations look for scalable databases that allow organizations to leverage widely available skillsets in SQL.
The graphic below illustrates the platform’s Enterprise Edition user interface for monitoring the performance of clusters gives users real-time visibility into cluster performance with respect to the ingestion and transformation of IoT and machine data:
On October 8, Amazon Web Services announced the release of AWS IoT, its platform for Internet of Things data, at its AWS re:Invent conference in Las Vegas. The AWS IoT platform facilitates the connection of devices such as sensors, automobiles and appliances to enable data acquisition of internet of things data, the application of rules to ingested data and the ability to remain connected to devices, even when they are offline. Devices connect to AWS IoT by means of a Device Gateway that leverages the HTTP and Message Queue Telemetry Transport protocol for sensors and mobile devices. Upon ingestion, the AWS IoT rules engine allows customers to create rules to route and filter data and send alerts and notifications to connected devices and related applications. The AWS IoT rules engine empowers customers to route data to the appropriate AWS product (Redshift, Kinesis, S3, Lambda, etc.) for long-term storage and analytics. Devices can stay connected to AWS IoT even when they are offline by means of the platform’s ability to create a virtual, “shadow” version of the device that enables other devices to interact with the latest available data for that device. The AWS IoT platform builds upon Amazon’s acquisition of 2lemetry in March 2015 and represents a much awaited addition to the cloud behemoth’s portfolio, particularly given the proliferation of competing products from Microsoft, Pivotal and ParStream. Expect AWS to integrate advanced analytics into its IoT platform that enhance the insights currently available via its rules engine in upcoming months.
Benjamin Black, one of the key resources behind the concept and implementation of Amazon Web Services, joined Pivotal Cloud Foundry as Senior Director of Technology in late February. Black collaborated with Chris Pinkham to write the proposal to build Amazon’s Elastic Cloud Compute (EC2) platform that was subsequently approved by Amazon CEO Jeff Bezos. In his new role at Pivotal Cloud Foundry, Black will lead an Internet of Things lab in Seattle that represents the cusp of Pivotal’s thought leadership about IoT data aggregation and analytics. After leading an engineering team at Amazon Web Services, Black worked at Microsoft before becoming CEO and Founder of Boundary. Black’s addition to the Pivotal Cloud Foundry team constitutes the latest of high profile hires including Joshua McKenty, former CEO of Piston Cloud Computing and OpenStack co-founder and Andrew Clay Shafer, co-founder of Puppet Labs. The announcement of Black’s hire comes head on the heels of Pivotal’s decision to open source its Big Data Suite and enter into a strategic partnership with Hortonworks. Pivotal is a spinoff of EMC and VMware that aims to drive a transformation of contemporary IT by bringing the power of cloud computing, Big Data, agile application development and real-time analytics to the modern enterprise.
Santa Clara-based machine data analytics vendor Glassbeam recently revealed details of a new version of Glassbeam SCALAR marked by deep integration with Apache Spark. Apache Spark is a parallel data processing framework that facilitates real-time analytics, machine learning and real-time analytics by storing the results of data operators in memory and performing low latency, iterative calculations on in memory computational results. Known for its ability to automate the parallelization of tasks and jobs, Spark boasts operational efficiencies over MapReduce by a factor of 100 with respect to the execution of calculations on large datasets. Glassbeam SCALAR’s integration with Apache Spark enhances its computational capabilities as well as the platform’s machine learning functionality and capacity to perform real-time analytics on streaming datasets by means of the Spark Streaming and MLLib components of the Spark stack. Built on Cassandra, Spark’s addition to the Glassbeam’s cloud analytics platform gives it the benefits of Cassandra’s distributed data management architecture in addition to Spark’s computational, analytic and machine learning functionality. As such, today’s announcement strengthens Glassbeam’s position in the nascent but exploding internet of things analytics space by augmenting its ability to ingest, process and analyze massive amounts of data as well as enhancing Glassbeam SCALAR’s advanced analytics, machine learning and predictive analytics capabilities.