Sqlstream

SQLstream is a distributed, SQL standards-compliant plus Java stream processing platform. SQLstream, Inc. is based in San Francisco, California and was launched in 2009 by Damian Black, Edan Kabatchnik and Julian Hyde, author of the open source Mondrian Relational OLAP Server Engine.

Technology

The rapid increase in the volume of available service, device and sensor data has led to new, real-time market segments which augment the traditional monitoring, business intelligence and data warehousing domains.^[1] The Internet of Things promises to bring hundreds of billions of connected devices to the Internet, all streaming out data that need to be processed in aggregate in real-time in order to power smart services that can react and respond to their environment through these sensors. Stored data analytics systems where one continually updates the data store with newly arriving data and re-traverse the stored data in order to perform analysis on the data do not scale up to the very large volumes of data emitted in the Internet of Things. They are not designed for issuing queries or analyses for each of millions of records per second. This is where technologies like SQLstream come in, that process the data incrementally and continually, without first storing the data. Such an approach is called Stream Processing.

SQLstream provides a relational stream processing platform called SQLstream Blaze for analyzing large volumes of service, sensor and machine and log file data in real-time. It performs real-time collection, aggregation, integration, enrichment and real-time analytics on the streaming data. Data streams are analyzed using the industry standard SQL language, using the ANSI standard, functionally rich SQL window function to analyze and aggregate real-time streaming data over fixed or sliding time windows, which can be further partitioned by user defined keys. Unlike a traditional RDBMS SQL query, which returns a result and exits, streaming SQL queries do not exit, generating results continuously as soon as new data become available. Patterns and exception events in data streams are detected, analyzed and reported 'on the fly' as the data arrive, that is, before the data are stored. Like a database or data warehouse, SQLstream allows you to create multiple views over the data so that different applications and users can each get their own customized view of the streaming data. The partitioning allows many different analytics to be incrementally computed using a single SQL statement or window., effectively processing potentially millions of streams with a single statement. For example, partitioning by a customer id would maintain a separate computation for each distinct customer. This is extremely concise, but also allows for efficient parallel execution. SQLstream Blaze also allows changes to be made to the queries and views without bringing down and recompiling existing applications. This is very important for many Internet of Things and other smart services that must operate 24x7 on a continuous real-time basis, where application changes must be made without needing to bringing down the service or rebuild the application. Part of SQLstream Blaze, StreamLab takes advantage of this capability in order to guide users who wish to explore data streams and understand their structure while the data are still flowing by generating new SQL queries on the fly based on user direction and analysis of data values driven by rules. In this way, it provides an effective platform for performing real-time operational intelligence, which you can view as real-time business intelligence over streaming operational data. SQLstream utilizes dataflow technology to execute many queries over high-velocity high-volume Big Data with a massively parallel standards-compliant SQL engine where the queries are executed concurrently and incrementally. Unlike databases, SQL in SQLstream becomes a language for performing continuous parallel processing, in contrast to a language for data retrieval as commonly found in relational databases. SQLstream is able to execute its queries in an optimized C++ multi-threaded dataflow engine which operates lock-free. This enables people to create lock-free parallel processing applications easily, which otherwise require specialist skillets and are often difficult to get working and are often error prone.

Applications of SQLstream Blaze include real-time service and sensor (Internet of Things) data management, real-time data integration, streaming log file analytics and real-time data warehousing.^[2][3] SQLstream Blaze provides an effective way of processing large volumes of data in real-time enabling a wide variety of smart services to be real-time responsive to streaming data even at massive data volumes.

Products

SQLstream launched its first product to market in January 2008. Its stream processing software is called SQLstream Blaze, which comprises s-Server, s-Studio, s-Dashboard, s-Visualizer and StreamLab. It also has streaming application templates that can be edited and extended by users and these are known as StreamApps. SQLstream has a Smart Cities StreamApp, a Telecom StreamApp and an emergency services StreamApp. Professional services for strategy, certification, and training are also offered on the website.

See also

• Operational Intelligence
• Extract, transform and load (ETL)
• Wireless sensor network
• Intelligent Transportation Systems
• Real-time business intelligence

Notes

External links

• SQLstream website