CrateIO

CrateIO is a fully searchable document oriented data store. CrateIO is open source, written in Java and based on a shared nothing architecture. CrateIO is designed for high scalability and includes components from Facebook Presto, Apache Lucene, Elasticsearch and Netty.

History

Jodok Batlogg, an active open source contributor and creator since 2004, beginning with the Open Source Initiative Vorarlberg^[1] began the Crate project while at Lovely Systems in Dornbirn. As more data was added to the applications developed at Lovely Systems for its customers, he and Bernd Dorn observed that storing documents, blobs and supporting real-time searches was not simple to support and that a horizontally scalable solution was needed for data persistence. The data store was given the brand Crate in 2013 and the domain was purchased from a PyPi maintainer.^[2] The company raised its first round in April 2014.^[3] In June 2014 CrateIO won the judge's choice award at the GigaOm Structure Launchpad competition^[4][5] and in October 2014 they win the Techcrunch Disrupt Europe in London.^[6][7] The developer community is meeting at mountain hackathons.^[8]

Overview

CrateIO's language is SQL but it uses the document-oriented approach of NoSQL style databases. Crate uses the SQL parser from Facebook Presto, its own query analysis and distributed query engine. Elasticsearch and Lucene is used for the transport protocol and cluster discovery and Netty for asynchronous event driven network application framework.

CrateIO includes a built-in Administration Interface. The Command Line interface (Crate Shell – CraSh) allows interactive queries. Crate’s Python client is most advanced and features SQLAlchemy integration.

Example

This example uses a database with tweets and the crash command-line SQL shell shipped with CrateIO. First, a connection to a running node is established.

cr> connect 127.0.0.1:4200;
+------------------------+-----------+-----------+---------+
| server_url             | node_name | connected | message |
+------------------------+-----------+-----------+---------+
| http://127.0.0.1:...   | crate     | TRUE      | OK      |
+------------------------+-----------+-----------+---------+
CONNECT OK

Create the database:

cr> create table tweets (
...   created_at timestamp,
...   id string primary key,
...   retweeted boolean,
...   source string INDEX using fulltext,
...   text string INDEX using fulltext,
...   user_id string
... );
CREATE OK (... sec)

Inserting the first tweet:

cr> insert into tweets values(1394182937, '1', true, 'web', 'Don''t panic', 'Douglas');
INSERT OK, 1 row affected (... sec)

one more tweet

cr> insert into tweets values(1394182938, '2', true, 'web', 'Time is an illusion. Lunchtime doubly so', 'Ford');
INSERT OK, 1 row affected (... sec)

Looking up only Ford's tweets:

cr> select * from tweets where id = '2';
+------------+----+-----------+--------+------------------------------------------+---------+
| created_at | id | retweeted | source | text                                     | user_id |
+------------+----+-----------+--------+------------------------------------------+---------+
| 1394182938 | 2  | TRUE      | web    | Time is an illusion. Lunchtime doubly so | Ford    |
+------------+----+-----------+--------+------------------------------------------+---------+
SELECT 1 row in set (... sec)

References

External links

• Official website