NoSQL

In computing, NoSQL (sometimes expanded to "not only SQL") is a broad class of database management systems that differ from the classic model of the relational database management system (RDBMS) in some significant ways, most important being they do not use SQL as their query language. These data stores may not require fixed table schemas, usually avoid join operations, and typically scale horizontally. Academic researchers typically refer to these databases as structured storage,[1][2][3][4] a term that includes classic relational databases as a subset.

Often, NoSQL databases are categorized according to the way they store the data and it falls under categories such as Key-Value stores, BigTable Implementations, Document-Store databases and Graph Database. The rise of NoSQL database systems alongside with rise of major internet companies, such Google, Amazon, Twitter and Facebook which had significantly different challenges in dealing with data which the traditional RDBMS solutions could not cope with. With the rise of real-time web and need to provide curated information out of large volumes of data which more or less followed similar horizontal structures, these companies realized that performance and real time nature was more important than consistency, which traditional relational databases were spending a high amount of processing time to achieve.

Contents

History

Carlo Strozzi used the term NoSQL in 1998 to name his lightweight, open-source relational database that did not expose the standard SQL interface.[5] (Strozzi suggests that, as the current NoSQL movement "departs from the relational model altogether; it should therefore have been called more appropriately 'NoREL', or something to that effect.")[6]

Eric Evans, a Rackspace employee, reintroduced the term NoSQL in early 2009 when Johan Oskarsson of Last.fm wanted to organize an event to discuss open-source distributed databases.[7] The name attempted to label the emergence of a growing number of non-relational, distributed data stores that often did not attempt to provide ACID (atomicity, consistency, isolation, durability) guarantees, which are the key attributes of classic relational database systems such as Sybase, IBM DB2, MySQL, Microsoft SQL Server, PostgreSQL, Oracle RDBMS, Informix, Oracle Rdb, etc.

In 2011, work began on UnQL (Unstructured Query Language), a specification for a query language for NoSQL databases.[8] It is built to query collections (versus tables) of documents (versus rows) with loosely defined fields (versus columns). UnQL is a superset of SQL within which SQL is a very constrained type of UnQL for which the queries always return the same fields (same number, names and types). However, UnQL does not cover the data definition language (DDL) SQL statements like CREATE TABLE or CREATE INDEX.[9]

Architecture

Typical modern relational databases have shown poor performance on certain data-intensive applications, including indexing a large number of documents, serving pages on high-traffic websites, and delivering streaming media.[10] Typical RDBMS implementations are tuned either for small but frequent read/write transactions or for large batch transactions with rare write accesses. NoSQL, on the other hand, can service heavy read/write workloads.[10] Real-world NoSQL deployments include Digg's 3 TB for green badges (markers that indicate stories voted for by others in a social network)[11] and Facebook's 50 TB for inbox search.[12]

NoSQL architectures often provide weak consistency guarantees, such as eventual consistency, or transactions restricted to single data items. Some systems, however, provide full ACID guarantees in some instances by adding a supplementary middleware layer (e.g., AppScale and CloudTPS).[13][14] Two systems have been developed that provide snapshot isolation for column stores: Google's Percolator system based on BigTable,[15] and a transactional system for HBase developed at the University of Waterloo.[16] These systems, developed independently, use similar concepts to achieve multi-row distributed ACID transactions with snapshot isolation guarantee for the underlying column store, without the extra overhead of data management, middleware system deployment, or maintenance introduced by the middleware layer.

Several NoSQL systems employ a distributed architecture, with the data held in a redundant manner on several servers, often using a distributed hash table. In this way, the system can readily scale out by adding more servers, and failure of a server can be tolerated.[17]

Some NoSQL advocates promote very simple interfaces such as associative arrays or key-value pairs. Other systems, such as native XML databases, promote support of the XQuery standard. Newer systems such as CloudTPS also support join queries.[18]

Taxonomy

NoSQL implementations can be categorized by their manner of implementation:

Document store

Name Language Notes
BaseX Java, XQuery XML database
Clusterpoint C++ geared for Full text search
Apache CouchDB Erlang
eXist XQuery XML database
Jackrabbit Java
Lotus Notes and IBM Lotus Domino LotusScript, Java, IBM X Pages, others MultiValue
MarkLogic Server XQuery XML database
MongoDB C++ BSON (Binary format JSON)
OpenLink Virtuoso C++, C#, Java, SPARQL middleware and database engine hybrid that combines the functionality of a traditional RDBMS, ORDBMS, Graph Database, virtual database, RDF, XML, free-text, web application server and file server
OrientDB Java
SimpleDB Erlang
Terrastore Java

Graph

Name Language Notes
AllegroGraph SPARQL RDF GraphStore
DEX Java, C++ High-performance Graph Database
FlockDB Scala
InfiniteGraph Java High-performance, scalable, distributed Graph Database
Neo4j Java
OpenLink Virtuoso C++, C#, Java, SPARQL middleware and database engine hybrid that combines the functionality of a traditional RDBMS, ORDBMS, Graph Database, virtual database, RDF, XML, free-text, web application server and file server
OrientDB Java
Pregel
Sones GraphDB C# Graph database with query language called GraphQL

Key-value store

Key-value stores allow the application to store its data in a schema-less way. The data could be stored in a datatype of a programming language or an object. Because of this, there is no need for a fixed data model.[19] [20] The following types exist:

Eventually‐consistent key‐value store

Hierarchical key-value store

Hosted services

Key-value cache in RAM

Key-value stores on disk

Ordered key-value stores

Multivalue databases

Object database

Tabular

Tuple store

See also

References

  1. ^ Hamilton, James (3 November 2009). "Perspectives: One Size Does Not Fit All". http://perspectives.mvdirona.com/CommentView,guid,afe46691-a293-4f9a-8900-5688a597726a.aspx. Retrieved 13 November 2009. 
  2. ^ Lakshman, Avinash; Malik, Prashant. Cassandra — A Decentralized Structured Storage System. Cornell University. http://www.cs.cornell.edu/projects/ladis2009/papers/lakshman-ladis2009.pdf. Retrieved 13 November 2009. 
  3. ^ Chang, Fay; Jeffrey Dean, Sanjay Ghemawat, Wilson C. Hsieh, Deborah A. Wallach, Mike Burrows, Tushar Chandra, Andrew Fikes, and Robert E. Gruber. Bigtable: A Distributed Storage System for Structured Data. Google. http://static.googleusercontent.com/external_content/untrusted_dlcp/research.google.com/de//archive/bigtable-osdi06.pdf. Retrieved 22 December 2011. 
  4. ^ Kellerman, Jim. "HBase: structured storage of sparse data for Hadoop". http://blog.rapleaf.com/wp-content/uploads/2007/12/hbase.pdf. Retrieved 22 December 2011. 
  5. ^ Lith, Adam; Jakob Mattson (2010). "Investigating storage solutions for large data: A comparison of well performing and scalable data storage solutions for real time extraction and batch insertion of data" (PDF). Göteborg: Department of Computer Science and Engineering, Chalmers University of Technology. p. 15. http://publications.lib.chalmers.se/records/fulltext/123839.pdf. Retrieved 2011-05-12. "Carlo Strozzi first used the term NoSQL in 1998 as a name for his open source relational database that did not offer a SQL interface[...]" 
  6. ^ "NoSQL Relational Database Management System: Home Page". Strozzi.it. 2007-10-02. http://www.strozzi.it/cgi-bin/CSA/tw7/I/en_US/nosql/Home%20Page. Retrieved 2010-03-29. 
  7. ^ "NOSQL 2009". Blog.sym-link.com. 2009-05-12. http://blog.sym-link.com/2009/05/12/nosql_2009.html. Retrieved 2010-03-29. 
  8. ^ http://unqlspec.org/display/UnQL/Home
  9. ^ Avram, Abel (04). "Interview: Richard Hipp on UnQL, a New Query Language for Document Databases". http://www.infoq.com. http://www.infoq.com/news/2011/08/UnQL. Retrieved 7 September 2011. 
  10. ^ a b Agrawal, Rakesh et al. (2008). "The Claremont report on database research". SIGMOD Record (ACM) 37 (3): 9–19. doi:http://doi.acm.org/10.1145/1462571.1462573. ISSN 0163-5808. http://db.cs.berkeley.edu/claremont/claremontreport08.pdf. 
  11. ^ "Looking to the future with Cassandra | Digg About". About.digg.com. 2009-09-09. http://about.digg.com/blog/looking-future-cassandra. Retrieved 2010-03-29. 
  12. ^ "Cassandra". facebook.com. 2008-08-25. http://www.facebook.com/note.php?note_id=24413138919&id=9445547199&index=9. Retrieved 2011-08-19. 
  13. ^ "Datastore Agnostic Transaction Support for Cloud Infrastructures". IEEE. 2011-07-04. http://cs.ucsb.edu/~ckrintz/papers/ieeecloud11.pdf. 
  14. ^ "CloudTPS: Scalable Transactions for Web Applications in the Cloud". Globule.org. http://www.globule.org/publi/CSTWAC_ircs53.html. Retrieved 2010-03-29. 
  15. ^ "Large-scale Incremental Processing Using Distributed Transactions and Notifications". The 9th USENIX Symposium on Operating Systems Design and Implementation (OSDI 2010), Oct 4–6, 2010, Vancouver, BC, Canada. http://www.usenix.org/events/osdi10/tech/full_papers/Peng.pdf. Retrieved 2010-10-15. 
  16. ^ "Supporting Multi-row Distributed Transactions with Global Snapshot Isolation Using Bare-bones HBase". The 11th ACM/IEEE International Conference on Grid Computing (Grid 2010), Oct 25-29, 2010, Brussels, Belgium. http://www.cs.uwaterloo.ca/~c15zhang/ZhangDeSterckGrid2010.pdf. Retrieved 2010-10-15. 
  17. ^ "Cassandra: Structured Storage System over a P2P Network" (PDF). http://static.last.fm/johan/nosql-20090611/cassandra_nosql.pdf. Retrieved 2010-03-29. 
  18. ^ "Consistent Join Queries in Cloud Data Stores". Globule.org. http://www.globule.org/publi/CJQCDS_ircs68.html. Retrieved 2011-01-31. 
  19. ^ Sandy (2011-01-14). "Key Value stores and the NoSQL movement". http://dba.stackexchange.com/questions/607/what-is-a-key-value-store-database: Stackexchange. http://dba.stackexchange.com/a/619. Retrieved 2012-01-01. "Key value stores allow the application developer to store schema-less data. This data is usually consisting of a string that represents the key, and the actual data that is considered to be the value in the "key - value" relationship. The data itself is usually some kind of primitive of the programming language (a string, an integer, an array) or an object that is being marshalled by the programming languages bindings to the key value store. This replaces the need for fixed data model and makes the requirement for properly formatted." 
  20. ^ Marc Seeger (2009-09-21). "Key-Value Stores: a practical overview". http://blog.marc-seeger.de/2009/09/21/key-value-stores-a-practical-overview/: Marc Seeger. http://blog.marc-seeger.de/assets/papers/Ultra_Large_Sites_SS09-Seeger_Key_Value_Stores.pdf. Retrieved 2012-01-01. "Key-Value stores provide a high performance alternative to relational database systems when it comes to storing and acessing data. This paper provides a short overview over some of the currently available key-value stores and their interface to the Ruby programming language." 
  21. ^ "Riak: An Open Source Scalable Data Store". 28 November 2010. https://wiki.basho.com. Retrieved 28 November 2010. 

External links