D3.js

D3.js (or just D3 for Data-Driven Documents) is a JavaScript library that uses digital data to drive the creation and control of dynamic and interactive graphical forms which run in web browsers. It is a tool for data visualization in W3C-compliant computing, making use of the widely implemented Scalable Vector Graphics (SVG), JavaScript, HTML5, and Cascading Style Sheets (CSS3) standards. It is the successor to the earlier Protovis framework.^[1] In contrast to many other libraries, D3 allows great control over the final visual result.^[2] Its development was noted in 2011,^[3] as version 2.0.0 was released in August 2011.^[4] As of November 2013, the library is at version 3.3.11.^[5]

Context

First web browsers appeared in early 1990s. They were initially capable of displaying static web pages only. This means that the only way for user to interact with the web was through clicking links and scrolling pages. There were many efforts to overcome such limitations. One of the most significant was integration of JavaScript as the scripting language for web browsers. JavaScript gradually became de facto standard language for creating web pages that involve rich user interactivity. This played a crucial role in decision to use JavaScript as the language of D3.js.

At the same time, researchers, engineers, and practitioners from various branches of engineering and science looked for the tools that will enable web browsers to visually present data within web pages. There were multiple projects that reached that goal, and each of these projects had its successes and failures, and inspired next ones. Most notable examples were Prefuse, Flare, and Protovis toolkits that all can be considered as direct predecessors of D3.js.

Prefuse was visualization toolkit created in 2005 that required usage of Java, and visualizations were rendered within browsers with Java plug-in. Flare was similar toolkit from 2007 that used ActionScript, and required Flash plug-in for rendering.

In 2009, based on experience of developing and utilizing Prefuse and Flare, prof. Jeff Heer, Ph.D. student Mike Bostock, and M.S. student Vadim Ogievetsky of the Stanford University's Stanford Visualization Group created Protovis, a JavaScript library to generate SVG graphics from data. The library received a noticeable acceptance both by data visualization practitioners and academics.^[6]

In 2011, the development of Protovis was stopped to focus on a new project, D3.js. Informed by experiences with Protovis, Bostock, along with Heer and Ogievetsky, developed D3.js to provide a more expressive framework that at the same time focuses on web standards and provides improved performance (Bostock, Heer & Ogievetsky 2011).

Technical principles

Embedded within an HTML webpage, the JavaScript D3.js library uses pre-built JavaScript functions to select elements, create SVG objects, style them, or add transitions, dynamic effects or tooltips to them. These objects can also be widely styled using CSS. Large datasets can be easily bound to SVG objects using simple D3 functions to generate rich text/graphic charts and diagrams. The data can be in various formats, most commonly JSON, comma-separated values (CSV) or geoJSON, but, if required, JavaScript functions can be written to read other data formats.

Selections

The central principle of D3 design is to enable programmer to first use a CSS-style selector to select a given sets of Document Object Model (DOM) nodes, then use operators to manipulate them in a similar manner to jQuery (see Bostock, Ogievetsky & Heer 2011, chap. 3). For example, by using D3, one may select all HTML <p>...</p> elements, and then change their text color, e.g. to lavender:

 d3.selectAll("p")
   .style("color", "lavender");

Selection can be based on tag (as in the example above), class, identifier, attribute, or place in hierarchy. Once elements are selected, one can apply operations on them. This includes getting and setting attributes, display texts, and styles (as in the above example). Also, elements can be added and removed. This process of modifying, creating and removing HTML elements, can be made dependent on data - and this is the basic concept of using D3.js.

Transitions

By declaring a transition, values for attributes and styles can be smoothly interpolated over a certain time. Following code will make all HTML <p>...</p> elements on a page gradually change their text color to pink:

 d3.selectAll("p")
   .transition()
   .style("color", "pink");

Default duration of such transitions is 250ms. However, this value can be changed in the code. Besides duration, important characteristics of transitions are delay and ease.

API Structure

D3.js API contains several hundred functions, and they can be grouped into following logical units:^[7]

• Selections
• Transitions
• Arrays
• Math
• Color
• Scales
• SVG
• Time
• Layouts
• Geography
• Geometry
• Behaviors

Math

• Generation of pseudorandom numbers with normal, log-normal, Bates, and Irwin-Hall distributions.
• Transformations in 2D: translation, rotation, skew, and scaling.

Arrays

D3 array operations are built to complement existing array support in JavaScript (mutator methods: sort, reverse, splice, shift and unshift; accessor methods: concat, join, slice, indexOf and lastIndexOf; iteration methods: filter, every, forEach, map, some, reduce and reduceRight). D3 extends this functionality with:

• Functions for finding minimum, maximum, extent, sum, mean, median, and quantile of an array.
• Functions for ordering, shuffling, permuting, merging, and bisecting arrays.
• Functions for nesting arrays.
• Functions for manipulating associative arrays.
• Support for map and set collections.

Geometry

• Computing convex hull of a set of points.
• Computing Voronoi tesselation of a set of points.
• Support for point quadtree data structure.
• Support for basic operations on polygons.

Color

• Support for RGB, HSL, HCL, and L*a*b* color representation.
• Brightening, darkening, and interpolation of colors.

Development environment

D3.js doesn't have a widely accepted standard for development environment. Developers usually use their editor of choice for editing and organizing source code files, and browser of choice for testing and debugging. A fairly frequent organization of such environment is displayed in the next picture. It shows Sublime Text as editor, and Firefox as browser arranged side-by-side.

Another example of D3.js development environment is presented in the next picture. It features Aptana Studio as both editor and debugger. It doesn't contain browser, since internal preview window of Aptana Studio is used instead.

Many developers use web-based environment. One of them, JSFiddle is shown in the next picture.

Note that all three examples presented above display the same small project built with D3.js.

Usage

Data visualization company Datameer officially uses D3.js as its core technology,^[2] while The New York Times sometimes uses it for rich graphs.^[8] It is used by the iD editor for editing OpenStreetMap.^[9][10] D3.js has been extensively used for GIS map making, managing both GeoJSON and Topojson files as input.

Related Tools and Libraries

General visualization tools and libraries:

• Tableau
• R
• Processing

Chart libraries:

• Highcharts JS
• Flot
• Peity
• Timeline.js

Graph visualization:

• Sigma.js
• Arbor.js

Tools built on top of D3.js:

• Cubism
• Rickshaw
• NVD3

References

Further reading

Background on data visualization and information graphics:

• Kirk, Andy (December 2012), Data Visualization: a successful design process (1st ed.), Birmingham: Packt Publishing, ISBN 978-1-84969-346-2 
• Cairo, Alberto (August 2012), The Functional Art (1st ed.), Berkeley: Peachpit, ISBN 978-0321834737 

Background on D3.js itself:

• Bostock, Michael; Ogievetsky, Vadim; Heer, Jeffrey (October 2011), D3: Data-Driven Documents, IEEE Transactions on Visualization and Computer Graphics, IEEE Press 

Using D3.js - beginner level:

• Dewar, Mike (June 2012), Steele, Julie; Blanchette, Meghan, eds., Getting Started with D3, Creating Data-Driven Documents (1st ed.), Sebastopol, California: O’Reilly Media, ISBN 978-1-4493-2879-5 
• Timms, Simon (September 2013), Social Data Visualization with HTML5 and JavaScript (1st ed.), Birmingham: Packt Publishing, ISBN 978-1-7821-6654-2 

Using D3.js - intermediate level:

• Murray, Scott (March 2013), Interactive Data Visualization for the Web, An Introduction to Designing with D3 (1st ed.), Sebastopol, California: O’Reilly Media, ISBN 978-1-4493-3973-9 
• Qi Zhu, Nick (October 2013), Data Visualization with D3.js Cookbook (1st ed.), Birmingham: Packt Publishing, ISBN 978-1-7821-6216-2 

External links

• Official website
• D3.js Gallery