D3: Journey to the source

Hello

I'm a web developer

I started working with D3 about 18 months ago

Dress sizes

Baby names

more recently, force-directed graphs

So, THIS CAUGHT MY EYE RECENTLY...

Sometimes even programming gods

find it hard to get their head around D3...

"I’ve been reflecting on the

world-consuming popularity of

Mike Bostock and Jason Davies’s

D3 library...

everyone wants to do something

with it, but very few people

claim to understand it."

Mike Migurski

http://mike.teczno.com/notes/weeks/1837-typescripting.html

"...I attempted to explain the

“.data()” method to a friend

at JS.Geo and realized that

I also was missing

a critical piece myself."

http://mike.teczno.com/notes/weeks/1837-typescripting.html

WHY THIS TALK, WHY TODAY

1. I've never looked under the hood... maybe I'm actually not the only one?

2. Even now, I sometimes trip up on the same D3 basics, and I have to copy and paste code to get it right... maybe I'm not the only one?

3. Even when everything works, it feels like to do the really cool stuff (as opposed to just draw graphs) you need to understand what's going on under the hood.

THE QUESTIONS I SET OUT TO ANSWER...

1. What are .enter() and .exit() actually doing?

2. What is .data() actually doing?

3. How does the D3 source fit together?

4. How hard is the source to get your head around?

... we'll look at the answers in reverse order!

CAVEAT

If you're a JavaScript guru, please take everything I have to say with a pinch of salt...

JOURNEY TO THE SOURCE

GETTING STARTED

Pre-requesites: npm, git

# Clone from github

$ git clone https://github.com/mbostock/d3.git

# Get D3's required npm packages

$ npm install

npm reads package.json and installs:

jsdom (JavaScript implementation of the DOM)
uglify (for minification)
vows (for testing)

GETTING STARTED

# To run the tests

$ make test

(don't worry too much right now if they fail)

OK, let's see what we got...

IN ASCending order of interest...

component.json - config file for bower

index-browserify.js - used by npm
index.js - used by npm

package.json - config file for npm

d3.js - compiled D3 file
d3.min.js - compiled D3 file

lib/ - external dependencies (e.g. colorbrewer)

test/ - test coverage

globals.js - required for testing

Makefile - compiles source files to make D3.js
src/ - THE SOURCE

MAKING D3

D3 is compiled with a makefile. The best intro to make (if you're not used to it) that I know is by... Mike Bostock.

MAKING D3

# Get rid of existing files

$ make clean

# Make standard build (concatenate & minify)

$ make

# Make just core D3

$ make d3.core.js

# Make just D3.geo

$ make d3.geo.js

HOW THE MAKEFILE WORKS

.INTERMEDIATE d3.js: \

src/start.js \

d3.core.js \

d3.scale.js \

d3.svg.js \

d3.behavior.js \

d3.layout.js \

d3.dsv.js \

d3.geo.js \

d3.geom.js \

d3.time.js \

src/end.js

HOW THE MAKEFILE WORKS

d3.js =

src/start.js +

d3.core.js +

d3.scale.js +

d3.svg.js +

d3.behavior.js +

d3.layout.js +

d3.dsv.js +

d3.geo.js +

d3.geom.js +

d3.time.js +

src/end.js

Scope wrapper, each of the building blocks, end scope wrapper

LOGICAL, HIERARCHICAL STRUCTURE

1. Each of the building blocks corresponds (more or less)

to a directory under /src.

2. Each of the building blocks is made (more or less) by concatenating all the files in that directory.

FOLLOW"s D3's INternal structure...

...AND SO DO THE DOCS

EVERYTHING IS FALLING NICELY INTO PLACE!

WHERE ARE .Enter() and .DATA()?

Everything we're interested in is inside d3.core.js

Let's have a look in the /src/core folder

.enter(): src/core/selection-enter.js

.data(): src/core/selection-data.js

Back into the forest...

Recap

From Thinking with Joins:

var svg = d3.select('body').append('svg');

var data = [2,3,5,10];
var circle = svg.selectAll('circle').data(data);

circle.enter().append('circle').attr('r', 10);
circle.attr('cx', function(d) { return d*10 });
circle.attr('cy', function(d) { return d*10 });

circle.exit().remove();

SOURCE: THINKING WITH JOINS

THINKING WITH JOINS

WHAT .DATA() DOES (APPROX)

d3_selectionPrototype.data = function(value, key) {

  function bind(group, groupData) {
    // Bind data to nodes!
  }

  // enter, update and exit are also d3_selectionPrototypes - 
  // arrays with methods like .select and .data defined. 
  var enter = d3_selection_enter([]),
      update = d3_selection([]),
      exit = d3_selection([]);

  // Bind data to nodes! (which are in 'this')
  while (++i < n) {
    bind(group = this[i], value);
  }

  update.enter = function() { return enter; };
  update.exit = function() { return exit; };

  return update;
}

Full version in the source: line 1676 onwards.

What .DATA() DOES (PSEUDOCODE)

Called on 'd3 selection' objects - arrays of nodes with a bunch of extra functions defined, like .select() and .attr().
Take a dataset, and optionally a key function.
Defines new 'd3 selection' objects called update, enter and exit.
Runs bind for each node... see next slide.
Set enter and exit as properties on update, and return the update object.

HOW DATA IS BOUND

  function bind(group, groupData) {
    var i,
    n = group.length,
    m = groupData.length,
    n0 = Math.min(n, m),
    
    // for the length of the nodes, or the length of the data,
    // whichever is less, bind the data to the existing nodes.
    for (i = -1; ++i < n0;) {
      node = group[i];
      nodeData = groupData[i];
      node.__data__ = nodeData;
      updateNodes[i] = node;
    }
    // Add any leftover data to the enter selection.
    for (; i < m; ++i) {
      enterNodes[i] = d3_selection_dataNode(groupData[i]);
    }
    // And any leftover nodes to the exit selection.
    for (; i < n; ++i) {
      exitNodes[i] = group[i];
    }    
  }

Full version in the source: line 1687 onwards.

HOW BINDING WORKS (PSEUDOCODE)

Bind the data to the nodes:

For the length of the data or the nodes (whichever is shorter), update the __data__ attribute of each node, and add it to update.
For the extra length of the data (if any): create a new node and add it to enter.
For the extra length of the nodes (if any): add the node to exit.

THINKING WITH JOINS (MY VERSION)

The LAST WORDS

"A... useful metaphor I’ve been exploring

in code this week is D3 as nuclear fuel.

This starts to feel a bit more helpful,

because it suggests a clear approach

to D3 for mortal developers: use the power,

but keep it carefully contained."

http://mike.teczno.com/notes/weeks/1837-typescripting.html

.Enter()

stage LEFT