Building custom
data visualizations
Shirley Wu
My Fem workshops
Coming from React:
Data Visualization
for React Developers
↓
Intro to D3
↓
Building Custom
Data Visualizations
From elsewhere:
Intro to D3
↓
Building Custom
Data Visualizations
↓
(maybe)
Data Visualization
for React Developers
custom data visualizations can be categorized into two broad categories:
EXPOSITORY
VS.
EXPLORATORY
expository
- static dataset
- explore data for story
- communicate story to audience
exploratory
- dynamic dataset
- interview stakeholders
- build tool for stakeholders to explore the data
examples:
New York Times, The Pudding, The Washington Post, etc.
examples:
scientific visualizations, internal business tools at Netflix, Uber, Airbnb, etc.
expository
![](https://s3.amazonaws.com/media-p.slid.es/uploads/415741/images/5130181/Screenshot_2018-07-23_10.21.27.png)
Data: top 10 blockbusters every year for the last two decades
Goal: come up with a design and implement it together
(Yay participation!)
Data exploration
with Observables
and Vega-Lite
↓
Design with
and Gestalt Laws
↓
Code with
and D3 shapes, layouts
↓
Finish with
data exploration
data exploration:
How to use observables
To load external libraries: | d3 = require('d3') |
To write text: | md ` ... ` |
To set global variable: | globalVar = ... |
To write a block of javascript code: | { ... } |
To create an SVG element: | { const svg = DOM.svg(width, height) ... return svg; } |
data exploration:
data types
- Categorical (movie genres)
- Ordinal (t-shirt sizes)
- Quantitative (ratings/scores)
- Temporal (dates)
- Spatial (cities)
data exploration
![](https://s3.amazonaws.com/media-p.slid.es/uploads/415741/images/2022862/Screen_Shot_2015-12-02_at_10.10.31_AM.png)
data exploration:
some basic Chart types
Bar chart
For categorical comparisons
Domain: categorical
Range: quantitative
![](https://s3.amazonaws.com/media-p.slid.es/uploads/415741/images/5133372/bar.png)
data exploration:
some basic Chart types
Histogram
For categorical distributions
Domain: quantitative bins
Range: frequency of quantitative bin
![](https://s3.amazonaws.com/media-p.slid.es/uploads/415741/images/5133373/histogram.png)
data exploration:
some basic Chart types
Scatterplot
For correlation
2 categories, and the relationship between their quantitative values
![](https://s3.amazonaws.com/media-p.slid.es/uploads/415741/images/5133374/scatter.png)
data exploration:
some basic Chart types
Line chart
For temporal trends
Domain: temporal
Range: quantitative
![](https://s3.amazonaws.com/media-p.slid.es/uploads/415741/images/5133376/line.png)
data exploration:
vega-lite
A grammar of
interactive graphics
- Data source
- Mark (tick, bar, point, line, etc.)
- Encoding (x, y, color, etc.)
- For each encoding: type, field
data exploration
![](https://s3.amazonaws.com/media-p.slid.es/uploads/415741/images/2022862/Screen_Shot_2015-12-02_at_10.10.31_AM.png)
Brainstorm some charts
to answer the questions.
data exploration:
exercise (together)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/415741/images/3639414/giphy.gif)
data exploration:
advice
- Check for missing data, and the validity of the data
- Focus on one question at a time (it's very easy to get sidetracked with a tangent)
- If there IS an interesting tangent, make a note for later
- If the question leads to a dead-end, explore another question or the tangent you found earlier
- Don't be afraid to go out and look for additional data to aid your exploration
- Sometimes, no interesting pattern IS very interesting
Translate from
data to design
- Concentrate on the takeaways to communicate across
- What does that mean in terms of the data? (Individual or aggregate elements? Which attributes?)
- Map the relevant data to visual elements
Design:
marks & channels
Map individual or
aggregate data
elements to marks.
Map data attributes
to channels.
Design:
marks
![](https://s3.amazonaws.com/media-p.slid.es/uploads/415741/images/5133406/marks.png)
Visualization Analysis and Design. Tamara Munzner, with illustrations by Eamonn Maguire. A K Peters Visualization Series, CRC Press, 2014.
Design:
channels
Visualization Analysis and Design. Tamara Munzner, with illustrations by
Eamonn Maguire. A K Peters Visualization Series, CRC Press, 2014.
![](https://s3.amazonaws.com/media-p.slid.es/uploads/415741/images/5133411/channels.png)
Quantitative
- Position
- Size
- Color
Categorical
- Shape
- Texture
- Color
Temporal
- Animation
Design:
marks & channels
Visualization Analysis and Design. Tamara Munzner, with illustrations by
Eamonn Maguire. A K Peters Visualization Series, CRC Press, 2014.
![](https://s3.amazonaws.com/media-p.slid.es/uploads/415741/images/5133420/markschannels.png)
mark
bar
channels
x: category
y: quant
mark
point
channels
x: quant
y: quant
mark
point
channels
x: quant
y: quant
color: category
mark
point
channels
x: quant
y: quant
color: category
size: quant
Design:
channel effectiveness
Visualization Analysis and Design. Tamara Munzner, with illustrations by
Eamonn Maguire. A K Peters Visualization Series, CRC Press, 2014.
![](https://s3.amazonaws.com/media-p.slid.es/uploads/415741/images/5133421/effectiveness.png)
Design:
marks & channels
-
One-to-one mapping of data to channel
-
Multiple mappings of channel to mark (x, y, size, color usually)
-
Do not EVER map multiple data attributes to the same channel
Design:
Gestalt laws of grouping
the human mind naturally
groups individual elements
into patterns
↓
use in data visualization to
save processing time
Design:
Gestalt laws of grouping
Proximity
Put related objects near each other
![](https://s3.amazonaws.com/media-p.slid.es/uploads/415741/images/5141662/proximity.jpg)
(The Functional Art, Ch. 6 by Alberto Cairo)
Design:
Gestalt laws of grouping
Similarity
Indicate like objects (helpful if they can't be placed close to each other)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/415741/images/5141663/similarity.jpg)
(The Functional Art, Ch. 6 by Alberto Cairo)
Design:
Gestalt laws of grouping
Enclosure
Helpful when creating visualizations with multiple sections
![](https://s3.amazonaws.com/media-p.slid.es/uploads/415741/images/5141681/enclosure.jpg)
(The Functional Art, Ch. 6 by Alberto Cairo)
Design:
remix & overlay
"You don’t always need to start from scratch, remix what’s out there already"
- Nadieh Bremer
![](https://s3.amazonaws.com/media-p.slid.es/uploads/415741/images/5141906/Screenshot_2018-07-28_18.25.05.png)
But make sure they're the right visuals to communicate your message!
design:
exercise
![](https://s3.amazonaws.com/media-p.slid.es/uploads/415741/images/3639414/giphy.gif)
Sketch all the things!
- What is your main message(s)?
- What marks will you use? Do they represent individual data points, or aggregate?
- What channels will your marks use? How do they support your message?
![](https://s3.amazonaws.com/media-p.slid.es/uploads/415741/images/5152514/Paper.Talks_and_Workshops.13.png)
code:
design to code
-
Break it down! What do you need to draw the marks? What do you need to calculate the channels?
-
To draw marks: SVG (or canvas)
-
To calculate channels: D3 scales, shapes, and layouts (or straight-up math!)
SVG Elements
rect
x: x-coordinate of top-left
y: y-coordinate of top-left
width
height
circle
cx: x-coordinate of center
cy: y-coordinate of center
r: radius
text
x: x-coordinate
y: y-coordinate
dx: x-coordinate offset
dy: y-coordinate offset
text-anchor: horizontal text alignment
Hi!
path
d: path to follow
Moveto, Lineto, Curveto, Arcto
code:
D3 api
![](https://s3.amazonaws.com/media-p.slid.es/uploads/415741/images/5141033/d3_api_v2.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/415741/images/5141033/d3_api_v2.png)
For translating raw data to what SVG needs to draw
![](https://s3.amazonaws.com/media-p.slid.es/uploads/415741/images/5141033/d3_api_v2.png)
Take output of layout calculations and draw SVG elements
![](https://s3.amazonaws.com/media-p.slid.es/uploads/415741/images/5141033/d3_api_v2.png)
Sometimes all you need are scales to get from data to screen space
Often times, you may need specific layouts.
These output x/y positions
![](https://s3.amazonaws.com/media-p.slid.es/uploads/415741/images/5141033/d3_api_v2.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/415741/images/5141033/d3_api_v2.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/415741/images/5133373/histogram.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/415741/images/5138917/network.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/415741/images/5140907/beeswarm.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/415741/images/5141033/d3_api_v2.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/415741/images/5140915/treemap.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/415741/images/5140913/partition.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/415741/images/5140912/pack.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/415741/images/5138894/cluster.png)
And these generate path commands
![](https://s3.amazonaws.com/media-p.slid.es/uploads/415741/images/5141033/d3_api_v2.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/415741/images/5140909/chord.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/415741/images/5138887/chloropleth.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/415741/images/5141033/d3_api_v2.png)
Great dataviz-specific interactions
![](https://s3.amazonaws.com/media-p.slid.es/uploads/415741/images/5141033/d3_api_v2.png)
Code:
exercise (TOGETHER)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/415741/images/3639414/giphy.gif)
Implement the designs!
![](https://s3.amazonaws.com/media-p.slid.es/uploads/415741/images/5152673/Screenshot_2018-07-31_22.40.33.png)
Each curve represents a movie
-
x: release date
-
y: box office relative to mean
readability
Titles, descriptions, and legends
to explain the visualization
Axes and annotations
to describe the data
READABILITY:
axes & legends
![](https://s3.amazonaws.com/media-p.slid.es/uploads/415741/images/5141044/legend.jpg)
d3-legend by Susie Lu
![](https://s3.amazonaws.com/media-p.slid.es/uploads/415741/images/3639376/Screenshot_2017-03-27_20.23.18.png)
READABILITY:
Annotations
![](https://s3.amazonaws.com/media-p.slid.es/uploads/415741/images/5141045/annotation.png)
d3-annotation by Susie Lu
READABILITY:
exercise (TOGETHER)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/415741/images/3639414/giphy.gif)
Implement the designs!
- Add axes
- Add legends
- Add annotations
![](https://s3.amazonaws.com/media-p.slid.es/uploads/415741/images/5152675/Screenshot_2018-08-02_21.51.18.png)
more svg for
context & aesthetics
- Patterns
- Gradients
- Text on a path
- SVG filters
(blurs, drop-shadows) - Clipping & masking
![](https://s3.amazonaws.com/media-p.slid.es/uploads/415741/images/5152676/textures.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/415741/images/5152700/gradients.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/415741/images/5152703/textPath.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/415741/images/5152704/dropshadow.png)
more svg:
exercise (TOGETHER)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/415741/images/3639414/giphy.gif)
Implement the designs!
- Add textures for holidays
- Add dropshadow to movies
![](https://s3.amazonaws.com/media-p.slid.es/uploads/415741/images/5152705/Screenshot_2018-08-02_22.34.45.png)
interactions
D3:
hover, click, and
other simple interactions
D3 & React (or similar):
- Update after user manipulation of underlying data
- Link multiple visualizations
- Exploratory tools (filtering, aggregating)
final
visualization
![](https://s3.amazonaws.com/media-p.slid.es/uploads/415741/images/5159489/custom_dataviz.gif)
exploratory
Process (at Netflix with Susie & Elijah):
- Initial meeting with stakeholders to figure out most important questions
- Meeting with data engineers
- Mock-up in sketch, sandbox with Semiotics, see shape of data
- Prototype, iterate with stakeholders
exploratory
Advice (from Netflix):
- Different data sources, often SQL queries → plan for queries that take longer (important for interactions)
- Questions for stakeholders:
- What's the business question they're trying to answer?
- How do the metrics they're comparing fall into a decision?
exploratory
Advice, cont.:
- Level of aggregation that's most effective for decision making:
- Get it to ~7 things that are granular and meaningful enough
- If not, top 10 of a default
- Gain trust and credibility within org
- Have to compete with tables of data (detailed but hard to read)
- People will get to a state you never designed for, so think through edge cases
resources
Books:
The Functional Art by Alberto Cairo
Visual Analysis and Design by Tamara Munzner
Online:
The little of visualization design
Information is Beautiful Awards
Thank you's
Lisa Rost
Nadieh Bremer
Susie Lu
Elijah Meeks
Beta-testers
Kristin Henry
Micah Stubbs
Radames Ajna
Santhosh Soundararajan
Nathan Harris
Dylan Wootton
![](https://s3.amazonaws.com/media-p.slid.es/uploads/415741/images/5152725/custom-group.jpg)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/415741/images/3654588/giphy__1_.gif)
Front-end Masters: Building Custom Data Visualizations
By Shirley Wu
Front-end Masters: Building Custom Data Visualizations
- 8,404