"Follow The

White Rabbit"

The making of data driven stories

• Process
• Where to get the data from
• How to clean your data
• Data analysis
• Data Visualisation
• Tutorial 1
• Tutorial 2
• Tutorial 3
• Tutorial 4

Process

Moritz Stefaner: Process of Data Visualisation

1. Where to get the data from?

• Open data portals
• Organisations
• Data scraping
• personal data (first hand collection)

Scraping

• Kimono
• ScraperWiki
• Python

Investigative Reporting:

correctiv.org

• The process for investigative data journalism David Schraven’s organisation follows includes three major steps.
  
  “First of all we try to find out how to create a visualisation from it. We try to find a story in the big data sets, the key points, and what is important about all this data. 
  
  "Once we found the story, we try to find human beings who can talk about it and who have been affected by the issue we found,” he explained.

2. Cleaning Data: 

• Tutorial page

3. Analysis:

• Tutorial page 

Big Data

• Don't be afraid of big data sets!
  (start querying big data and make it small) 

• Apache Drill

  "There's no need to load the data, create and maintain schemas, or transform the data before it can be processed. Instead, simply include the path to a Hadoop directory, MongoDB collection or S3 bucket in the SQL query."

4. Data Visualisation:

Start wide, end narrow

• Tutorial page
• Exploratory vs Explanatory Data Visualisations
• Ben Shneiderman
• New UI components to present data stories on the web
• Simplicity in vogue

Exploratory vs Explanatory Data Visualisations

Exploratory

• Excel, Python, R / Shiny, Tableau, SAS/SPSS

Explanatory

• Illustrator, Javascript / D3.js, R (ggplot2), Tableau, Jupyter / IPython notebook
  

Ben Shneiderman

Ben Shneiderman’s 
Infovis 1996 mantra: 

"Overview First, Zoom and filter, then detail on demand..."

UI Storytelling on the web

UI components

Shorthand

knightlab.com

Datawrapper

Personalisation

Why/Why "not" to use data from...

IOT/Personal devices

...or have your preferences

• http://www.theguardian.com/society/ng-interactive/2015/sep/02/unaffordable-country-where-can-you-afford-to-buy-a-house

The Stepper

Tutorial

UI: Scroller

• Scoll.js
• ScollMagic.js
• Bloomberg
• NYT

http://www.nytimes.com/interactive/2014/10/28/sports/marathon-videos.html

http://www.tampabay.com/projects/2015/investigations/pinellas-failure-factories/chart-failing-black-students/

Filtering

Stories

http://www.theguardian.com/us-news/ng-interactive/2015/jun/01/the-counted-map-us-police-killings

Zoooooooming

http://www.nytimes.com/interactive/2014/11/04/upshot/senate-maps.html?_r=4

Data Dashboards

• http://www.theguardian.com/society/ng-interactive/2015/sep/02/unaffordable-country-where-can-you-afford-to-buy-a-house
• http://www.nytimes.com/interactive/2012/10/05/business/economy/one-report-diverging-perspectives.html?_r=1
• DataCandy.co.uk

Simulation:

• http://www.nytimes.com/interactive/2012/11/02/us/politics/paths-to-the-white-house.html

Thank you

@BenHeubl