The changing landscape of Data visualization tools in Large Organizations

Data Transfer

Data Cleaning 

Analytics

Analytics and other tasks sometimes get done by same software as data visualization

Everything done via code

Data Visualization

Analytics

Data Visualization

Data Transfer

Separate GUIs

code library

Tasks done by separate software & code

Hard to talk about just data visualization tools

Data Transer

Data Cleaning 

Analytics

Data Visualization

R inside GUI for these

GUI used for these

Single Software

GUI used for these

Data Cleaning 

Changing Tool landscape

What drove changes?

1976-2016

Past Landscape

Small Data Analytics in Large Organizations Dominated By 3 types of tools

"th

Originally, Not Much Between The Islands

Excel

Code

Industry

Specific

Desktop

GUIs

WHERE & HOW DATA VISUALIZATION HAPPENS

mainframe

pre-installed application on personal computer

Business Intelligence, expensive applications, if not excel

some data-viz specific code libraries but not pure JS, start of web-based 

pure JavaScript libraries, web default

Spreadsheets & Charts

1976-1993  Timeline

 

  • Spreadsheet analytics as way to sell computers
    • Pattern of tool development in small companies, then bought by larger companies to be provided to user for free with OS
  • Charting secondary to calculation

Early internet doldrums

 1994-2006   Timeline

  • Data Limit Implications
    • Data often stored on local machines or on-site servers
    • Data visualization too big for puny internet
    • Software not pre-installed must be physically delivered
      • require distribution, marketing, big scale or $ mark-up
  • Shared via print-outs and presentations
  • Early visualization libraries but not pure-JavaScript

download speeds were an early constraint

Many of the data visualizations that load today in <1 second today would take 10s of seconds to download (and then additional time for your browser to display) years ago

(you can use this calculator to figure out how long your data visualizations would take to download in the past) 

Nielsen's "Law" of Bandwidth

Edholm's "Law" of Bandwidth

both of these use top-of-the-line speeds at the time and show more or less the same thing

The last D3.js visualiation I made :  2 minutes in 1998, 5 seconds in 2003, <1 seconds in 2005

Evolution of the web

a google chrome experiment from 2012

Browser ability is an influence on web-based data visualization tools

Data on WEb is Easier

 2006-2016   Timeline

  • Browsers get more powerful
  • Important web standards
  • Internet speeds increase
  • Many JavaScript libraries for data visualization appear
  • Open-source increases rate of development
    • Pattern of finding ways to maximize use of web standards after multile iterations
    • More things with more perspectives
  • Universities continue to house pro-longed early development
  • More data = more need for data visualization tools
2000s
2000s

Raise your hand if

 

You have a university degree in computer science or similar field

Audience Question:

1996

2016

1. Everyone from 1996

 

2. A lot of people know a bit for work, often within another piece of software.

 

3. A bunch of students who were taught in a college class but aren't C.S. students.

 

4. code bootcamp students

 

5. Internet taught

Who writes code?

These groups are growing fast!

Stack Overflow 2016 Developer Survey found =

1. People with C.S. degrees

2. Hackers

3. People making things in their garages

More people are doing more advanced data visualization, because more people know how to code

majority don't have a traditional C.S. degree

What are recent trends

that are changing data visualization tools we use?

2016

 

 

~Arm waving~

 

Present Landscape

More Options

Excel

Code

Industry Specific Desktop GUIs

Salesforce

Tableau

D3.js

chart.js

Venga

QlikView

Domo

cloud-based

platforms as a service 

oil and gas data & analysis as online service

Spotfire

hundreds of BI options

Cost pressure?

More libraries

More GUIs have code as option

More Industry specific GUIs are being built with plug-in capability

bokeh

templates & add-ons purchased piecmeal

analytics software have r and python as default instead of vba

Altair

Microsoft BI

A MORE CROWDED LANDSCAPE WITH MORE HYBRIDS

 FASTER TO BUILD THINGS & MORE WAYS TO VISUALIZE DATA

open-source making its way into industry-specific software more

WEbGL > SVG

More Hybrids

assumptions of better data flow  across systems

 Writing code is faster

Galleries Save Time

User or 3rd party generated examples extensions, templates, and plug-ins

Instead of standing on the shoulders of long ago giants, you stand on the shoulders of anyone doing similar work, somewhere, right now.

 Speed of new things and diversity of things goes way up. Both open-source and $ license-based

Tableau

Petrel

D3.js

Spotfire

Tableau

Spotfire

D3.js

Ruths.ai template

Back Up Slides

near-future (0-5yr) trends

that are changing data visualization tools we use?

~Even more Arm waving~

 

Recent & near Future Trends

  • Component over monolithic architecture and APIs everywhere leading to easier data flow between software
  • Data, analysis, visualization, etc. as a service
  •  WebGL  is becoming more common (> SVG?)
  • More 3D in maps (mapbox, ArcGIS, googlemaps etc.)

General Software Trends

trend to create on pixel instead of line basis

Future trends ?

  • VR: desktop and web-based data visualizations (more dimensions!)
  • more latency, due to more VR and 3D
  • AI in data prep & chart style selection
    • the return of clippy? but less annoying?
  • Continued focus on minimizing data prep through data architecture
  • Even more blending of BI & Data Science & IT?
    • through better BI (Tableau that does everything)
    • Or easier flow between different components?
  • More people do data visualization as a "part" of their job
  • More definition for what a 100% data visualization person does?
    • more "data visualization" jobs on linkedIN right now than a year ago
  • Less grunt work (due to better data engineering, better data prep tools)
  • APIs that talk to APIs that talk to APIs (tools, IoT, storage, code, GUIs, etc.)

Audience Question

What Are you excited about that is almost here?

 Changes Currently Pushing new tool adoption 

IT Architecture is changing

more data and increasingly complex data require different tools

data interpretations increasingly need to be shared & not only presented

Internet is faster & cloud is normalized

more competition more open-source, & prices are coming down

Infrastructure

New Tools

& New features

People

more people know how to code

new features might generate better understanding, faster understanding, or more people to be exposed to the information

more real-time / mobile expectations

Data

Task

data visualization being applied in new ways

Data_Visualization_Tools_HistoryOnly

By Justin Gosses

Data_Visualization_Tools_HistoryOnly

  • 1,495