The changing landscape of Data visualization tools in Large Organizations
Data Transfer
Data Cleaning
Analytics
Analytics and other tasks sometimes get done by same software as data visualization
Everything done via code
Data Visualization
Analytics
Data Visualization
Data Transfer
Separate GUIs
code library
Tasks done by separate software & code
Hard to talk about just data visualization tools
Data Transer
Data Cleaning
Analytics
Data Visualization
R inside GUI for these
GUI used for these
Single Software
GUI used for these
Data Cleaning
Changing Tool landscape
What drove changes?
1976-2016
Past Landscape
Small Data Analytics in Large Organizations Dominated By 3 types of tools
"th
Originally, Not Much Between The Islands
Excel
Code
Industry
Specific
Desktop
GUIs
WHERE & HOW DATA VISUALIZATION HAPPENS
mainframe
pre-installed application on personal computer
Business Intelligence, expensive applications, if not excel
some data-viz specific code libraries but not pure JS, start of web-based
pure JavaScript libraries, web default
Spreadsheets & Charts
1976-1993 Timeline
- Spreadsheet analytics as way to sell computers
- Pattern of tool development in small companies, then bought by larger companies to be provided to user for free with OS
- Charting secondary to calculation
Early internet doldrums
1994-2006 Timeline
- Data Limit Implications
- Data often stored on local machines or on-site servers
- Data visualization too big for puny internet
-
Software not pre-installed must be physically delivered
- require distribution, marketing, big scale or $ mark-up
- Shared via print-outs and presentations
- Early visualization libraries but not pure-JavaScript
download speeds were an early constraint
Many of the data visualizations that load today in <1 second today would take 10s of seconds to download (and then additional time for your browser to display) years ago
(you can use this calculator to figure out how long your data visualizations would take to download in the past)
Nielsen's "Law" of Bandwidth
Edholm's "Law" of Bandwidth
both of these use top-of-the-line speeds at the time and show more or less the same thing
The last D3.js visualiation I made : 2 minutes in 1998, 5 seconds in 2003, <1 seconds in 2005
Evolution of the web
a google chrome experiment from 2012
Browser ability is an influence on web-based data visualization tools
Data on WEb is Easier
2006-2016 Timeline
- Browsers get more powerful
- Important web standards
- Internet speeds increase
- Many JavaScript libraries for data visualization appear
-
Open-source increases rate of development
- Pattern of finding ways to maximize use of web standards after multile iterations
- More things with more perspectives
- Universities continue to house pro-longed early development
- More data = more need for data visualization tools
2000s
2000s
Raise your hand if
You have a university degree in computer science or similar field
Audience Question:
1996
2016
1. Everyone from 1996
2. A lot of people know a bit for work, often within another piece of software.
3. A bunch of students who were taught in a college class but aren't C.S. students.
4. code bootcamp students
5. Internet taught
Who writes code?
These groups are growing fast!
Stack Overflow 2016 Developer Survey found =
1. People with C.S. degrees
2. Hackers
3. People making things in their garages
More people are doing more advanced data visualization, because more people know how to code
majority don't have a traditional C.S. degree
What are recent trends
that are changing data visualization tools we use?
2016
~Arm waving~
Present Landscape
More Options
Excel
Code
Industry Specific Desktop GUIs
Salesforce
Tableau
D3.js
chart.js
Venga
QlikView
Domo
cloud-based
platforms as a service
oil and gas data & analysis as online service
Spotfire
hundreds of BI options
Cost pressure?
More libraries
More GUIs have code as option
More Industry specific GUIs are being built with plug-in capability
bokeh
templates & add-ons purchased piecmeal
analytics software have r and python as default instead of vba
Altair
Microsoft BI
A MORE CROWDED LANDSCAPE WITH MORE HYBRIDS
FASTER TO BUILD THINGS & MORE WAYS TO VISUALIZE DATA
open-source making its way into industry-specific software more
WEbGL > SVG
More Hybrids
assumptions of better data flow across systems
Writing code is faster
Galleries Save Time
User or 3rd party generated examples extensions, templates, and plug-ins
Instead of standing on the shoulders of long ago giants, you stand on the shoulders of anyone doing similar work, somewhere, right now.
Speed of new things and diversity of things goes way up. Both open-source and $ license-based
Tableau
Petrel
D3.js
Spotfire
Tableau
Spotfire
D3.js
Ruths.ai template
Back Up Slides
near-future (0-5yr) trends
that are changing data visualization tools we use?
~Even more Arm waving~
Recent & near Future Trends
- Component over monolithic architecture and APIs everywhere leading to easier data flow between software
- Data, analysis, visualization, etc. as a service
- WebGL is becoming more common (> SVG?)
- More 3D in maps (mapbox, ArcGIS, googlemaps etc.)
General Software Trends
trend to create on pixel instead of line basis
Future trends ?
- VR: desktop and web-based data visualizations (more dimensions!)
- more latency, due to more VR and 3D
-
AI in data prep & chart style selection
- the return of clippy? but less annoying?
- Continued focus on minimizing data prep through data architecture
-
Even more blending of BI & Data Science & IT?
- through better BI (Tableau that does everything)
- Or easier flow between different components?
- More people do data visualization as a "part" of their job
-
More definition for what a 100% data visualization person does?
- more "data visualization" jobs on linkedIN right now than a year ago
- Less grunt work (due to better data engineering, better data prep tools)
- APIs that talk to APIs that talk to APIs (tools, IoT, storage, code, GUIs, etc.)
Audience Question
What Are you excited about that is almost here?
Changes Currently Pushing new tool adoption
IT Architecture is changing
more data and increasingly complex data require different tools
data interpretations increasingly need to be shared & not only presented
Internet is faster & cloud is normalized
more competition more open-source, & prices are coming down
Infrastructure
New Tools
& New features
People
more people know how to code
new features might generate better understanding, faster understanding, or more people to be exposed to the information
more real-time / mobile expectations
Data
Task
data visualization being applied in new ways