From the wet lab

to the web lab

Anisha Keshavan

University of Washington

eScience Institute & Institute for Neuroengineering


In Medicine

  • Even though we are all unique individuals with unique biology


  • We are treated the same way

Precision Medicine

What if we tailor treatment

to your unique biology?

Meet my mom

  • Oncologist
  • Colon cancer
  • No risk factors
  • Side effects of treatment
  • Not a "mistake"

Treatment = f(Disease)

Treatment = f(Disease,Genotype,Phenotype,Environment)

So, where is precision medicine?

We must bridge the domain divide, first

A single domain

A domain divide

We need a web

We need the web

Goals for this talk

  • Web-based tools for MRI neuroimaging
    • Bridge the domain divide
  • Not a neuroimager?
    • what are your bottlenecks?
    • can the web help?


PhD: Multiple Sclerosis

Disease Progression

Relapse Remitting vs Progressive

Treatment = f(Disease progression)

But we don't know Dp early on

 A clue to Dp ?

Big Data Collection

500 patients, 12 years

= 6,000 MRI scans

Data Analysis Workflow



Image Features






Image Features




Expertise does not scale



And the domain divide makes it hard

Computer Science

Radiology / Neuroanatomy

High dimensionality = lost intuition

Image Features x 1000




Neurology / Neuroscience / Biology

We need data visualization & exploration

Web Interfaces

to cross the divide

Postdoctoral Research

Data Science

Domain Science

Scaling Expertise

Data Science

Domain Science

Data Scientist

  • Needs image annotations
  • In a machine-readable format

Domain Expert

  • Know UNIX to annotate image
  • Save annotations consistently


A dashboard for collaborative brain quality control and annotation

A fruitful collaboration @ UCSF

(under review)

  • Also deployed at:

    • UT Austin Medical School

    • NIMH

    • University of Cambridge, Turing Institute

Now, as postdoc:


Mental Health

  • Public dataset, 10k images

  • Motion → bad quality images

  • Visual inspection before & during analysis

  • Too time consuming in the “wet lab”

Let’s try the web lab


Healthy Brain Network

Scaling expertise in the web lab

  1. Start with small, expertly labelled training set
  2. Train citizen scientists to amplify training set
  3. Train a model on amplified labels

Expert Feedback

Mindcontrol is nice

but I want to play on my phone"

Dr. Satra Ghosh, MIT

Scaling Expertise in the web lab

  1. Start with small, expertly labelled training set
  2. Train citizen scientists to amplify training set
  3. Train a model on amplified labels


Brain Data Review

Keshavan, Anisha, Jason Yeatman, and Ariel Rokem. "Combining citizen science and deep learning to amplify expertise in neuroimaging." bioRxiv (2018): 363382. (under review)


Swipe Left

Swipe Right

Win Prizes


Many Ratings

But often, no agreement

How to aggregate unreliable ratings?

Need to "down-weight" some raters

image id rater 0 rater 1
1 0 1
2 1 -

each rater is a feature

each image

is a sample

with a model that handles missing data

Rater importance vs # ratings

Corrected Ratings

Improvement over average

held out AUC

Scaling expertise in the web lab

  1. Start with small, expertly labelled training set
  2. Train citizen scientists to amplify training set
  3. Train a model on amplified labels

Predict image quality

> 100,000 ratings
> 400 users
0.99 AUC = near perfect

How do I make my own web lab?

maybe you're asking

Web Technology

Domain Science

Thanks eLife Innovation!

Data Visualization & Exploration

Data Science

Domain Science

Linked Visualizations

Rule #1: always create

Brain Morphometry



Klein A, Ghosh SS, Bao FS, Giard J, Häme Y, Stavsky E, Lee N, Rossa B, Reuter M, Neto EC, Keshavan A. Mindboggling morphometry of human brains. PLoS computational biology. 2017 Feb 23;13(2):e1005350.

Keshavan, Anisha, Arno Klein, and Ben Cipollini. "Interactive online brain shape visualization." Research Ideas and Outcomes 3 (2017): e12358.

Information Visualization

Scientific Data Visualization



Rule #2: make it easy to 

Diffusion Imaging

Tract Profile


Yeatman, J. D., Richie-Halford, A., Smith, J. K., Keshavan, A., & Rokem, A. (2018). A browser-based tool for visualization and analysis of diffusion MRI data. Nature communications, 9(1), 940.

  • We need to easily explore data
  • We need to easily share data
    • tabular data

Web Technology

Domain Science

pip install AFQ-Browser
afqbrowser-assemble /path/to/afq.mat
afqbrowser-publish /path/to/target/ reponame

Easy to Share


Big Data has bottlenecks

they delay us from developing precision medicine

they delay us from scientific discovery


We need to bridge the domain divide

The web browser can get us there

If our Big Data science is accessible

People will help

If our Big Data science is intuitive

People will understand

Thanks to my Web Lab!

Ariel Rokem

Jason Yeatman

Satra Ghosh

Dylan Nielson

Sharabesh Ramesh

Dave Kennedy

JB Poline

Neel Somani

Sharabesh Ramesh

Katie Bottenhorn

Arno Klein

Amanda Easson

Adam Richie-Halford

Josh Smith

Greg Kiar

Sook-Lei Liew

Chris Madan

Chris Markiewicz

Elizabeth Dupre

Kesshi Jordan

Rob Fatland

Shima Abadi

Naomi Penfold

Andreea Hrincu

Adam Thomas




Microsoft Azure

eLife Innovation

UW Institute for Neuroengineering

eScience Institute


All braindr raters


From the wet lab to the web lab

By Anisha Keshavan

From the wet lab to the web lab

  • 737