Wet lab → Web lab
Open Science in Research
Open Science Symposium 2018
Anisha Keshavan
University of Washington
eScience Institute & Institute for Neuroengineering
The Brain.
Development
Mental Health
Pathology
Aging
We need a lot of data.
Because we are all so unique
Scaling up is easy, right?
We have computers
source: flickr
We have algorithms
A markov chain (wikimedia)
But we're human.
We have our strengths
45 MPH
STOP!
We should combine our strengths
Algorithms
- fast
- sometimes stupid
Humans
- slow
- sometimes smart
By designing optimal interfaces
Design matters.
Open design matters.
we can overcome our limitations
and optimize for our strengths
- we are slow: design for collaboration
- we are smart: design for the human visual system
in science,
The internet.
Pediatric Mental Health
- Healthy Brain Network Biobank = 10k MRI
- How do brain tissue volumes change during development?
- Children move = bad quality image
- Visual inspection @ every stage of analysis
Lets go to the Web Lab
Mindcontrol
keshavan et al., 2017
Expert Feedback
Mindcontrol is nice, but its slow.
I want to play on my phone"
Dr. Satra Ghosh, MIT
Braindr
Brain Data Review
Keshavan, Anisha, Jason Yeatman, and Ariel Rokem. "Combining citizen science and deep learning to amplify expertise in neuroimaging." bioRxiv (2018): 363382. (under review)
Instructions
Swipe Left
Swipe Right
Win Prizes
Many Ratings
But often, no agreement
Fail
Pass
Need to "down-weight" some raters
| image id | rater 0 | rater 1 |
|---|---|---|
| 1 | 0 | 1 |
| 2 | 1 | - |
| Expert |
|---|
| 1 |
| 0 |
each rater is a feature
each image
is a sample
missing data
Rater importance vs # ratings
Corrected Ratings
Predict image quality
> 100,000 ratings
> 400 users
0.99 AUC = near perfect
Swipes for Science
Thanks eLife Innovation!
Appstract
the app to annotate abstracts
Reproducibility crisis in neuroimaging
- Results are hard to replicate
- Neuroimaging datasets are small
- Meta-analysis → more stable results?
- we need metadata
Appstract
- Start small, annotate only sample size
- Client-side NLP to highlight numbers
- Tap on the numbers corresponding to sample size
# participants in neuroimaging studies on Autism
How do brain tissue volumes change during development?
back to our question:
A fine line
TL;DR
too long, didn't read in internet-speak
open science is honorable
of course,
open science is strategic
in the age of data-driven discovery
Thanks to my Web Lab!
Ariel Rokem
Jason Yeatman
Satra Ghosh
Dylan Nielson
Sharabesh Ramesh
Dave Kennedy
JB Poline
Neel Somani
Sharabesh Ramesh
Katie Bottenhorn
Arno Klein
Amanda Easson
Adam Richie-Halford
Josh Smith
Greg Kiar
Sook-Lei Liew
Chris Madan
Chris Markiewicz
Elizabeth Dupre
Kesshi Jordan
Rob Fatland
Shima Abadi
Naomi Penfold
Andreea Hrincu
Adam Thomas
NVIDIA
AWS
Microsoft Azure
eLife Innovation
UW Institute for Neuroengineering
eScience Institute
All braindr raters
Questions?
wet to web, longer
By Anisha Keshavan
