Swipes For Science
Anisha Keshavan
Research Scientist, Child Mind Institute
Affiliate @ UW eScience Institute
Healthy Brain Network
10,000 MRI scans
![](https://media.giphy.com/media/bSJB8Ju3063qU/giphy.gif)
Why not algorithms?
![](https://www.bleepstatic.com/content/posts/2017/08/07/Road-Signs.jpg)
45 MPH
STOP!
![](https://media.kasperskydaily.com/wp-content/uploads/sites/92/2017/08/01093113/invidible-panda.png)
We should work together
Algorithms
- fast
- sometimes stupid
- needs lots of data to learn
Humans
- slow
- sometimes smart
- learn from sparse data
Braindr
Brain Data Review
Keshavan, Anisha, Jason Yeatman, and Ariel Rokem. "Combining citizen science and deep learning to amplify expertise in neuroimaging." bioRxiv (2018): 363382. (under review)
Instructions
![](https://s3.amazonaws.com/media-p.slid.es/uploads/943523/images/5363445/Fail_img.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/943523/images/5363444/passImg.png)
Swipe Left
Swipe Right
Win Prizes
![](https://s3.amazonaws.com/media-p.slid.es/uploads/943523/images/5363448/virtualPrizes.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/943523/images/5363451/leaderboard.png)
Many Ratings
![](https://s3.amazonaws.com/media-p.slid.es/uploads/943523/images/5363454/ratings.png)
But often, no agreement
![](https://s3.amazonaws.com/media-p.slid.es/uploads/943523/images/5363458/noAgreement.png)
Fail
Pass
Need to "down-weight" some raters
image id | rater 0 | rater 1 |
---|---|---|
1 | 0 | 1 |
2 | 1 | - |
Expert |
---|
1 |
0 |
each rater is a feature
each image
is a sample
missing data
![](https://s3.amazonaws.com/media-p.slid.es/uploads/943523/images/5363469/aggregations.png)
Rater importance vs # ratings
![](https://s3.amazonaws.com/media-p.slid.es/uploads/943523/images/5363471/raterscore1.png)
Corrected Ratings
![](https://s3.amazonaws.com/media-p.slid.es/uploads/943523/images/5363475/correctedRatings.png)
Predict image quality
![](https://s3.amazonaws.com/media-p.slid.es/uploads/943523/images/5363489/rocCurveModel.png)
> 100,000 ratings
> 400 users
0.99 AUC = near perfect
Swipes for Science
![](https://s3.amazonaws.com/media-p.slid.es/uploads/943523/images/5363511/braindrles.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/943523/images/5363509/brainspot.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/943523/images/5363514/whaledr.png)
![](https://media.giphy.com/media/fxR1CmdpkHMVVarKlp/giphy.gif)
Thanks eLife Innovation!
see also:
Swipes For Science
By Anisha Keshavan
Swipes For Science
- 909