Wet lab  Web lab

Open Science in Research

Open Science Symposium 2018

Anisha Keshavan

University of Washington

eScience Institute & Institute for Neuroengineering

The Brain.

Development

Mental Health

Pathology

Aging

We need a lot of data.

Because we are all so unique

Scaling up is easy, right?

We have computers

source: flickr

We have algorithms

A markov chain (wikimedia)

But we're human.

We have our strengths

45 MPH

STOP!

We should combine our strengths

Algorithms

  • fast
  • sometimes stupid

Humans

  • slow
  • sometimes smart

By designing optimal interfaces

Design matters.

Open design matters.

we can overcome our limitations

and optimize for our strengths

  • we are slow: design for collaboration
  • we are smart: design for the human visual system

in science,

The internet.

Pediatric Mental Health

  • Healthy Brain Network Biobank = 10k MRI
  • How do brain tissue volumes change during development?
  • Children move = bad quality image
  • Visual inspection @ every stage of analysis

Lets go to the Web Lab

Mindcontrol

keshavan et al., 2017

Expert Feedback

Mindcontrol is nice, but its slow.

 I want to play on my phone"

Dr. Satra Ghosh, MIT

Braindr

Brain Data Review

braindr.us

Keshavan, Anisha, Jason Yeatman, and Ariel Rokem. "Combining citizen science and deep learning to amplify expertise in neuroimaging." bioRxiv (2018): 363382. (under review)

Instructions

Swipe Left

Swipe Right

Win Prizes

Many Ratings

But often, no agreement

Fail

Pass

Need to "down-weight" some raters

image id rater 0 rater 1
1 0 1
2 1 -
Expert
1
0

each rater is a feature

each image

is a sample

missing data

Rater importance vs # ratings

Corrected Ratings

Predict image quality

> 100,000 ratings
> 400 users
0.99 AUC = near perfect

Thanks eLife Innovation!

Appstract

the app to annotate abstracts

Reproducibility crisis in neuroimaging

  • Results are hard to replicate
  • Neuroimaging datasets are small
  • Meta-analysis → more stable results?
    • we need metadata

Appstract

  • Start small, annotate only sample size
  • Client-side NLP to highlight numbers
  • Tap on the numbers corresponding to sample size

# participants in neuroimaging studies on Autism

How do brain tissue volumes change during development?

back to our question:

A fine line

TL;DR

too long, didn't read in internet-speak

open science is honorable

of course,

open science is strategic

in the age of data-driven discovery

Thanks to my Web Lab!

Ariel Rokem

Jason Yeatman

Satra Ghosh

Dylan Nielson

Sharabesh Ramesh

Dave Kennedy

JB Poline

Neel Somani

Sharabesh Ramesh

Katie Bottenhorn

Arno Klein

Amanda Easson

Adam Richie-Halford

Josh Smith

Greg Kiar

Sook-Lei Liew

Chris Madan

Chris Markiewicz

Elizabeth Dupre

Kesshi Jordan

Rob Fatland

Shima Abadi

Naomi Penfold

Andreea Hrincu

Adam Thomas

 

NVIDIA

AWS

Microsoft Azure

eLife Innovation

UW Institute for Neuroengineering

eScience Institute

 

All braindr raters

Questions?

wet to web, longer

By Anisha Keshavan

wet to web, longer

  • 851