ChRIS - A Service for Collecting, Organizing, Processing, and Sharing Medical Image Data in the Cloud
Rudolph Pienaar, PhD
Technical Director
Fetal-Neonatal Neuroimaging and Developmental Science Center
Staff Scientist
Boston Children's Hospital
introduction
Healthcare informatics is at an inflection point
This talk considers a particular perspective on the problem and proposes a solution
ChRIS
Research
Integration
Service
ChRIS
overview
Some trends confronting computing in healthcare and shaping the landscape
An introduction to our web-based system called ChRIS
Logical topology / design
UI considerations
Containerization
Healthcare informatics landscape
Clouds and untethered computing
Historical legacy
overview
overview
overview
healthcare informatics landscape...
Healthcare informatics is at an inflection point
This talk considers a particular perspective on the problem and proposes a solution
explosion of sensors
complex analytics
healthcare... philosophy shapes informatics
historically, healthcare is differential, not integrative
informatics is poorly understood and not seen necessarily as fundamental
classically information relationship is best described as 1:1
the practice of medicine is less integrative
healthcare "compute"...
provided by vendors with very specialized skills and market
thin, vertical stacks
silos...
Multiple, deep data sets exist in the healthcare dataverse
Largely disconnected and independent of each other
clinical
billing
health records
connections between data
data points, rich as they are, can be insufficient...
the more connections between points...
connections between data
the better!
some trends in informatics...
disparate, disconnected computing
the primacy of the browser as client platform
the rise of the ...
clouds and clouds and clouds...
what are "clouds"?
remember above all others that a "cloud" is really just...
someone else's computer
what do clouds do?
well... they rain...
essentially, we often think of "cloud" computing as less of "computing" and more as storage
we use "clouds" as hard drives in the sky
pull data (i.e. rain) and consume it locally
compute should be "free"...
If we abstract "clouds" to "containing" data...
then we can think of compute as being applied to data and somewhat independent of it
ChRIS...
ChRIS is a bio-medical data workflow manage that allows easy and intuitive collection, analysis, and sharing of data between parties.
- allows for advanced view rendering
- allows for sharing of images
- allows for immersive real time collaboration
- allows for post-processing of images
many data sources... many compute sources...
via here
connect data here
to here...
and compute!
system topology
Main web server local data repository
remote compute nodes
data source (e.g. in hospital PACS)
distributed design...
web server
coordinator
data handler
compute handler
distributed design...
Red Hat collaboration
containerization
containerization
compute and io...
disparate, disconnected computing
data needs to be transferred from server host to remote location
data needs to be processed on remote location
io
local web server
data descriptor
data handler
io
pfurl --verb POST --raw --http 172.17.0.2:5055/api/v1/cmd --msg \
'{ "action": "pushPath",
"meta": {
"remote": {
"key": "someKey"
},
"local": {
"path": "/home/data"
},
"transport": {
"mechanism": "compress",
"compress": {
"encoding": "base64",
"archive": "zip",
"unpack": true,
"cleanup": true
}
}
}
}' --quiet --jsonpprintindent 4
compute
local web server
exec descriptor
exec handler
compute
pfurl --verb POST --raw --http 172.17.0.2:5010/api/v1/cmd \
--jsonwrapper 'payload' --msg \
'{ "action": "run",
"meta": {
"cmd": "cal 7 1970",
"auid": "rudolphpienaar",
"jid": "cal-job-1234",
"threaded": true
}
}' --quiet --jsonpprintindent 4
compute... using a container
pfurl --verb POST --raw --http 10.17.24.163:5010/api/v1/cmd
--jsonwrapper 'payload' --msg '
{ "action": "run",
"meta": {
"cmd": "$execshell $selfpath/$selfexec --prefix test- --sleepLength 0 /share/incoming /share/outgoing",
"auid": "rudolphpienaar",
"jid": "simpledsapp-1",
"threaded": true,
"container": {
"target": {
"image": "fnndsc/pl-simpledsapp",
"cmdParse": true
},
"manager": {
"image": "fnndsc/swarm",
"app": "swarm.py",
"env": {
"shareDir": "/home/tmp/share",
"serviceType": "docker",
"serviceName": "testService"
}
}
}
}
}
'
coordination
timing can be complex...
PUSH data
separate coordinator service
WAIT!
EXEC on data
WAIT!
PULL data
WAIT!
coordination
coordinate
"meta-store": {
"comment": "The <meta-store> defines which paragraph to process to determine the <key> field to pass to <pfioh>. In this case, the <jid> field of the <meta-compute> will be used as the <key> field.",
"meta": "meta-compute",
"key": "jid"
},
"meta-data": {
"remote": {
"comment": "The <key> to use is a lookup in the <meta-store>.",
"key": "%meta-store"
},
"localSource": {
"comment": "The path containing data to process.",
"path": "/neuro/users/rudolphpienaar/Pictures"
},
"localTarget": {
"comment": "Once processed at the remote compute, return results to this path.",
"path": "/neuro/users/rudolphpienaar/tmp/Pictures",
"createDir": true
},
"specialHandling": {
"comment": "This triggers some special processing in <pfioh> -- basically the <incoming> and <outgoing> directories are created and populated in a manner that the plugins expect.",
"op": "plugin"
},
"transport": {
"mechanism": "compress",
"compress": {
"encoding": "none",
"archive": "zip",
"unpack": true,
"cleanup": true
}
},
"comment": "The <service> name is hardcoded in the pfcon source and denotes the IP and port of the <pfioh> service.",
"service": "pangea"
},
coordinate
"meta-compute": {
"comment": "The $ strings in the <cmd> are substituted with values found my running the plugin w/o any args. NB: the incoming and outgoing dirs need to be properly handled! In this example they are mapped from the <key> location in <pfioh> appropriately.",
"cmd": "$execshell $selfpath/$selfexec --prefix test- --sleepLength 0 /share/incoming /share/outgoing",
"auid": "rudolphpienaar",
"jid": "simpledsapp-1",
"threaded": true,
"container": {
"target": {
"comment": "The plugin to run. The <cmdParse>, if true, means substitute $ in the <cmd> string.",
"image": "fnndsc/pl-simpledsapp",
"cmdParse": true
},
"manager": {
"comment": "The manager container to run, with relevant <env> settings.",
"image": "fnndsc/swarm",
"app": "swarm.py",
"env": {
"meta-store": "key",
"serviceType": "docker",
"shareDir": "%shareDir",
"serviceName": "testService"
}
}
},
"comment": "The <service> name is hardcoded in the pfcon source and denotes the IP and port of the <pfioh> service.",
"service": "pangea"
}
}
security
- Medical data is constrained by many external factors
- Logging and control
- Single shutoff point
- Communication pathways
- HIPAA
- Local regulatory
- IRB
secure architecturally
secure communication
big data
- Imaging
- Genomics
- Pathology
- Input DICOM space
- Processed result space
- Structured clinical reporting
big data...
UI concepts
UI concepts
UI concepts
conclusion
- Healthcare compute is at an inflection point
- Lags larger computing -verse
- Many opportunities and challenges exist
references... ChRIS
-
https://github.com/FNNDSC/ChRIS_ultron_backEnd
-
https://github.com/FNNDSC/ChRIS_ultron_frontEnd
references... services
-
https://github.com/FNNDSC/viewerjs
-
https://github.com/FNNDSC/gcjs
-
https://github.com/FNNDSC/fmjs
-
https://github.com/FNNDSC/toolbarjs
-
https://github.com/FNNDSC/rboxjs
-
https://github.com/FNNDSC/rendererjs
-
https://github.com/FNNDSC/thbarjs
-
https://github.com/FNNDSC/pman
-
https://github.com/FNNDSC/pfioh
-
https://github.com/FNNDSC/pfcon
references... viewers
-
https://goxtk.com
-
https://github.com/FNNDSC/ami
team
- Jorge "To Be" Bernal, PhD
- Nicolas "VJ" Rannou, MSc
- Rudolph "Cat Herder" Pienaar, PhD
- Daniel "X Slicer" Haehn, PhD
- Daniel "Game On" Ginsburg, MSc
- Ellen "Doc" Grant, MD
fin!
Thank you!
rudolph.pienaar@childrens.harvard.edu
Web apps..?
-
Simple, single-purpose
- Show static information: text and/or images
- Provide single-focus dynamic data
- Complex, integrative, but still generically single-purpose
- Mobile: instagram, snapchat, whatsapp, etc...
Web apps..?
Web apps...
are
- Accessible from anywhere!
- Desktops, Laptops
- (Smart) Phones
- Tablets
- Are "simple"
- require little end user maintenance
- just go to a link
- but...
- Are typically not as "powerful" as traditional client apps.
Medical Image Viewers
How to view images...
... easily?
- MedXT
- Zebra-med
- Cornerstone (Chris Chafey)
- SliceDrop
- MedView
- mi2b2 Viewer
Medical Image Viewers
Medical Image Viewers
Medical Image Viewers
Medical Image Viewers
Medical Image Viewers
Viewer infrastructure...
ChRIS "reloaded..."
Web applications are increasingly more powerful, easy to use, and accessible
- more familiar interface metaphor
- web 2.0 "social" interaction
- posting/sharing
- moves the burden of maintenance away from end user
- allows for mobile
what is ChRIS?
ChRIS is a bio-medical data workflow manage that allows easy and intuitive collection, analysis, and sharing of data between parties.
what is ChRIS?
ChRIS also...
- allows for advanced view rendering
- allows for sharing of images
- allows for immersive real time collaboration
- allows for post-processing of images
components
- collect
- manage
- analyze
- view
- collaborate
ChRIS "Ultron"... REST
overview
FNNDSC...
... software efforts
- Compute infrastructure
- HPC
- "front end" cluster
- "back end" cluster
- HPC
- Data "movement"
- file system abstractions
- cloud-based access
- Compute "movement"
- dockers
further references...
-
https://github.com/FNNDSC/chrisreloaded
-
https://github.com/FNNDSC/ChRIS_API
-
https://github.com/FNNDSC/viewerjs
-
https://github.com/FNNDSC/mi2b2
-
https://github.com/FNNDSC/vjs
-
https://github.com/slicedrop
-
https://goxtk.com
Development team
- Jorge "To Be" Bernal, PhD
- Nicolas "VJ" Rannou, MSc
- Rudolph "Cat Herder" Pienaar, PhD
- Daniel "X Slicer" Haehn, PhD
- Daniel "Game On" Ginsburg, MSc
- Ellen "Doc" Grant, MD
thank you...
ChRIS -- A Service for Collecting, Organizing, Processing, and Sharing Medical Image Data in the Cloud
By Rudolph Pienaar
ChRIS -- A Service for Collecting, Organizing, Processing, and Sharing Medical Image Data in the Cloud
A general overview of ChRIS
- 1,350