CHRIS - A Service for Collecting, Organizing, Processing, and Sharing Medical Image Data in the OpenCloud
Rudolph Pienaar, PhD
Jorge Bernal, PhD
Ata Turk, PhD
Boston Children's Hospital
Boston University
RedHat, Inc.
introduction
new technologies and "paradigms" are changing how we think of "data" and "compute".
both are becoming increasingly "mobile"
ChRIS
Research
Integration
Service
ChRIS
need to address the problem of "simple" distributed/cloud computing from medical compute perspective
overview / goals
as low as possible barrier of entry to developer
not be concerned about infrastructure details
write/develop as if running on a local (*nix) environment
be able to deploy the entire "cloud" system locally for testing/developing/deployment
overview / goals
overview / goals
how do I run this on the cloud?
how do I run this on "someone else's computer"?
overview / goals
overview / goals
containerization about the app itself
pervasive use of docker and docker hub
system handles I/O using simple FS abstractions
RESTful services -- multiple front ends to a single backend
overview / goals
overview / goals
containerization
CHRIS...
CHRIS is a bio-medical data workflow manage that allows easy and intuitive collection, analysis, and sharing of data between parties.
- allows for advanced view rendering
- allows for sharing of images
- allows for immersive real time collaboration
- allows for post-processing of images
many data sources... many compute sources...
via here
connect data here
to here...
and compute!
system topology
Main web server local data repository
remote compute nodes
data source (e.g. in hospital PACS)
distributed design...
web server
coordinator
data handler
compute handler
distributed design...
Red Hat collaboration
io
local web server
data descriptor
data handler
io
pfurl --verb POST --raw --http 172.17.0.2:5055/api/v1/cmd --msg \
'{ "action": "pushPath",
"meta": {
"remote": {
"key": "someKey"
},
"local": {
"path": "/home/data"
},
"transport": {
"mechanism": "compress",
"compress": {
"encoding": "base64",
"archive": "zip",
"unpack": true,
"cleanup": true
}
}
}
}' --quiet --jsonpprintindent 4
compute
local web server
exec descriptor
exec handler
compute
pfurl --verb POST --raw --http 172.17.0.2:5010/api/v1/cmd \
--jsonwrapper 'payload' --msg \
'{ "action": "run",
"meta": {
"cmd": "cal 7 1970",
"auid": "rudolphpienaar",
"jid": "cal-job-1234",
"threaded": true
}
}' --quiet --jsonpprintindent 4
compute... using a container
pfurl --verb POST --raw --http 10.17.24.163:5010/api/v1/cmd
--jsonwrapper 'payload' --msg '
{ "action": "run",
"meta": {
"cmd": "$execshell $selfpath/$selfexec --prefix test- --sleepLength 0 /share/incoming /share/outgoing",
"auid": "rudolphpienaar",
"jid": "simpledsapp-1",
"threaded": true,
"container": {
"target": {
"image": "fnndsc/pl-simpledsapp",
"cmdParse": true
},
"manager": {
"image": "fnndsc/swarm",
"app": "swarm.py",
"env": {
"shareDir": "/home/tmp/share",
"serviceType": "docker",
"serviceName": "testService"
}
}
}
}
}
'
coordination
timing can be complex...
PUSH data
separate coordinator service
WAIT!
EXEC on data
WAIT!
PULL data
WAIT!
coordination
coordination
"meta-compute": {
"comment": "The $ strings in the <cmd> are substituted with values found my running the plugin w/o any args. NB: the incoming and outgoing dirs need to be properly handled! In this example they are mapped from the <key> location in <pfioh> appropriately.",
"cmd": "$execshell $selfpath/$selfexec --prefix test- --sleepLength 0 /share/incoming /share/outgoing",
"auid": "rudolphpienaar",
"jid": "simpledsapp-1",
"threaded": true,
"container": {
"target": {
"comment": "The plugin to run. The <cmdParse>, if true, means substitute $ in the <cmd> string.",
"image": "fnndsc/pl-simpledsapp",
"cmdParse": true
},
"manager": {
"comment": "The manager container to run, with relevant <env> settings.",
"image": "fnndsc/swarm",
"app": "swarm.py",
"env": {
"meta-store": "key",
"serviceType": "docker",
"shareDir": "%shareDir",
"serviceName": "testService"
}
}
},
"comment": "The <service> name is hardcoded in the pfcon source and denotes the IP and port of the <pfioh> service.",
"service": "pangea"
}
}
UI concepts
UI concepts
UI concepts
app to CHRIS...
BU partnership
Boston University Computer Science
Mentoring students on Cloud computing
Academic partner on the MOC
RedHat partnership
RedHat is active partner developing CHRIS infrastructure to the MOC
CHRIS as platform to help foster community and opensource development in the cloud using this paradigm
RedHat interested in helping use CHRIS to be a platform for "hack-a-thons" based on openswift, kubernetes, etc.
MOC
references... CHIPS/ChRIS
-
https://github.com/FNNDSC/ChRIS_ultron_backEnd
-
https://github.com/FNNDSC/ChRIS_ultron_frontEnd
references... services
-
https://github.com/FNNDSC/viewerjs
-
https://github.com/FNNDSC/gcjs
-
https://github.com/FNNDSC/fmjs
-
https://github.com/FNNDSC/toolbarjs
-
https://github.com/FNNDSC/rboxjs
-
https://github.com/FNNDSC/rendererjs
-
https://github.com/FNNDSC/thbarjs
-
https://github.com/FNNDSC/pman
-
https://github.com/FNNDSC/pfioh
-
https://github.com/FNNDSC/pfcon
references... viewers
-
https://goxtk.com
-
https://github.com/FNNDSC/ami
team
- Jorge Bernal
- Nicolas Rannou
- Rudolph Pienaar
- Yangming Ou
- Daniel Haehn
- Daniel Ginsburg
- Ellen Grant
BCH Team
BU Team
- Orran Krieger
- Ata Turk
- Aditya Awalker
RedHat
- Dan McPherson
- and others!
fin!
Thank you!
NAMIC_2018
By Rudolph Pienaar
NAMIC_2018
NAMIC project week, MIT, Jan 2018
- 1,395