CHRIS - A Service for Collecting, Organizing, Processing, and Sharing Medical Image Data in the OpenCloud

Rudolph Pienaar, PhD

Jorge Bernal, PhD

Ata Turk, PhD

Boston Children's Hospital

Boston University

RedHat, Inc.

introduction

new technologies and "paradigms" are changing how we think of "data" and "compute".

both are becoming increasingly "mobile"

ChRIS

Research

Integration

Service

ChRIS

need to address the problem of "simple" distributed/cloud computing from medical compute perspective

overview / goals

as low as possible barrier of entry to developer

not be concerned about infrastructure details

write/develop as if running on a local (*nix) environment

be able to deploy the entire "cloud" system locally for testing/developing/deployment

overview / goals

how do I run this on the cloud?

how do I run this on "someone else's computer"?

overview / goals

containerization about the app itself

pervasive use of docker and docker hub

system handles I/O using simple FS abstractions

RESTful services -- multiple front ends to a single backend

overview / goals

containerization

CHRIS...

CHRIS is a bio-medical data workflow manage that allows easy and intuitive collection, analysis, and sharing of data between parties.

allows for advanced view rendering
allows for sharing of images
allows for immersive real time collaboration
allows for post-processing of images

many data sources... many compute sources...

via here

connect data here

to here...

and compute!

system topology

Main web server local data repository

remote compute nodes

data source (e.g. in hospital PACS)

distributed design...

web server

coordinator

data handler

compute handler

distributed design...

Red Hat collaboration

local web server

data descriptor

data handler

pfurl --verb POST --raw --http 172.17.0.2:5055/api/v1/cmd --msg \
'{  "action": "pushPath",
    "meta": {
        "remote": {
            "key":         "someKey"
        },
        "local": {
            "path":         "/home/data"
        },
        "transport": {
            "mechanism":    "compress",
            "compress": {
                "encoding": "base64",
                "archive":  "zip",
                "unpack":   true,
                "cleanup":  true
            }
        }
    }
}' --quiet --jsonpprintindent 4

compute

local web server

exec descriptor

exec handler

compute

pfurl --verb POST --raw --http 172.17.0.2:5010/api/v1/cmd \
     --jsonwrapper 'payload' --msg \
'{  "action": "run",
        "meta": {
                "cmd":      "cal 7 1970",
                "auid":     "rudolphpienaar",
                "jid":      "cal-job-1234",
                "threaded": true
        }
}' --quiet --jsonpprintindent 4

compute... using a container

pfurl --verb POST --raw --http 10.17.24.163:5010/api/v1/cmd 
--jsonwrapper 'payload' --msg '
        {   "action": "run",
            "meta":  {
                "cmd":      "$execshell $selfpath/$selfexec --prefix test- --sleepLength 0 /share/incoming /share/outgoing",
                "auid":     "rudolphpienaar",
                "jid":      "simpledsapp-1",
                "threaded": true,
                "container":   {
                        "target": {
                            "image":            "fnndsc/pl-simpledsapp",
                            "cmdParse":         true
                        },
                        "manager": {
                            "image":            "fnndsc/swarm",
                            "app":              "swarm.py",
                            "env":  {
                                "shareDir":     "/home/tmp/share",
                                "serviceType":  "docker",
                                "serviceName":  "testService"
                            }
                        }
                }
            }
        }
'

coordination

timing can be complex...

PUSH data

separate coordinator service

WAIT!

EXEC on data

WAIT!

PULL data

WAIT!

coordination

            "meta-compute":  {
                "comment":  "The $ strings in the <cmd> are substituted with values found my running the plugin w/o any args. NB: the incoming and outgoing dirs need to be properly handled! In this example they are mapped from the <key> location in <pfioh> appropriately.",
                "cmd":      "$execshell $selfpath/$selfexec --prefix test- --sleepLength 0 /share/incoming /share/outgoing",
                "auid":     "rudolphpienaar",
                "jid":      "simpledsapp-1",
                "threaded": true,
                "container":   {
                        "target": {
                            "comment":          "The plugin to run. The <cmdParse>, if true, means substitute $ in the <cmd> string.",
                            "image":            "fnndsc/pl-simpledsapp",
                            "cmdParse":         true
                        },
                        "manager": {
                            "comment":          "The manager container to run, with relevant <env> settings.",
                            "image":            "fnndsc/swarm",
                            "app":              "swarm.py",
                            "env":  {
                                "meta-store":   "key",
                                "serviceType":  "docker",
                                "shareDir":     "%shareDir",
                                "serviceName":  "testService"
                            }
                        }
                },
                "comment":              "The <service> name is hardcoded in the pfcon source and denotes the IP and port of the <pfioh> service.",
                "service":              "pangea"
            }
        }

UI concepts

app to CHRIS...

BU partnership

Boston University Computer Science

Mentoring students on Cloud computing

Academic partner on the MOC

RedHat partnership

RedHat is active partner developing CHRIS infrastructure to the MOC

CHRIS as platform to help foster community and opensource development in the cloud using this paradigm

RedHat interested in helping use CHRIS to be a platform for "hack-a-thons" based on openswift, kubernetes, etc.

MOC

references... CHIPS/ChRIS

https://github.com/FNNDSC/ChRIS_ultron_backEnd

https://github.com/FNNDSC/ChRIS_ultron_frontEnd

references... services

```
https://github.com/FNNDSC/viewerjs
```
```
https://github.com/FNNDSC/gcjs
```
```
https://github.com/FNNDSC/fmjs
```
```
https://github.com/FNNDSC/toolbarjs
```
```
https://github.com/FNNDSC/rboxjs
```
```
https://github.com/FNNDSC/rendererjs
```
```
https://github.com/FNNDSC/thbarjs
```
```
https://github.com/FNNDSC/pman
```
```
https://github.com/FNNDSC/pfioh
```
```
https://github.com/FNNDSC/pfcon
```

references... viewers

```
https://goxtk.com
```
```
https://github.com/FNNDSC/ami
```

team

Jorge Bernal
Nicolas Rannou
Rudolph Pienaar
Yangming Ou

Daniel Haehn
Daniel Ginsburg

Ellen Grant

BCH Team

BU Team

Orran Krieger
Ata Turk
Aditya Awalker

RedHat

Dan McPherson
and others!

fin!

Thank you!

NAMIC_2018

By Rudolph Pienaar

NAMIC_2018

NAMIC project week, MIT, Jan 2018

1,409

Rudolph Pienaar PRO

xegere

CHRIS - A Service for Collecting, Organizing, Processing, and Sharing Medical Image Data in the OpenCloud

ChRIS

NAMIC_2018

More from Rudolph Pienaar