Computational Workflows
Stian Soiland-Reyes,
The University of Manchester
They ride with what I refer to as the four horsemen of the reproducibility apocalypse:
- Publication bias
- Low statistical power
- P-value hacking
- HARKing (hypothesizing after results are known)
Reproducibility?
Reproducibility?
Automation
– Automate computational aspects
– Repetitive pipelines, sweep campaigns
Scaling—compute cycles
– Make use of computational infrastructure
– Handle large data
Abstraction—people cycles
– Shield complexity and incompatibilities
– Report, re-usue, evolve, share, compare
– Repeat—Tweak—Repeat
– First-class commodities
Provenance—reporting
– Capture, report and utilize log and data lineage
– Auto-documentation
– Tracable evolution, audit, transparency
– Reproducible science
Findable
Accessible
Interoperable
Reusable
(Reproducible)
Why use workflows?
Adapted from Bertram Ludäscher (2015)
https://www.slideshare.net/ludaesch/works-2015provenancemileage
https://doi.org/10.1007/s13222-012-0100-z
cwlVersion: v1.0
class: Workflow
inputs:
inp: File
ex: string
outputs:
classout:
type: File
outputSource: compile/classfile
steps:
untar:
run: tar-param.cwl
in:
tarfile: inp
extractfile: ex
out: [example_out]
compile:
run: arguments.cwl
in:
src: untar/example_out
out: [classfile]
Nature 573, 149-150 (2019)
https://doi.org/10.1038/d41586-019-02619-z
cwlVersion: v1.0
class: Workflow
inputs:
toConvert: File
outputs:
converted:
type: File
outputSource: convertMethylation/converted
combined:
type: File
outputSource: mergeSymmetric/combined
steps:
convertMethylation:
run: interconverter.cwl
in:
toConvert: toConvert
out: [converted]
mergeSymmetric:
run: symmetriccpgs.cwl
in:
toCombine: convertMethylation/converted
out: [combined]
cwlVersion: v1.0
class: CommandLineTool
inputs:
toConvert:
type: File
inputBinding:
prefix: -i
outputs:
converted:
type: File
outputBinding:
glob: "*.meth"
baseCommand: interconverter.sh
arguments: ["-d", $(runtime.outdir)]
hints:
- class: DockerRequirement
dockerPull: "quay.io/neksa/screw-tool"
cwlVersion: v1.0
class: CommandLineTool
inputs:
toCombine:
type: File
inputBinding:
prefix: -i
outputs:
combined:
type: File
outputBinding:
glob: "*.sym"
baseCommand: symmetriccpgs.sh
arguments: ["-d", $(runtime.outdir)]
hints:
- class: DockerRequirement
dockerPull: "quay.io/neksa/screw-tool"
