This project has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement № 730965
Stian Soiland-Reyes
This work is licensed under a
Creative Commons Attribution 4.0 International License.
Not (just) an experiment!
After-the-fact provenance
Investigation
Study
Assay
(+ Data, Samples, ...)
…a computational protocol?
Performs a computational analysis
Explains a method
Reusable and tweakable
Prospective provenance
#!/usr/bin/env cwl-runner
cwlVersion: v1.0
class: Workflow
inputs:
rulesfile: File
sourcefile: File
sinkfile: File
reverse: boolean
max-steps: int?
outputs:
compounds:
type: File
outputSource: rp2paths/compounds
reactions:
type: File
outputSource: rp2paths/reactions
sinks:
type: File
outputSource: rp2paths/sinks
steps:
rp2:
run: ../tools/RetroPath2/RetroPath2.cwl
in:
input.rulesfile: rulesfile
input.sourcefile: sourcefile
input.sinkfile: sinkfile
input.max-steps: max-steps
out: [solutionfile]
rp2paths:
run: ../tools/rp2paths/rp2paths.cwl
in:
infile: rp2/solutionfile
reverse: reverse
out: [compounds, reactions, sinks]
hints:
dep:Dependencies:
dependencies:
- upstream: https://raw.githubusercontent.com/ibisba/RetroPath2-cwl/0.0.1/tools/RetroPath2.cwl
installTo: ../tools/RetroPath2
- upstream: https://raw.githubusercontent.com/ibisba/rp2paths-cwl/1.0.2-1/tools/rp2paths.cwl
installTo: ../tools/rp2paths
$namespaces:
dep: http://commonwl.org/cwldep#
…a (potential) step of a workflow
Executes some tool (may itself be a workflow)
Explains computational method: "How"
Informs workflow authors - parameters, configurations, versions
Versioning
Algorithm
Packaging
Description
Findability
…a kind of Assay?
What data sources? — inputs
How did it run? — workflow
Where did it run? — machine(s)
Which results were created? — outputs
Why did it run? — study/investigation
What does it mean? — evaluation
Retrospective provenance
What data sources? — inputs
How to analyse? — workflow
Where to run it? — machine(s)
Which results to expect? — outputs
Why run it? — study/investigation
Did it work? — evaluation
Prospective & concurrent provenance!
Cromwell
(Broad Institute)
Airflow
Copyright © 2013 W3C® (MIT, ERCIM, Keio, Beihang), All Rights Reserved. W3C liability, trademark and document use rules apply.
PROV Model Primer
Khan et al,
Sharing interoperable workflow provenance: A review of best practices and their practical application in CWLProv
Submitted to GigaScience
https://doi.org/10.5281/zenodo.1966881
$ cwlprov --help
usage: cwlprov [-h] [--version] [--directory DIRECTORY] [--relative]
[--absolute] [--output OUTPUT] [--verbose] [--quiet] [--hints]
[--no-hints]
{validate,info,who,prov,inputs,outputs,run,runs,rerun,derived,runtimes}
...
cwlprov explores Research Objects containing provenance of Common Workflow
Language executions. <https://w3id.org/cwl/prov/>
commands:
{validate,info,who,prov,inputs,outputs,run,runs,rerun,derived,runtimes}
validate Validate the CWLProv Research Object
info show research object Metadata
who show Who ran the workflow
prov export workflow execution Provenance in PROV format
inputs list workflow/step Input files/values
outputs list workflow/step Output files/values
run show workflow Execution log
runs List all workflow executions in RO
rerun Rerun a workflow or step
derived list what was Derived from a data item, based on
activity usage/generation
runtimes calculate average step execution RuntimesBy Stian Soiland-Reyes
