Technical architect at myGrid, School of Computer Science, University of Manchester. Open Source research software engineer with interests in Linked Data, RESTful web services, provenance, annotations, open science, reproducible research results.
Workflows and Pipelines
This work is licensed under a
Creative Commons Attribution 4.0 International License .
BioExcel Workflow Training, Barcelona, 2016-11-21
Covering pros and cons of these workflow system, practical workflow design and setup, discussion in this group also covers deployment questions on HPC/cloud infrastructure, like the use of Docker for packaging command line codes.
What should we discuss?
Selecting and comparing workflow systems
Given an idea for a particular workflow,
what wf system would be most appropriate?
Integrations and extensions
Platform integration (e.g. PRACE, EGI)
Workflow integration (e.g. use wf from own portal)
Improving execution speed of workflows
Swapping tools or execution infrastructure
Develop Best Practice
Share common knowledge
"How do I combine .."
Considering alternative approaches
What is special for each wf system,
what is common for all of them?
"Which tool should I use for ...?"
Where to talk about workflows?
Face to Face
BioExcel has funds to support IG meetups
What could we do face 2 face?
Bring Your Own Workflow
"I wish I had a better workflow for X"
Collaborative wf design
Build same wf in multiple wf managers?
Interesting? Need common problem space
First try: Today at 14:00
If the tools don't work well with your workflow,
let's fix the tool integration
Bring in BioExcel developers (GROMACS, HADDOCK, CPMD, Taverna, COMPSs)
and external devs (e.g. CWL, Galaxy, BCBio)
Learning: How to wrap a tool for the Galaxy shed or describe it for CWL
Developer focused - what about users?
"Exchange notes" on technology and best practices
Science show & tell - try out your research ideas
on a small audience
"Handheld" try out new tools and techniques
Goal: Write a joint paper (draft)
Focus on science or tech?
Best Practices - e.g. "10 Simple Rules for building biomolecular workflows"
WF/tool Review - Comparisons and experiences
These were just some proposals!
Interest Group shaped by its members
IG decide itself on what is most useful,
when/how to meet, technologies, etc.
What services could BioExcel provide
to support workflow users?
Support, optimization, guidance, consulting, ... ?
Free tier, on-demand tier, subscription tier ?
Next Webinar: Globus Genomics
2016-11-16 16:00 CET
Ravi Madduri, Argonne / University of Chicago
Large scale Galaxy workflows for Next-gen Sequencing
NIH Big Data for Discovery Science
Join the Workflow
2016-11-21 BioExcel Workflows IG
By Stian Soiland-Reyes