A brief history of Taverna development

... or at least according to a quick look at
sourceforge/cvs/svn/git history
and occassional dips into Stian's memory

2001-2006: Prototyping

myGrid: 6 academic institutions and 8 industry partners

Challenge: Create a graphical workbench for bioinformaticians to combine data and web services
 

Taverna Workbench releases:

• 2003: 0.1 beta of Taverna Scufl workbench
• 2005: 1.0 release, using Freefluo engine

Open source:

Source+binary releases on SourceForge.net

Local CVS servers → SourceForge.net CVS

2006-2009: Productizing

Version 1.4 (2006):

Apache Maven-based build

executeworkflow command line

Version 1.5:

Raven plugin system

Version 1.7:

Taverna Remote Execution Service

Preview of t2core workflow engine

2007-2014: Taverna 2.x

Version 2.0 (2008)

SourceForge CVS → GoogleCode SVN

t2core workflow engine

Reimplemented workbench

Version 2.1.2 (2009)

Improved support for 3rd-party plugins

Version 2.2 (2010)

Taverna Server + ruby gem

Version 2.5 (2014)

Domain-specific editions 
(astronomy, bioinformatics, biodiversity, digital preservation)

GoogleCode SVN → GitHub

Taverna 2 issues

Plugin system too strict on versioning of dependencies

Every new release of Taverna Workbench made existing plugins incompatible

Maintaining our own classloader implementation - hard!

Building a release difficult -
several weeks including testing

2012-: Taverna 3

Version 3.0 alpha (2013)

OSGi-based plugin system

Engine separated from workbench/commandline/server

Independent APIs for workflow and data format

Taverna Platform - execution API

Version 3.0 (2015)

Apache Taverna (incubating)

Apache Incubator

Graduation path for becoming an Apache project

Taverna Incubator Proposal

Champion: Andy Seaborne (Apache Jena)

Proposal submitted 2014-09-23

Volunteering mentors:

• Chris Mattmann
• Suresh Srinivas
• Suresh Marru
• Marlon Pierce
• Michael Joyce

Questions raised

• Third-party LGPL dependencies? (incompatible with Apache releases)
• Should future developer-contributed plugins be part of "Apache Taverna"? Danger of fragmenting existing community
• How to diversify core development outside University of Manchester?
• Adequate rights to change the license wholesale?
• Build infrastructure required by Apache? Migration of 85 git repositories a concern.

Incubator voting

+13 votes: Accepted!

 Next: house keeping

1. Migrate to new mailing lists, e.g. dev@taverna.incubator.apache.org
2. Apache Contributor License Agreement (CLA) needs to be signed for each Taverna committer
3. Apache account per committer - e.g. stain@apache.org
4. Paperwork handover from Univ. of Manchester
5. Creation of (stub) website at http://taverna.incubator.apache.org/

Initial goals

1. Fully investigate/resolve incompatibly licensed dependencies

2. Stage git repositories for move at https://github.com/taverna-incubator :

   • Restructure git repositories (to ~ 10 repos?)
   • Update headers/metadata to indicate Apache License 2.0
   • Rename Maven groupIds to org.apache.taverna.*
   • Rename packages to org.apache.taverna.*
   • Move staged Github repositories to Apache git
3. Automated builds in Apache's Jenkins
4. Update to latest dependencies
5. Avoid dependencies on myGrid Maven repository
6. Propose updated release and testing procedure
7. Move Website and documentation wiki

Git repositories

1. Reduce our current 85 github repositories to a more managable number
2. Stage merged repositories at https://github.com/taverna-incubator
3. Request Apache INFRA team to import code at git.apache.org
4. Mirrored back to https://github.com/apache
   1. Committers: push to git.apache.org
   2. Others: Github pull requests
   3. Pull requests merged manually by a committer

Proposed git structure

# Git repository               groupId / Java package name
apache/taverna-maven-parent    org.apache.taverna parent
apache/taverna-utils           org.apache.taverna   # Still neded?
apache/taverna-engine-api      org.apache.taverna.engine
    taverna-core-api              org.apache.taverna.core
    taverna-platform-api          org.apache.taverna.platform
apache/taverna-engine-impl     org.apache.taverna.engine.impl
    taverna-core-impl             org.apache.taverna.core.impl
    taverna-platform-impl         org.apache.taverna.platform.impl
    taverna-rest-activity         org.apache.taverna.rest.activity
    taverna-script-activity       org.apache.taverna.script.activity
    taverna-interaction-activity  org.apache.taverna.interaction.activity
apache/taverna-scufl2          org.apache.taverna.scufl2
apache/taverna-databundle      org.apache.taverna.databundle
apache/taverna-workbench-api   org.apache.taverna.workbench.api
                                  org.apache.taverna.workbench.api.activitypalette
                                  org.apache.taverna.workbench.api.menu
apache/taverna-workbench-impl  org.apache.taverna.workbench.impl
                                  org.apache.taverna.workbench.impl.activitypalette
                                  org.apache.taverna.workbench.impl.menu
                                  org.apache.taverna.workbench.impl.diagram
                                  org.apache.taverna.rest.workbench
                                  org.apache.taverna.script.workbench
apache/taverna-commandline-product
apache/taverna-workbench-product
apache/taverna-server          org.apache.taverna.server

apache/taverna-plugin-biomart  org.apache.taverna.biomart
                                  org.apache.taverna.biomart.activity
                                  org.apache.taverna.biomart.workbench
apache/taverna-plugin-soaplab  org.apache.taverna.soaplab

# Optional non-Apache plugins (e.g. due to LGPL/GPL dependencies)
taverna/taverna-plugin-rshell                  com.github.taverna.rshell    
taverna/taverna-plugin-beanshell               com.github.taverna.beanshell
astrotaverna/taverna-plugin-astrotaverna       com.github.astrotaverna.plugin
astrotaverna/astrotaverna-workbench-product    com.github.astrotaverna.workbench
johndoe/taverna-plugin-helloworld              com.example.helloworld.taverna

Suggested release plan

1. Support libraries (e.g. taverna-scufl2 and taverna-databundle)
2. Apache Taverna Command Line 3.0 beta 3 (infrastructure test)
3. Apache Taverna Command Line 3.0
4. Apache Taverna Server 3.0
5. Apache Taverna Workbench 3.0 beta 3
6. Apache Taverna Workbench 3.0 (core edition)
7. (Third-party plugins/editions, e.g. AstroTaverna)

What about Taverna 2.x?

Not going into Apache Incubator

"The final release": Taverna 2.5.1 (really soon now)

Security fixes only until Apache Taverna 3.0.0 is released

Meritocracy

Earn it through doing well

Who is the Taverna developer community?

Plugin developers (e.g. Kevin, Dmitry)

Integrators (e.g. Vadim, Susheel, Yassene)

Core developers (e.g. Manchester, Dmitry)

Scientific Workflow researchers (e.g. Gabor, Pinar)

How do I become a committer?

Submitting patches/pull requests

Contributing to mailing lists

Contributing to testing and documentation

Contributing a plugin

How can we recruit committers?

Be active on mailing lists

Find and invite plugin developers and integrators

Code documentation/examples/tutorials

Community webinars?

"I found a bug, but I am not sure how to fix it"
→ Online mentoring?

... your suggestion?