Sustaining research software at
the Apache Software Foundation

Apache Taverna (incubating)

BOSC, Dublin 2015-07-11

Stian Soiland-Reyes
Univ of Manchester
stain@apache.org
@soilandreyes
http://orcid.org/0000-0001-9842-9718

What is Apache Taverna?

 

Scientific Workflow system

 

Taverna Workbench (design/run on Desktop)

Taverna Command Line (run/inspect wf)

Taverna Server (REST/WSDL for running)

Taverna Player (web UI for running)

... and more!

http://taverna.incubator.apache.org/

History of Taverna

  • 2001-2006: Prototyping  (0.1 ➔ 1.3)
  • 2006-2009: Productizing Taverna 1.x (1.4 ➔1.6)
  • 2007-2014: Taverna 2.x (2.0 ➔ 2.5)
  • 2012-: Taverna 3.x
  • 2014-: Apache Taverna (incubating)

Evolution of
Research Software (1)

  • Ideas, discussions, current research
    ➔ prototypes, proof of concept
  • Collaborations, Funding calls
    ➔ proposal for $proj1

Evolution of
Research Software (2)

  • Funded for $proj1
    • Develop v1.x
    • Releasing as open source
    • Slowly building user-base
  • Project-specific requirements
    ➔ custom code
  • Danger: Code for Review symptom

Evolution of
Research Software (3)

 

  • Funded for $proj2
    • Slightly different direction
      ➔ generalize code-base
    • Develop v2.x - open source
  • User-base from $proj1 now growing
    • ..but want v1.x support
    • hampered by earlier customizations

Evolution of
Research Software (4)

 

  • Third-party developers showing up
    • Code documentation
    • Plugin and integration support
    • When to release?
  • Different ambitions for the software
    • Conflicting directions

Evolution of
Research Software (5)

 

  • Further funding - but kind of unrelated
    • .. $proj1 and $proj2 users still need support
    • limited time available from original devs
  • Third-party developers increased activity
    - but do they have any control of core code?
  • Where are decisions made? Who?
    • Is the process open?
    • How can you tell it is open?

Need for Open Development

  • Open Source since 2001
    • but anything "core" still done by Manchester
  • Manchester effectively leader
    • .. major developments happening elsewhere
  • Questions about sustainability
    • Single institution is more vulnerable
    • "Have to" add Manchester to grant proposals?

Moving towards
Open Development

Office discussions, meetings
➔ issue tracker, mailing lists, wiki

 

Projects ➔ People

 

Building collective ownership

 

Getting the community more involved

Political challenges

Admitting that software has bugs

 

Admitting that we can't do everything

 

Important People

 

Who decides?

... but it's My Baby!

Who could "own" Taverna?

Apache Software Foundation

Non-profit organization, forming a community of open-source software projects.

 

Strong emphasis on openness, collaboration and a consensus-based development process.

 

Examples: Apache HTTP server, Tomcat, Maven,  
Jena, CXF and OpenOffice.

Why Apache?

Fully open development

 

Encourage further developer involvement

 

Remove impression of Univ of Manchester as leader

..reducing us vs them conversations

 

Independent and neutral

Organizational independence

Open and strong governance

 

Centrally managed infrastructure

 

Less political
easier to say "We'll use and contribute to Apache Taverna" in a grant proposal

 

Longer term sustainability - self-managed community

  1. Find a champion within Apache
  2. Create and discuss proposal
  3. Find mentors
  4. Voted over by Apache Incubator
  5. Invited as a podling
  6. Transition infrastructure
  7. Develop and grow community
  8. Release under Apache
  9. Graduate to Top Level Project

Becoming part of Apache Software Foundation

Apache lingo

TLP: Top-Level Project, self-managed Apache project

Incubator: A special TLP for new projects

Podling (incubating): New project within the Incubator

PMC: Project Management Committee

IPMC / PPMC: Incubator/Podling PMC

 

Apache roles

Champion: Helps at proposal stage

Mentor: Guides towards graduation

Committer: Granted write-access to a project

PMC member: Votes on releases and committership

Apache Member: Decides on Apache-wide rules

PMC Chair: Vice President, reports to the Board

Mentors of Apache Taverna

Andy Seaborne

Chris Mattmann

Suresh Marru

Marlon Pierce

Suresh Srinivas

 

Thank you!

Apache License

Open Source license

 

Attributions required

 

Commercially friendly:

Binaries allowed

Proprietary additions allowed

 

Contributions automatically covered

 

http://www.apache.org/licenses/LICENSE-2.0

Changing the license

Taverna was licensed as LGPL 2.1

 

Needed to change to Apache License

 

Univ of Manchester main copyright holder

 

Contributor License Agreements

 

 

The boring bit

package org.apache.taverna.scufl2.api.core;

/*
 * Licensed to the Apache Software Foundation (ASF) under one
 * or more contributor license agreements.  See the NOTICE file
 * distributed with this work for additional information
 * regarding copyright ownership.  The ASF licenses this file
 * to you under the Apache License, Version 2.0 (the
 * "License"); you may not use this file except in compliance
 * with the License.  You may obtain a copy of the License at
 * 
 *   http://www.apache.org/licenses/LICENSE-2.0
 * 
 * Unless required by applicable law or agreed to in writing,
 * software distributed under the License is distributed on an
 * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
 * KIND, either express or implied.  See the License for the
 * specific language governing permissions and limitations
 * under the License.
 */


import java.net.URI;
import java.util.ArrayList;
import java.util.Collection;

NOTICE

Apache Taverna Language
Copyright 2010-2014 University of Manchester, UK
Copyright 2015 The Apache Software Foundation

This product includes software developed at
The Apache Software Foundation (http://www.apache.org/).

---------------------------------------------------------


ODFDOM
Copyright 2008, 2010 Oracle and/or its affiliates. All rights reserved.

---------------------------------------------------------

PAV ontology http://purl.org/pav/
(c) Copyright 2008-2014 Massachusetts General Hospital; Harvard Medical School;
Balboa Systems; University of Manchester

---------------------------------------------------------

W3C PROVenance Interchange Ontology (PROV-O)
http://www.w3.org/ns/prov-o.rdf
http://www.w3.org/TR/2013/REC-prov-o-20130430/

W3C PROV Access and Query Ontology (PROV-AQ)
http://www.w3.org/ns/prov-aq.owl
http://www.w3.org/TR/2013/NOTE-prov-aq-20130430/

Copyright (c) 2011-2013 W3C(R) (MIT, ERCIM, Keio, Beihang), All Rights Reserved.
W3C liability, trademark and document use rules apply.
http://www.w3.org/Consortium/Legal/2015/doc-license

---------------------------------------------------------

XML Encryption Syntax and Processing
W3C Recommendation 10 December 2002
http://www.w3.org/TR/xmlenc-core/xenc-schema.xsd
http://www.w3.org/TR/xmlenc-core/

Copyright (c) 2002 World Wide Web Consortium, (Massachusetts
Institute of Technology, Institut National de Recherche en Informatique et en
Automatique, Keio University). All Rights Reserved.
http://www.w3.org/Consortium/Legal/

---------------------------------------------------------

Schema for XML Signatures
http://www.w3.org/2000/09/xmldsig#
$Revision: 1.1 $ on $Date: 2002/02/08 20:32:26 $ by $Author: reagle $

Copyright 2001 The Internet Society and W3C (Massachusetts Institute
of Technology, Institut National de Recherche en Informatique et en
Automatique, Keio University). All Rights Reserved.
http://www.w3.org/Consortium/Legal/

This document is governed by the W3C Software License [1] as described
in the FAQ [2].

[1] http://www.w3.org/Consortium/Legal/copyright-software-19980720
[2] http://www.w3.org/Consortium/Legal/IPR-FAQ-20000620.html#DTD


---------------------------------------------------------

Friend of a Friend (FOAF) RDF vocabulary
http://xmlns.com/foaf/spec/20140114.rdf

Copyright (c) 2000-2014 Dan Brickley and Libby Miller
This work is licensed under a Creative Commons Attribution License.
http://creativecommons.org/licenses/by/1.0/

---------------------------------------------------------

DCMI metadata terms     
http://purl.org/wf4ever/dcterms_od
Modified to reduce owl:imports

Adapted from Dublin Core in OWL 2
http://purl.org/NET/dc_owl2dl/terms_od
Modified for OWL compliance

Adapted from Dublin Core
http://purl.org/dc/terms/


DCMI Abstract model
http://purl.org/wf4ever/dcam
Modified URI of the owl:Ontology
     
Adapted from Dublin Core in OWL 2
http://purl.org/NET/dc_owl2dl/dcam
Modified for OWL compliance

Adapted from Dublin Core
http://purl.org/dc/dcam/

Copyright (c) 2012-2015 University of Manchester
Copyright (c) 2009-2014 Simon Reinhardt (http://simon-reinhardt.de/)
Copyright (c) 1995-2012 DCMI, the Dublin Core Metadata Initiative (http://dublincore.org/)

Licensed under a Creative Commons Attribution 3.0 Unported License (CC BY 3.0).
http://creativecommons.org/licenses/by/3.0/

---------------------------------------------------------

Open Annotation Data Model
http://www.w3.org/ns/oa

Copyright (c) 2012-2013 the Contributors to the Open Annotation Core Data Model
Specification, published by the Open Annotation Community Group under the W3C
Community Contributor License Agreement (CLA). A human-readable summary is
available.      

http://www.w3.org/community/about/agreements/cla/

---------------------------------------------------------

Open Document Format for Office Applications (OpenDocument) Version 1.2
OASIS Standard, 29 September 2011
Manifest Relax-NG Schema
Source: http://docs.oasis-open.org/office/v1.2/os/
Copyright (c) OASIS Open 2002-2011. All Rights Reserved.


---------------------------------------------------------

Wf4Ever Research Object Model 1.0 
https://w3id.org/ro/

wfdesc ontology
https://w3id.org/ro/wfdesc

wfprov ontology
https://w3id.org/ro/wfprov

roterms ontology
http://purl.org/wf4ever/roterms

wf4ever ontology
https://w3id.org/ro/wf4ever

Copyright (c) 2011-2014
  Stian Soiland-Reyes, University of Manchester
  Sean Bechhofer, University of Manchester
  Khalid Belhajjame, University of Manchester
  Graham Klyne, University of Oxford
  Daniel Garijo, UPM
  Oscar Corcho, UPM
  Esteban Garcia Cuesta, iSOCO
  Raul Palma, PSNC

Licensed under a Creative Commons Attribution 3.0 License.
http://creativecommons.org/licenses/by/3.0/


------------------------------------------------------

xml.xsd
Extensible Markup Language (XML) 1.0 (Fifth Edition)
W3C Recommendation 26 November 2008
http://www.w3.org/2001/xml.xsd
http://www.w3.org/TR/xml/

Copyright (c) 2015 W3C(R) (MIT, ERCIM, Keio, Beihang). This software or 
document includes material copied from or derived from:

W3C liability, trademark and document use rules apply.
http://www.w3.org/Consortium/Legal/2015/doc-license

------------------------------------------------------

roevo.xsd derived from 
Research Object Evolution Ontology (roevo)
http://w3id.org/ro/roevo

Copyright (c) 2011-2014
    Raul Palma, PSNC
    Jun Zhao, University of Oxford
    Khalid Belhajjame, University of Manchester
    Stian Soiland-Reyes, University of Manchester
    Oscar Corcho, UPM

  Creative Commons Attribution 3.0 License 
  http://creativecommons.org/licenses/by/3.0

Legal protection

 

Software must have clean Intellectual Property

➔ Software Grants, CLAs, NOTICE.txt

 

Commercial users can use your software

without fearing a law suit

 

 

Voting on releases

From: Stian Soiland-Reyes <stain@apache.org>
Date: 11 June 2015 at 15:48
Subject: [VOTE] Release Apache Taverna Language 0.15.0-incubating RC2 (shortened)
To: dev@taverna.incubator.apache.org


I am pleased to be calling this vote for the source release of

  Apache Taverna Language 0.15.0-incubating

The release candidates to be voted over are available at:

  https://dist.apache.org/repos/dist/dev/incubator/taverna/source/taverna-language-0.15.0-incubating-RC2/

  SHA-1 checksum:  7032e9d2be834f7c029aae562b62613481bf6939

Build the release candidate in the above order, using:  mvn clean install

The release candidates correspond to the following git commits:

  https://git-wip-us.apache.org/repos/asf?p=incubator-taverna-language.git;a=commit;h=3340e2090e604b40ac0b88675f57c1d12032d060

Release candidates are signed with a GPG key available at:

  https://dist.apache.org/repos/dist/release/incubator/taverna/KEYS

The vote is open for at least 72 hours and passes if a
majority of at least three +1 Apache Taverna PPMC votes are cast.

[ ] +1 Release this package
[ ]  0 I don't feel strongly about it, but don't object
[ ] -1 Do not release this package because...

The Apache Way

Community over Code

 

Open:
If it didn't happen on the mailing list, it didn't happen

Everybody has a say

 

Merit - win by doing it

Personal merit - not organizational

 

 

Community development

Users and Developers

..are they really separate?

 

Ask not what can I do for the user,
but what can the user do for me

 

Give teasers to potential committers

"I think the problem is somewhere in this file"

... without saying "You're on your own, mate"!

 

Recognize any contributions, not just code

Docs, training, examples, help, discussions, ...

How to become an

open source committer?

 

Submitting patches/pull requests

Contributing to mailing lists

Contributing to testing, documentation, website

Contributing a plugin or new feature

Participate in votes and discussions

Act as if you are already a committer


PMC vote (in private) on a proposed new committer

Google Summer of Code

Students paid to work on open source projects

 

Google sponsored

 

2015: 137 organizations, 988 students

 

ASF: 49 students

 

Apache Taverna: 3 students

Where do research software communities form?

Project mailing lists/forum

Issue trackers

Support mail

Private communications

Chat channels

Domain-specific forums

StackOverflow and Biostars.org

Domain-specific conferences and workshops

Transitioning to Apache

Moving all infrastructure:

Mailing lists, Source code repositories, wiki,

issue tracker, web pages

 

GitHub pull request workflow

integrated with mailing list

 

All transitioning discussed in public

..might scare people :-(

Github pull requests

Is moving easy?

Depends..

 

Taverna - lots of infrastructure
Jira w/1000s of old and new issues,
Wordpress,
30 GitHub repositories

months

 

Commons RDF: Single GitHub repository, GitHub pages. 1 week

Conclusions

ASF gives strong template for community building

.. and initial bureaucracy :-(

Worth it in the long run!

 

Licensing is boring, but necessary

Mentors are kind and helpful

Community building takes effort

Community growth is fun

Giving up "control" gives peace and openness

 

Is Apache fit for my project?

It's not your project, but your community's project!

So your existing community should decide

 

Find a champion from an existing Apache project

 

Are you ready for Open Development?

Are you already there?

 

Do you have time to guide the community

towards "the Apache Way"?

 

Apache not a magic bullet

You still need to do the work!

Team Taverna (PPMC)

Andy Seaborne, Chris Mattmann, Suresh Srinivas,
Suresh Marru, Marlon Pierce, Alan R Williams,
Aleksandra Nenadic, Christian Y. Brenninkmeijer,
Dmitriy Repchevsky, Donal K. Fellows, Ian Dunlop,
Julián Garrido, Robert Haines, Shoaib Sufi,
Stian Soiland-Reyes, Stuart Owen