Managing Academic Software Development

Dr Sam Mangham

Who Am I

  • Senior RSE @ University of Southampton
  • Trustee @ Society of Research Software Engineering
  • RSE @ Software Sustainability Institute
  • Generalist, interdisciplinary RSE, training, community

Background

  • PhD in Astrophysics
    • HPC monte carlo radiation transfer code for supermassive black holes
  • Neutronics @ Culham Centre for Fusion Energy
    • HPC monte carlo radiation transfer code for fusion
  • Both large legacy HPC codes!

Why?

Enterprise

  • Often large teams
  • Formal training
  • Formal project management frameworks & staff
  • Software is the product

Academic

  • Small/single teams
  • Large numbers of loose collaborators
  • Limited training
  • Ad-hoc management (by other researchers) or self-management
  • Papers are the product

Research Institutes

  • Somewhere in-between
  • Vary with scale, focus, discipline

Outline

  • Development
  • Usage
  • Publication

Managing Development

Project Boards

  • Break a project into components
  • Subdivide as you go!
  • Track progress publicly

"Programmers tend to start coding right away.

Sometimes this works." - Eric Larsen, 2018

Project Boards

  • Document process on tasks
    • GitHub/GitLab etc. let you turn issues into lab books
  • BUS FACTOR
    • Collaboration
    • Future You is a collaborator
    • Knowledge decays quickly

Prioritisation

  • Time estimates
  • MoSCoW
  • Consider and revise!

Must
(60%)

Should (20%)

Could (20%)

Won't
 

Prioritisation

  • Won'ts aren't forever
  • Typical won'ts
    • Future research avenues
    • Features you don't need right now
    • Bugs that don't stop work
  • Acknowledge them publicly
    • Help others plan around you
  • Leave time for testing & documentation!

Version Control

  • Protection against disaster
  • Test and verify changes are intended
  • Avoid having to rerun entire papers' worth of analysis to avoid version mismatches

Branching Workflows

  • New branches for new features
    • Link branches to tasks
    • Easy to parallelise work
    • Easy to switch to working on another feature
  • Regularly merge branches back to development!
    • Otherwise each developer ends up with a divergent version
  • Review pull requests

Write Sustainable Code

  • Proactively avoid technical debt
  • Share and collaborate more easily
    • No code worth writing is disposable!
  • Write for collaborators and community
  • Can't reproduce results if the code isn't sustainable

Write Readable Code

Write Readable Code

  • Descriptive variable names
    • Minimise potential for collision!
    • Not 'c', 'e', 'hb'
  • Code completion & IDEs
  • Modular code
    • You will have to refactor!
    • You can't predict your code's future

Document Your Code

  • Bus factor again
  • Optionally: Document then design
    • Test-driven development
  • Automated tools
  • Automatic hosting
  • Call graph generation
  • docs-like-code

Test-Driven Development

  • Continuous Integration
  • Many more detailed talks on this!

Questions?

Managing Usage

Public Documentation

  • Easy onboarding
  • Quick reference for yourself
  • Online documentation platforms
    • ReadTheDocs again
    • GitHub Pages
    • GitHub wikis

Public Issues

  • Facilitate problem solving
    • Searchable if possible!
  • Own up to the code's limitations
    • Benefits far outweigh embarassment!
  • Issues are a dialogue with your users

Questions?

Managing Release

Release Your Software

  • Majority of research relies on software
  • Much is paperware
  • Public release is required for reproducibility!

Structured Releases

  • GitHub Releases
  • Citation.CFF
  • Zenodo
    • Provides citeable DOIs
  • Include all info
    • Library versions
    • Compiler versions
    • Compiler flags

Software Licenses

  • Previous HPC-BP talk
  • No License
    • Automatically copyrighted
    • No rights for others to do anything
  • Open-Source
    • Copyleft (e.g. GPL3)
    • Permissive (e.g. MIT)
  • Proprietary License
    • Lawyers are expensive
  • choosealicense

Advertisements

Any questions?

2022-11-09-HPC-BP Managing Academic Software Development

By Sam Mangham

2022-11-09-HPC-BP Managing Academic Software Development

Presentation for HPC Best Practises online seminar series on how to manage development of academic software.

  • 114