Software testing in bioinformatics

- a call to action -

Microbiome-VIF 9

Inês Mendes

UMMI - ULisboa

@ines_cim

cimendes

Can person X, with the same data and the same methodology, obtain the same conclusions as person Y? 

Reproducibility

 | The needs

Reproducibility

 | The needs

https://doi.org/10.1016/j.patter.2020.100099

In the field of microbial bioinformatics, good software engineering practices are not yet widely adopted. (...) This paper serves as a resource that could help microbial bioinformaticians get started with software testing if they have not had formal training.

Code Development

 | Approaches

Design

Code

Test

Test

Code

Refactor

Traditional Technique Approach

Test Driven Development Approach

ASSEMBLY_TEST =  "test/data/assembly.fasta"

def test_fasta_iter():
    fasta_iterator = utils.fasta_iter(ASSEMBLY_TEST)
    first_header, first_contig = fasta_iterator.__next__()
    
    assert first_header == 'k141_4'
    assert len(first_contig) == 1061
    assert sorted(set(first_contig)) == ['A', 'C', 'G', 'T']

Basic Test #2

Parse a fasta file

test_args = ['chewBBACA.py', 'AlleleCall', '-i', 'data/prep_data/empty_dir']

def test_invalid_input(test_args, expected):

    # create empty dir for empty dir test
    if 'empty_dir' in test_args[3] and os.path.isdir(test_args[3]) is False:
        os.mkdir(test_args[3])

    with pytest.raises(SystemExit) as e:
        with patch.object(sys, 'argv', test_args):
            chewBBACA.main()

    assert e.type == SystemExit
    assert e.value.code != 0 

Basic Test #2

Clean exit if input directory is empty

Recommendation #1

Establish software needs and testing goals

Recommendation #1

Recommendation #1

Recommendation #2

Input test files: the good, the bad, and the ugly

Include test files with known expected outcomes for a successful run.

 

Include files or other inputs on which the tool is expected to fail.

Recommendation #3

Use an established framework to implement testing

unittest (https://docs.python.org/3/library/unittest.html)

pytest (https://docs.pytest.org/en/stable/)

(https://testthat.r-lib.org/) testthat

Recommendation #4

Testing is good, automated testing is better

Recommendation #4

Recommendation #4

Recommendation #5

 

Ensure portability by testing on several platforms

Recommendation #6

Showcase the tests

Recommendation #7

Encourage others to test your software!

Try it out!

Software testing - Microbiome IVF

By Inês Mendes

Software testing - Microbiome IVF

Slide deck for the XV CAML PhD Students Meeting

  • 311