Mark Woodbridge, Mayeul d’Avezac, Jeremy Cohen
Research Computing Service
Imperial College London
Text in corner of slide = A git branch
git checkout <branch>
Demonstrate a set of tools and a workflow that can be used to automate some valuable quality checks for Python software projects
You will leave with:
Why is this important?
Software sustainability:
Ultimately:
We are not aiming to:
Our toy project is Conway’s Game of Life
sudo apt-get install -y atom
atom .Alternative: VS Code
1. Launch the VM, open a terminal and switch directory:
git fetch
git reset --hard origin/mastercd woodbridge2. Get the most up-to-date copy of this tutorial:
3. Install and launch Atom (password "workshops")
Strict (automated) code formatting results in:
from scipy import signal
def count_neighbours(board):
"""Return an array of neighbour counts for each element of `board`"""
return signal.convolve2d( board ,
[[1, 1, 1], [1, 0, 1], [1, 1, 1]], mode ='same')1. Paste in the following (poorly formatted) code into a new file named `life.py`
2. Save and observe automatic reformatting
-r requirements.txt
pytest==3.7.41. Create a new file `requirements-dev.txt`:
2. Update installed packages:
pip install -r requirements-dev.txtAlternative: unittest
from life import count_neighbours
def test_count_neighbours():
board = [
[0, 0, 0, 0, 0],
[0, 0, 0, 0, 0],
[0, 1, 1, 1, 0],
[0, 0, 0, 0, 0],
[0, 0, 0, 0, 0],
]
assert count_neighbours(board).tolist() == [
[0, 0, 0, 0, 0],
[1, 2, 3, 2, 1],
[1, 1, 2, 1, 1],
[1, 2, 3, 2, 1],
[0, 0, 0, 0, 0],
]1. Create a new file `test_life.py`:
2. Run `pytest` and ensure your test passes
-r requirements.txt
pytest==3.7.4
pytest-cov==2.5.1[pytest]
addopts = --cov=life --cov-report term-missing3. Create `pytest.ini`:
4. Run `pytest` and observe statistics
1. Update `requirements-dev.txt`:
2. Update installed packages:
pip install -r requirements-dev.txtdef step(board):
"""Return a new board corresponding to one step of the game"""
nbrs_count = count_neighbours(board)
return (nbrs_count == 3) | (board & (nbrs_count == 2))1. Append to `life.py`:
3. Add a test for `step` to `test_life.py` (similar to `test_count_neighbours`) and re-run `pytest`
2. Re-run `pytest` and observe coverage
1. Add a `board` fixture to `test_life.py`:
import pytest
@pytest.fixture
def board():
return [
[0, 0, 0, 0, 0],
[0, 0, 0, 0, 0],
[0, 1, 1, 1, 0],
[0, 0, 0, 0, 0],
[0, 0, 0, 0, 0],
]2. Change the signatures of the relevant functions e.g.
def test_count_neighbours(board):3. Re-run your tests
1. Add a `play` function to `life.py`:
def play(board, iterations):
"""Return a new board corresponding to `iterations` steps of the game"""
for _ in range(iterations):
board = step(board)
return board.tolist()2. Add a`test_play` function to `test_life.py` using a fixture
3. Run your tests and ensure they all pass
Hint: consider what `play(board, 2)` should return
def test_play_wrap():
board = [
[0, 0, 0, 0, 0],
[0, 0, 1, 0, 0],
[0, 0, 0, 1, 0],
[0, 1, 1, 1, 0],
[0, 0, 0, 0, 0],
]
assert play(board, 20) == board1. Add the following test case to `test_life.py`
2. Run `pytest` and observe output
3. Resolve the root cause of the failure for this glider
Hint: Review the SciPy documentation for `convolve2d`
...
flake8==3.5.0[flake8]
exclude = venv/,.atom,.tox
max-line-length = 883. Create a `.flake8` file in the current directory:
4. Run `flake8`
1. Update `requirements-dev.txt`:
2. Update installed packages:
pip install -r requirements-dev.txt1. Modify `test_life.py` so that it tries to invoke `play` with a non-integer number of iterations:
def test_play(board) -> None:
assert play(board, 2.5) == board3. Add a type annotation to `life.py` to protect against this:
-def play(board, iterations):
+def play(board, iterations: int):4. Observe the resultant warning in Atom
2. Run `pytest` and observe output
1. Create a `.gitlab-ci.yml` file:
test:
script:
- apt-get update -y && apt-get install -y tox
- tox2. Create a `tox.ini` file:
[tox]
envlist = py3, flake8
skipsdist = True
[testenv]
deps = -rrequirements-dev.txt
commands = pytest
[testenv:flake8]
deps = flake8
commands = flake81. Create a GitLab account (if necessary)
git remote add gitlab https://gitlab.com/<username>/rse18.git3. Add a `gitlab` remote:
4. Push to GitLab:
git push gitlab5. Visit https://gitlab.com/<username>/rse18
2. Create an "api" scope Access Token (if required)
Ensure that you save this somewhere
Enable badges for your repository:
pytest=3.7.4
...
hypothesis==3.70.00from hypothesis import given
from hypothesis.strategies import integers, lists
@given(lists(lists(integers(0, 1))), integers(max_value=20))
def test_play_fuzz(board, iterations):
play(board, iterations)
4. Run `pytest` and observe output
1. Update `requirements-dev.txt`:
2. Update installed packages:
pip install -r requirements-dev.txt3. Update `test_life.py`:
pytest=3.7.4
...
nbval==0.9.1[pytest]
addopts = --cov=life --cov-report term-missing --nbval4. Create `life.ipynb`
5. Run `pytest` and observe output
1. Update `requirements-dev.txt`:
2. Update installed packages:
pip install -r requirements-dev.txt3. Update `pytest.ini`:
pytest=3.7.4
...
pytest-benchmark==3.1.1[pytest]
addopts = ... --benchmark-autosave --benchmark-compare --benchmark-compare-fail=min:5%3. Update `pytest.ini`:
4. Run `pytest` twice and observe statistics
1. Update `requirements-dev.txt`:
2. Update installed packages:
pip install -r requirements-dev.txtIgnore warnings on first run
import numpy as np
def count_neighbours(board):
"""Return an array of neighbour counts for each element of `board`"""
return sum(
np.roll(np.roll(board, i, 0), j, 1)
for i in (-1, 0, 1)
for j in (-1, 0, 1)
if (i != 0 or j != 0)
)2. Run `pytest` and observe statistics
1. Update `life.py` to use numpy rather than scipy:
Jake VanderPlas: Conway's Game of Life in Python