Testing with pytest

Danilo Bargen

Webrepublic AG

2014-01-10

Why pytest?

unittest

import unittest

class TestTheAnswer(unittest.TestCase):
    def testIt(self):
        self.assertEquals(19 + 23, 42)

pytest

def test_the_answer():
    assert 19 + 23 == 42

Features

• Simple assertions
• Dependency injection
• Parametrized tests
  
• Test discovery
• Test runner
• Distributed testing
• Plugin support (e.g. PEP8,
  coverage or pytest-django)
• Conditonal skipping
  
• Easy to get started
• ...
  

Features

/

Examples

Test Discovery

• Good practice: Place tests in a tests/ directory
• Naming
  
• Module naming: test_*.py or *_test.py
• Function naming: test_* or *_test
• Class naming: Test* or *Test
• Customizable!
• Run py.test command
  

Simple Tests

def test_a_feature():
    spam = do_stuff()
    assert spam > 9000, "Spam is not over nine thousand"

Exceptions

import pytest

def raise_exception():
    raise RuntimeError('OMGWTF')

def test_exception_is_raised():
    with pytest.raises(RuntimeError):
        raise_exception()

Explicit Fails

import pytest

def test_eggs_import():
    try:
        import eggs
    except ImportError:
        pytest.fail('Could not import eggs.')

Grouping Tests

class TestClass(object):

    def test_one(self):
        x = 'this'
        assert 'h' in x

    def test_two(self):
        x = 'hello'
        assert hasattr(x, 'check')

Setup / Teardown (1/2)

Module Level

def setup_module(module):

def teardown_module(module):

@classmethod
def setup_class(cls):

@classmethod
def teardown_class(cls):

Setup / Teardown (2/2)

Method Level

def setup_method(self, method):

def teardown_method(self, method):

def setup_function(function):

def teardown_function(function):

Fixtures

Many preprovided fixtures.

Dependency injection.

def test_write_tempfile(tmpdir):
    p = tmpdir.mkdir('sub').join('hello.txt')
    p.write('content')
    assert p.read() == "content"
    assert len(tmpdir.listdir()) == 1

Custom Fixtures

import pytest

@pytest.fixture
def smtp():
    import smtplib
    return smtplib.SMTP('mail.webrepublic.ch')

def test_ehlo(smtp):
    response, msg = smtp.ehlo()
    assert response == 250
    assert 'webrepublic' in msg

Parametrized Tests

A test will be generated for each argument.

import pytest

@pytest.mark.parametrize(['a', 'b'], [
    (19, 23),
    (13, 29),
    (42, 0),
    (65, -23),
])
def test_answer_sum(a, b):
    assert a + b == 42

Drop to Debugger

It's possible to drop into PDB or IPDB

when a test fails.

$ py.test [--pdb|--ipdb]

Case Study:

Campaign Radar

requirements.txt

pytest
pytest-django
pytest-pep8
coverage

pytest.ini

pytest]
DJANGO_SETTINGS_MODULE = config.settings
addopts = --pep8 --tb=short
python_files = test_*.py
pep8ignore =
    *.py E124 E126 E127 E128
    setup.py ALL
    settings.py ALL
    */migrations/* ALL
    */tests/* ALL
pep8maxlinelength = 99

.coveragerc (1/2)

[run]
source =
    front
    worker
omit =
    */migrations/*
    */tests/*

.coveragerc (2/2)

[report]
# Regexes for lines to exclude from consideration
exclude_lines =
    # Don't complain about missing debug-only code:
    def __repr__
    if self\.debug

    # Don't complain if tests don't hit defensive assertion code:
    raise AssertionError
    raise NotImplementedError

    # Don't complain if non-runnable code isn't run:
    if 0:
    if __name__ == .__main__.:

Testing

Install current module:

pip install -e .

py.test

coverage run -m py.test
coverage report

Testing Output

================================================================
test session starts
=================================================================
platform linux2 -- Python 2.7.6 -- pytest-2.4.2
plugins: pep8, cov, cache, django
collected 63 items 

manage.py s
tests/front/test_testing.py .
tests/worker/test_error_detection.py ................................
tests/worker/test_tasks.py .
...

=======================================================
34 passed, 29 skipped in 5.24 seconds
========================================================

Coverage Report

Name                                                  Stmts   Miss  Cover
-------------------------------------------------------------------------
front/__init__                                            0      0   100%
front/ajax                                               64     64     0%
front/crispy_layouts                                      4      4     0%
front/forms                                              28     28     0%
front/mixins                                              6      6     0%
worker/error_detection                                   51      6    88%
worker/management/commands/recurring_checks              39     39     0%
worker/models                                             1      0   100%
worker/redis                                             14     10    29%
worker/tasks                                            246    201    18%
worker/views                                              0      0   100%
...
-------------------------------------------------------------------------
TOTAL                                                   947    770    19%

Questions?

Future Topics:

- Tox

- Jenkins

- Mocking