pytest

what is pytest?

pytest:

is a python test framework

pytest:

is a python test framework

makes it easy to write small tests

pytest:

is a python test framework

makes it easy to write small tests

scales to support complex testing

pytest:

is a python test framework

makes it easy to write small tests

scales to support complex testing

helps you write better code

but whytest?

Tests show that your code works.

Testing improves confidence that your code changes are non-breaking.

Tested code is less error-prone.

Tests serve as living documentation.

Testable code is more maintainable and understandable code.

swapy

Introducing swapy,
the exciting new python library which wraps swapi!

But what is swapi?

... the Star Wars API!

https://swapi.co/

swapi is an excellent, free, and completely open API which supports both JSON and Wookiee data formats

I built a toy API wrapper (swapy) around swapi to provide some actual working code that we can test using pytest

Install pytest (if you don't have it installed already) via your favorite env flavor:

pip install pytest

conda install pytest

To get started:

Clone the repo from here:

https://github.com/rachhouse/pytest-tutorial

Repo Layout

swapy/

tests/

The toy codebase lives in this folder.

example_test_suite/

test_tutorials/

This folder contains an example of a working test suite for swapy.

This folder contains the code for the examples in these slides.

swapy architecture and design

The wrapper code was written to be simple, functional, and,
most importantly, testable.

pytest basics

pytest integrates with popular editors such as PyCharm and Sublime Text, but we are going to kick it old school with the command line.

It all starts with the pytest command.

Note that all commands will be relative to your pytest-tutorial directory, i.e wherever you cloned the repo, so:

YMMV on the pytest output displayed in these slides versus exactly what you see on your machine (given differences in environments), but it should be reasonably similar.

cd wherever/you/cloned/pytest-tutorial

pytest tests/test_tutorials/test_01_run_pytest/

==================================================== test session starts =====================================================
platform darwin -- Python 3.7.3, pytest-5.1.2, py-1.8.0, pluggy-0.13.0
rootdir: /Users/rachel_house/dev/pytest-tutorial
plugins: arraydiff-0.3, doctestplus-0.4.0, openfiles-0.4.0, pylint-0.14.1, remotedata-0.3.1, cov-2.7.1, dash-1.3.0
collected 1 item

tests/test_tutorials/test_01_run_pytest/test_01.py F                                                                   [100%]

========================================================== FAILURES ==========================================================
__________________________________________________ test_create_swapy_object __________________________________________________

    def test_create_swapy_object():

>       swapy = Swapy(use_cache=False, my_crazy_arg='trekkies4eva')
E       TypeError: __init__() got an unexpected keyword argument 'my_crazy_arg'

tests/test_tutorials/test_01_run_pytest/test_01.py:8: TypeError
===================================================== 1 failed in 0.18s ======================================================

Run this command:

Now take a look at the results:

Let's unpack what just happened.

1. We wrote a .py file containing a test.

We used assert to verify test expectations, namely,
that the newly init'd Swapy object would successfully init and have the correct default _wookie setting.

import pytest

from swapy import Swapy


def test_create_swapy_object():

    swapy = Swapy(use_cache=False, my_crazy_arg='trekkies4eva')

    assert swapy._wookiee == False, 'default wookiee setting should be false'

2. We ran pytest on the command line.

Note that we pointed pytest to the directory, not the actual file.
pytest autodiscovered the appropriate test(s) to run.

pytest tests/test_tutorials/test_01_run_pytest/

If no args supplied, starts from testpaths (if configured) or the current directory

pytest discovery conventions

Within directories, search for test_*.py or *_test.py files:
- From those files, collect:
  - test-prefixed test functions or methods
  - test-prefixed test functions or methods inside Test prefixed test classes (without an __init__ method)

Recurse into directories, unless they match norecursedirs

pytest determines a rootdir for each test run which depends on the command line arguments (specified test files, paths) and on the existence of ini-files.

Also relevant and useful to know...

==================================================== test session starts =====================================================
platform darwin -- Python 3.7.3, pytest-5.1.2, py-1.8.0, pluggy-0.13.0
rootdir: /Users/rachel_house/dev/pytest-tutorial
plugins: arraydiff-0.3, doctestplus-0.4.0, openfiles-0.4.0, pylint-0.14.1, remotedata-0.3.1, cov-2.7.1, dash-1.3.0
collected 1 item

tests/test_tutorials/test_01_run_pytest/test_01.py F                                                                   [100%]

========================================================== FAILURES ==========================================================
__________________________________________________ test_create_swapy_object __________________________________________________

    def test_create_swapy_object():

>       swapy = Swapy(use_cache=False, my_crazy_arg='trekkies4eva')
E       TypeError: __init__() got an unexpected keyword argument 'my_crazy_arg'

tests/test_tutorials/test_01_run_pytest/test_01.py:8: TypeError
===================================================== 1 failed in 0.18s ======================================================

The determined rootdir and ini-file are printed as part of the pytest header during startup.

(no ini-file specified in this example)

3. Something clearly went wrong with the test.

==================================================== test session starts =====================================================
platform darwin -- Python 3.7.3, pytest-5.1.2, py-1.8.0, pluggy-0.13.0
rootdir: /Users/rachel_house/dev/pytest-tutorial
plugins: arraydiff-0.3, doctestplus-0.4.0, openfiles-0.4.0, pylint-0.14.1, remotedata-0.3.1, cov-2.7.1, dash-1.3.0
collected 1 item

tests/test_tutorials/test_01_run_pytest/test_01.py F                                                                   [100%]

========================================================== FAILURES ==========================================================
__________________________________________________ test_create_swapy_object __________________________________________________

    def test_create_swapy_object():

>       swapy = Swapy(use_cache=False, my_crazy_arg='trekkies4eva')
E       TypeError: __init__() got an unexpected keyword argument 'my_crazy_arg'

tests/test_tutorials/test_01_run_pytest/test_01.py:8: TypeError
===================================================== 1 failed in 0.18s ======================================================

pytest kindly displays what tests it ran, and, for any failures, where the error occured in in the test.

Here, we can see that our mistake was trying to start a holy war between Star Wars and Star Trek.

Let's amend this error and rerun pytest.

pytest tests/test_tutorials/test_02_run_pytest_no_failure/

import pytest

from swapy import Swapy


def test_create_swapy_object():

    swapy = Swapy(use_cache=False)

    assert swapy._wookiee == False, 'default wookiee setting should be false'

======================================================= test session starts ========================================================
platform darwin -- Python 3.7.3, pytest-5.1.2, py-1.8.0, pluggy-0.13.0
rootdir: /Users/rachel_house/dev/pytest-tutorial/tests
plugins: arraydiff-0.3, doctestplus-0.4.0, openfiles-0.4.0, pylint-0.14.1, remotedata-0.3.1, cov-2.7.1, dash-1.3.0
collected 1 item

test_tutorials/test_02_run_pytest_no_failure/test_02.py .                                                                    [100%]

======================================================== 1 passed in 0.02s =========================================================

Success!

We can see from the pytest output that our corrected test passed.

Congratulations!

You have now used pytest to run a simple failing and passing test.

Just one more thing before we move onto pytest fixtures...

Previously, we saw one of our tests fail because we supplied an incorrect argument when initializing the Swapy() object.

Testing that a known, expected failure occurs is a useful thing to do, and pytest provides an easy mechanism to do so: pytest.raises

import pytest

from swapy import Swapy


def test_create_swapy_object_fails_with_bad_args():

    with pytest.raises(TypeError):
        swapy = Swapy(use_cache=False, my_crazy_arg='trekkies4eva')

pytest tests/test_tutorials/test_03_pytest_raises/

pytest fixtures

Test fixtures provide a fixed baseline upon which tests can reliably and repeatedly execute.

Fixtures allow your test functions to easily receive and work against specific pre-initialized application objects.

It’s dependency injection: fixture functions take the role of the injector and test functions are the consumers of fixture objects.

Bottom line: test fixtures are incredibly useful, and pytest makes them super easy to create and use.

Test functions can receive pytest fixture objects by naming them as an input arguments.

pytest fixture functions are registered by marking them with the @pytest. fixture decorator

import pytest

from swapy import SwapyBase


@pytest.fixture()
def swapi_base_url():
    return 'https://swapi.co/api'


def test_assemble_swapi_url(swapi_base_url):
    '''Test that swapi api request urls are assembled correctly'''

    swapy_base = SwapyBase()

    # no args to _assemble_swapi_url yields swapi base url
    assert swapy_base._assemble_swapi_url() == swapi_base_url

    # test return of swapi schema url
    resource = 'planets'
    expected_url = '{}/planets/schema'.format(swapi_base_url)
    assert swapy_base._assemble_swapi_url(resource, schema=True) == expected_url

import pytest

from swapy import SwapyBase


@pytest.fixture()
def swapi_base_url():
    return 'https://swapi.co/api'


def test_assemble_swapi_url(swapi_base_url):
    '''Test that swapi api request urls are assembled correctly'''

    swapy_base = SwapyBase()

    # no args to _assemble_swapi_url yields swapi base url
    assert swapy_base._assemble_swapi_url() == swapi_base_url

    # test return of swapi schema url
    resource = 'planets'
    expected_url = '{}/planets/schema'.format(swapi_base_url)
    assert swapy_base._assemble_swapi_url(resource, schema=True) == expected_url

pytest tests/test_tutorials/test_04_simple_fixture/

But wait, there's more!

Cool, right?

conftest.py

conftest.py allows you to share fixture functions across multiple test files

Stick it in your root directory, define those fixtures, and get ready for some sweet fixture reuse across your test suite.

pytest --fixtures test_directory_or_file/

A handy hint:

will show you all your currently defined and available fixtures

monkeypatching

Code often contain dependencies.

Dependencies can include such things as:

a database connection
an external service, accessed via api calls

At test time, we may not be able to (nor want to) access these dependencies:

Dependencies might not be available from the
testing environment
The cost of accessing the dependency may be too high

Mocking allows you to create values, functions, and objects which simulate the behavior of your dependencies.

Mocking is a powerful testing technique.

Possibly more powerful than The Force.

The pytest mechanism for mocking is called monkeypatching.

monkeypatching functions

(slides to come, we'll just dive into the code now)

ohmytest

extremely useful pytest fixtures

tmpdir

(under construction)

datadir

(also under construction)

pdb

python

debugging

best!

(is the)

The debugger is a place where you can step through the guts of your code to see what is happening.

It is also very useful for diagnosing failing tests.

pytest --pdb

pytest --trace

Dropping into the debugger from pytest

Drop into debugger on every failure (or KeyboardInterrupt)

Drop into debugger at the start of every test

the most useful pdb commands

p	print (value of expression)
pp	pretty print
l	list (print) lines of code around current position
n	continue to next line
s	step into
b	set a breakpoint
c	continue execution until next breakpoint or end
q	quit

pytest tricks & treats

pytest marks

the pytest.mark decorator lets you set metadata on your test functions

You can then run tests based on your pytest marks.

To get a list of the n slowest test durations run:

pytest --durations=n

Also a useful little trick:

questions?

resources

The Pytest Website:

https://docs.pytest.org/en/2.8.7/index.html

pytest PDF Documentation:

https://media.readthedocs.org/pdf/pytest/latest/pytest.pdf

The Star Wars API (SWAPI)
https://swapi.co/

pytest

what is pytest?

pytest:

pytest:

pytest:

pytest:

but whytest?

swapy

To get started:

Repo Layout

pytest basics

pytest discovery conventions

pytest fixtures

conftest.py

monkeypatching

ohmytest

pdb

python

debugging

best!

(is the)

Dropping into the debugger from pytest

the most useful pdb commands

pytest tricks & treats

questions?

resources

Resources