Unit testing with Python

Introduction
    - Benefits of Unit testing

Python tools
    - Python different test tools.

Writing tests for different scenarios
    - Simple test case
    - data driven/Decorated test (@ddt)
    - Generative testing
    - Mock
        - The Mock object
        - Different ways of using
        - Patch

Running tests
    - Command line interface

Integrating with CI (Travis)
    - Github and Travis example

What we will cover today

Unit tests are

  • fast
  • accurate
  • reliable
  • simple
  • highlight weaknesses (when not easy)
  • great value

Python tools available

unittest

the standard library included in Python.
Familiar to anyone who has used any of JUnit/nUnit/CppUnit series of tools

py.test

Also popular and an alternative to Python's standard unittest module.
It boasts a simple syntax

Some
Test
Examples

Simple test case (to get started)

import json
import datetime

class activity_ConvertJATS():

    def add_update_date_to_json(self, json_string, update_date):
        try:
            json_obj = json.loads(json_string)
            updated_date = datetime.datetime.strptime(update_date, "%Y-%m-%dT%H:%M:%SZ")
            update_date_string = updated_date.strftime('%Y-%m-%dT%H:%M:%SZ')
            json_obj['update'] = update_date_string
            json_string = json.dumps(json_obj)
        except:
            if self.logger:
                self.logger.error("Unable to set the update date in the json")
        return json_string
import unittest
from simple_sample import activity_ConvertJATS
import json

input = open("tests/test_data/input.json","r").read()
output = json.loads(open("tests/test_data/output.json","r").read())

class MyTestCase(unittest.TestCase):

    def test_add_update_to_json(self):
        self.jats = activity_ConvertJATS()
        result = self.jats.add_update_date_to_json(input,'2012-12-13T00:00:00Z')
        self.assertDictEqual(json.loads(result), output)

if __name__ == '__main__':
    unittest.main()

Data driven/decorated test

class activity_DepositAssets():

    def get_no_download_extensions(self, no_download_extensions):
        return [x.strip() for x in no_download_extensions.split(',')]
import unittest
from ddt import ddt, data, unpack
from sample import activity_DepositAssets

@ddt
class MyTestCase(unittest.TestCase):
    def setUp(self):
        self.depositassets = activity_DepositAssets()

    @unpack
    @data({'input': '.tif', 'expected': ['.tif']},
          {'input': '.jpg, .tiff, .png', 'expected':['.jpg', '.tiff', '.png']})
    def test_get_no_download_extensions(self, input, expected):
        result = self.depositassets.get_no_download_extensions(input)
        self.assertListEqual(result, expected)


if __name__ == '__main__':
    unittest.main()

sample.py

tests/test_sample_ddt_unpack.py

Generative/Property Based Testing

Improve product quality and find bugs faster by generating tests

Property based testing is a method of testing functions pioneered by the Haskell community. From Hackage:

 

QuickCheck is a library for random testing of program properties.

The programmer provides a specification of the program, in the form of properties which functions should satisfy, and QuickCheck then tests that the properties hold in a large number of randomly generated cases.

  • A developer who writes a function will naturally think of  examples that work. Generators are more like (super powerful) QAs. They will find examples that don't work. 
  • Rather than a handful of examples, generators describe your data explicitly.
  • Null values, empty strings, divide by zero are obvious examples often forgotten and generators will show them straight away.
def encode(input_string):
    count = 1
    prev = ''
    lst = []
    for character in input_string:
        if character != prev:
            if prev:
                entry = (prev, count)
                lst.append(entry)
            count = 1
            prev = character
            else:
                count += 1
        else:
            entry = (character, count)
            lst.append(entry)
    return lst


def decode(lst):
    q = ''
    for character, count in lst:
        q += character * count
    return q
from hypothesis import given
from hypothesis.strategies import text

@given(text())
def test_decode_inverts_encode(s):
    assert decode(encode(s)) == s

First try

Falsifying example: test_decode_inverts_encode(s='')

UnboundLocalError: local variable 'character' referenced before assignment

This code is simply wrong when called on an empty string.

Fix

if not input_string:
    return []

Mocking
with
Python

What is Mocking?

"Sometimes, you need "other" code resources for your test setup. But those resources may be unavailable, unstable, or just too unwieldy to use. You could try and find a replacement for the missing resource; or you could simulate it by creating what is known as a mock. Mocks let us simulate resources that are either unavailable or too unwieldy for unit testing."

http://www.drdobbs.com/testing/using-mocks-in-python/240168251

Mocking in Python is done by using patch to hijack an API function or object creation call. When patch intercepts a call, it returns a MagicMock object by default. By setting properties on the MagicMock object, you can mock the API call to return any value you want or raise an Exception.

https://blog.fugue.co/2016-02-11-python-mocking-101.html

How do we mock in Python?

@patch('requests.post')
def test_schedule_article_publication(self, mock_requests_post):

    mock_requests_post.return_value = FakeResponse(200, {'result': 'success'})

    input = '{"articles":{"article-identifier":"03430","scheduled":"1463151540"}}'
    resp = self.client.post('/api/schedule_article_publication', data=input)
    self.assertDictEqual(json.loads(resp.data), {'result': 'success'})

Most used attributes of a MagicMock instance:

return_value

@data(data_published_lax)
@patch.object(activity_VerifyPublishResponse, 'publication_authority')
@patch.object(activity_VerifyPublishResponse, 'emit_monitor_event')
def test_do_activity(self, data, fake_emit_monitor, fake_publication_authority):
    fake_publication_authority.return_value = "elife2.0"
    result = self.verifypublishresponse.do_activity(data)
    fake_emit_monitor.assert_called_with(settings_mock,
                                         data["article_id"],
                                         data["version"],
                                         data["run"],
                                         self.verifypublishresponse.pretty_name,
                                         "end",
                                         "Finished Verification " + data["article_id"])
    self.assertEqual(result, self.verifypublishresponse.ACTIVITY_SUCCESS)

assert_called_with

Running unit tests

Examples:
  python -m unittest test_module                           - run tests from test_module
  python -m unittest module.TestClass                  - run tests from module.TestClass
  python -m unittest module.Class.test_method  - run specified test method

The unittest module can be used from the command line to run tests from modules, classes or even individual test methods:

 

Integrating with CI

Any questions?