I hate writing tests, that's why I use Hypothesis

Cheuk Ting Ho

Cheukting

@cheukting_ho

https://cheuk.dev

🔗 slides.com/cheukting_ho/hypothesis-ghostwriter

What is the most difficult part of writing tests?

  • Importing the code
     
  • Organizing the tests
     
  • Think of test cases

Have you heard of property-based testing?

Property

  • the given is obvious

     
  • works extra well with typing

     
  • edge case automatically be found

Example

  • need to think of what is and what is not
     
  • take extra steps to write examples

     
  • may overlook edge cases

Testing by...

How does Hypothesis do it?

decorators

entry point to modify the test

strategies

generating test data

def encode(input_string):
    count = 1
    prev = ""
    lst = []
    for character in input_string:
        if character != prev:
            if prev:
                entry = (prev, count)
                lst.append(entry)
            count = 1
            prev = character
        else:
            count += 1
    entry = (character, count)
    lst.append(entry)
    return lst


def decode(lst):
    q = ""
    for character, count in lst:
        q += character * count
    return q
from hypothesis import given
from hypothesis.strategies import text


@given(text())
def test_decode_inverts_encode(s):
    assert decode(encode(s)) == s

Details: https://hypothesis.readthedocs.io/en/latest/quickstart.html

Hypothesis for the scientific stack

Numpy

Pandas

What if I am really lazy?

But I hope you are still using typing 🙃

We have the 👻 ghostwriter for you

  • Using the typing as a hint
     
  • pick the right strategy for input parameters
     
  • CLI tool included
     
  • Automatically `black` the code for you

We have the 👻 ghostwriter for you

hypothesis write gzip

Just type in command line...

Ghostwriters 👻 available are...

Details: https://hypothesis.readthedocs.io/en/latest/ghostwriter.html

Fuzz

checks that valid input only leads to expected exceptions

from re import compile, error

from hypothesis.extra import ghostwriter

ghostwriter.fuzz(compile, except_=error)

Idempotent

result does not change when use the funciton on its own output

from typing import Sequence

from hypothesis.extra import ghostwriter


def timsort(seq: Sequence[int]) -> Sequence[int]:
    return sorted(seq)


ghostwriter.idempotent(timsort)

Roundtrip

calling the 2nd function to the result of the 1st one will go back to the input

import json

from hypothesis.extra import ghostwriter

ghostwriter.roundtrip(json.dumps, json.loads)

Equivalent

check the 1st function has the same effect as the 2nd function

import math

from hypothesis.extra import ghostwriter


def my_pow(x, y):
  result = 1.0
  for _ in range(y):
    result *= x
  return result


ghostwriter.equivalent(my_pow, math.pow)

... and 2 more 

binary_operation : for testing binary operators

ufunc : for Numpy array ufunc

Using the CLI tool

Let's try 😉

... few things to consider though...

  • tests can run much slower
    (generate strategies are expansive)
     
  • tests can be harder to understand
     
  • if no typing not much can be done
    (don't be too lazy 👻)