COMP1531

9.3 - SDLC Testing - Property-based Testing

 

Testing: what's the problem?

  • Good testing gives us strong guarantees about our software, but...
    • Tests take a lot of time to write
    • It's easy to miss edge cases (even with coverage checking)
    • These problems only get worse with larger and more complicated systems

Can the machines do it for us?

  • To write a program that generates tests for us requires:
    • A means of generating test data
    • Knowing what behaviour is correct?
    • Some way for the computer to report test failures to us in an understandable way

Example: Bubblesort

def bubblesort(numbers):
    numbers = numbers.copy()
    for _ in range(len(numbers) - 1):
        for i in range(len(numbers) - 1):
            if numbers[i] > numbers[i+1]:
                numbers[i], numbers[i+1] = numbers[i+1], numbers[i]
    return numbers

Property-based testing

  • A method of testing where tests are defined as general properties (i.e. parameterised predicates)
  • Test input is generated automatically by supplying a strategy for generating that input
  • The testing framework runs the test many times to ensure the properties are true for each input
  • In the event of a test failure, the framework will shrink the generated input to find the smallest value that still fails the test

Hypothesis

  • Hypothesis is the name of property-based testing framework for python

  • Can be installed via: ​pip3 hypothesis

  • Parameterised tests are decorated with @given to supply strategies for generating test input
  • See bubblesort.py

What properties to test?

  • It's not always easy to find testable properties
  • Can you think of one for the Zune bug example?
  • Software designs with testable properties tend to be good designs...
Made with Slides.com