The Good, The Bad and The Repeat

Pedro Costa
@pfac on GitHub

UTRUST_guest

utrustwelcome

Goal for today:

Show you something new

About testing

Types of
Functional
Tests

Unit tests

  • We built something right
  • Based on example inputs,
    and expected outputs

Property tests

  • Examples are limited
  • Define full space of valid inputs
  • Define properties the output must hold
  • Randomly generate inputs and test them

Integration tests

(tests for the integrations)

  • Test communication w/ outside
  • Assume a contract
  • Check the implementation respects it

Acceptance tests

  • We built the right thing
  • Entire system is set up
  • Mimic user interaction
  • Based on requirements

Mutation tests

  • Test the tests
  • Predict simple regressions
  • Anticipate human error

Who watches the watchmen?

Step 1

Create a mutant
of your code

Step 2

Run the tests
for that code

Step 3

Tests must fail

https://github.com/dnlserrano/exavier

Practical Test Pyramid

Property
tests

Mutation tests

Integration
tests

More property
tests?

More unit tests

  • Cover known regressions
  • Cover known edge cases
  • Show expected usage

Property test with care

  • Not enough iterations, not enough input
  • Too many iterations, too slow
  • Recommended: less locally, more on CI

Test each integration

  • Each boundary has a contract
  • Make sure you uphold the contract
  • Check if the contract didn't change

Acceptance test
for each requirement

  • Use as measurement of progress
  • Provides confidence stuff works
  • Run on CI, locally only as needed

Mutation test
critical components

  • Fortify tests to cover human error
  • Cover for regressions before they happen
  • Slow AF, tread lightly

Mutation tests
❤️
Property tests

Resources

Thanks!

Pedro Costa
@pfac on GitHub

Made with Slides.com