Property
Based
Testing
Overview
- What is PBT?
- Why PBT?
- How to PBT?
What is property based testing?
Example based testing
suite : Test
suite =
describe "Add"
[ test "one plus two is three" <|
\_ ->
let
expected =
3
actual =
add 1 2
in
Expect.equal expected actual
]
Example: List.reverse
suite : Test
suite =
describe "The List module"
[ describe "List.reverse"
[ fuzz (list string) "length of list doesn't change" <|
\strList ->
let
originalLength =
List.length strList
in
strList
|> List.reverse
|> List.length
|> Expect.equal originalLength
]
]When a list is reversed, the length should not change
Okay, so what is PBT?
Property based testing relies on properties.
When given any randomly generated value, the property will always be true.
AKA test check useful characteristics that must be seen in the output
Why use property based testing?
Decrease human invlovement
A human is deciding the input and output of example tests.
Decreasing human involvement results is less developer bias.
Which leads to more bugs being caught early on.
You have to think harder
What kind of input is supported?
How can the input be generated?
What statements can we make about the output?
How to property based test?
Categories of properties
- Different paths, same destination
- There and back again
- Some things never change
- The more things change, the more they stay the same
- Hard to prove, easy to verify
Different paths, same destination
[2, 3, 1][3, 4, 2][2, 3, 4][1, 2, 3]sort
sort
+1
+1
There and back again
[3, 4, 2][4, 3, 2]reverse
reverse
Some things never change
[3, 4, 2][2, 3, 4]sort
.length
.length
The more things change, the more they stay the same
[1, 2, 2, 1][1, 2]unique
[1, 2]unique
[1, 2]unique
Hard to prove, easy to verify
[3, 1, 2]sort
implies pairwise order
[1, 2, 3](1 <= 2) (2 <= 3)Let's see an Elm example!
Conclusion
- Property based testing is great!
- But you have to think hard
- When used with example tests, you can rule the world 💪
References
- https://hackernoon.com/property-based-testing-4330e3e77381
- http://blog.jessitron.com/2013/04/property-based-testing-what-is-it.html
- https://lucasmreis.github.io/blog/learning-elm-part-4/
- https://fsharpforfunandprofit.com/posts/property-based-testing/
- https://fsharpforfunandprofit.com/posts/property-based-testing-2/
- https://github.com/elm-community/elm-test/issues/154
- Cucumber Podcast : April 26th, 2017
Property Based Testing
By sarahdherr
