Property
Based
Testing

Overview

  • What is PBT?
  • Why PBT?
  • How to PBT?

What is property based testing?

Example based testing

suite : Test
suite =
    describe "Add"
        [ test "one plus two is three" <|
                \_ ->
                    let
                        expected =
                            3

                        actual =
                            add 1 2
                    in
                    Expect.equal expected actual
        ]

Example: List.reverse

suite : Test
suite =
    describe "The List module"
        [ describe "List.reverse"
            [ fuzz (list string) "length of list doesn't change" <|
                \strList ->
                    let
                        originalLength =
                            List.length strList
                    in
                    strList
                        |> List.reverse
                        |> List.length
                        |> Expect.equal originalLength
            ]
        ]

When a list is reversed, the length should not change

Okay, so what is PBT?

Property based testing relies on properties.

 

When given any randomly generated value, the property will always be true.

AKA test check useful characteristics that must be seen in the output

Why use property based testing?

Decrease human invlovement

A human is deciding the input and output of example tests.

 

Decreasing human involvement results is less developer bias.

 

Which leads to more bugs being caught early on.

You have to think harder

What kind of input is supported?

 

How can the input be generated?

 

What statements can we make about the output?

How to property based test?

Categories of properties

  • Different paths, same destination
  • There and back again
  • Some things never change
  • The more things change, the more they stay the same
  • Hard to prove, easy to verify

Different paths, same destination

[2, 3, 1]
[3, 4, 2]
[2, 3, 4]
[1, 2, 3]

sort

sort

+1

+1

There and back again

[3, 4, 2]
[4, 3, 2]

reverse

reverse

Some things never change

[3, 4, 2]
[2, 3, 4]

sort

.length

.length

The more things change, the more they stay the same

[1, 2, 2, 1]
[1, 2]

unique

[1, 2]

unique

[1, 2]

unique

Hard to prove, easy to verify

[3, 1, 2]

sort

implies pairwise order

[1, 2, 3]
(1 <= 2) (2 <= 3)

Let's see an Elm example!

Conclusion

  • Property based testing is great!
  • But you have to think hard
  • When used with example tests, you can rule the world 💪

References

  • https://hackernoon.com/property-based-testing-4330e3e77381
  • http://blog.jessitron.com/2013/04/property-based-testing-what-is-it.html
  • https://lucasmreis.github.io/blog/learning-elm-part-4/
  • https://fsharpforfunandprofit.com/posts/property-based-testing/
  • https://fsharpforfunandprofit.com/posts/property-based-testing-2/
  • https://github.com/elm-community/elm-test/issues/154
  • Cucumber Podcast : April 26th, 2017
Made with Slides.com