Property-based testing
Pavel Lavreshin
Test pyramid
Pros
Cheap to implement
Cons
Fixed set of test data
Unit testing
Easy to maintain
Flexible to extend
Error-prone when writing
hundreds of cases manually
May forget edge cases
Property based testing
to the rescue!
- Implementing general properties rather specific test cases
- Generating test cases automatically rather than writing those manually
- Covering lots of edge cases out of the box
Features:
(originated from Quickcheck, Haskell)
Scalacheck Concepts
- Properties - a testable unit, can be combined or extended
val p1 = forAll(...)
val p2 = forAll(...)
val p3 = p1 && p2
val p4 = p1 || p2
val p5 = p1 == p2
val p6 = all(p1, p2) // same as p1 && p2
val p7 = atLeastOne(p1, p2) // same as p1 || p2- Generators - random or specified test data generators
- ArbitraryValues - test data for property parameters
Code snippets
// sample palindrome generator
val palindromeGen = for {
s <- arbitrary[String]
opt <- Gen.option(arbitrary[Char])
} yield s + opt.getOrElse("") + s.reverse
// sample numeric generator
val numeric = Gen.choose(2, 9)
// sample Ranks generator
val RanksArbitrary: Arbitrary[Rank] = Arbitrary(Gen.oneOf(Rank.All))
// longer sample
implicit val PlayerArbitrary: Arbitrary[Player] = Arbitrary {
for {
firstName <- Gen.alphaUpperStr
lastName <- Gen.alphaUpperStr
playerId <- arbitrary[Long]
state <- PlayerStateArbitrary.arbitrary
} yield Player(
firstName = firstName,
lastName = lastName,
playerId = playerId,
state = state)
}
Tests
property("checkPalindrome") = {
forAll(palindromeGen) { x => isPalindrome(x) }
}
property("checkPalindromeWithCollect") = {
forAll(palindromeGen) { x => collect(x)(isPalindrome(x)) }
}
property("checkRanksWhenRead") = {
forAll(Gen.choose(2, 9)) { x => collect(x)(isValidRankWhenRead(x)) }
}
property("checkRanksWhenConverted") = {
forAll(RanksArbitrary.arbitrary) { x => collect(x)(isValidRankWhenConverted(x)) }
}References
- Scalacheck - scalacheck.org
- Scalatest - scalatest.org
- Repository - github.com/plavreshin/property-based-testing
- User guide - https://github.com/rickynils/scalacheck/blob/master/doc/UserGuide.md