Property-based testing
Magda Stożek
Let your testing library work for you
BigDecimal calculateDiscount(Customer customer, LocalDate now);Example: Calculating discount
Years:
0-1
1-2
2 +
0%
30%
10%
20%
birthday
class Customer {
private final String name;
private final LocalDate joinedAt;
private final LocalDate dateOfBirth;
// (...)
}Example-based tests
@Test
void shouldCalculateNoDiscount() {...}@Test
void shouldCalculate10percentDiscount() {...}@Test
void shouldCalculate20percentDiscount() {...}@Test
void shouldCalculateBirthdayDiscount() {...}@Test
void shouldCalculate20percentDiscountFor3Years() {...}Happy path
Edge cases
What about properties?
BigDecimal calculateDiscount(Customer customer, LocalDate now);Example: Calculating discount
Years:
0-1
1-2
2 +
0%
30%
10%
20%
birthday
class Customer {
private final String name;
private final LocalDate joinedAt;
private final LocalDate dateOfBirth;
// (...)
}Property-based tests
@Property
void shouldNotBeLowerThan0() {...}@Property
void shouldNotBeOver30percent() {...}@Property
void shouldBeProportionalToMembershipYears() {...}Properties:
@Property
void shouldNotBeGreaterThanBirthdayDiscount() {...}Libraries
- First:
- 1999: QuickCheck (Haskell) / Quviq Quickcheck (Erlang)
- Koen Claessen and John Hughes
- 1999: QuickCheck (Haskell) / Quviq Quickcheck (Erlang)
- Later:
-
C, C++, C#, Chicken, Clojure, Common Lisp, D, Elm, Elixir, Erlang, F#, Factor, Go, Io, Java, JavaScript, Julia, Kotlin, Logtalk, Lua, Node.js, Objective-C, OCaml, Perl, Prolog, PHP, Pony, Python, R, Racket, Ruby, Rust, Scala, Scheme, Smalltalk, Standard ML, Swift, Typescript, VB.NET, ...
-
What about Java?
Property: discount is never larger than 30%
@Property
boolean shouldNotBeOver30percent(
@ForAll Customer customer,
@ForAll("futureDate") LocalDate now)
{
BigDecimal discount = calculator.calculateDiscount(customer, now);
Assertions.assertThat(discount).compareTo(new BigDecimal("0.3")) <= 0;
} |-------------------jqwik-------------------
tries = 374 | # of calls to property
checks = 374 | # of not rejected calls
generation-mode = RANDOMIZED | parameters are randomly generated
after-failure = PREVIOUS_SEED | use the previous seed
seed = -272292185905571017 | random seed to reproduce generated values
sample =
[Customer{name='AAA', joinedAt=2010-01-01, dateOfBirth=1980-02-29}, 2019-10-06]
original-sample =
[Customer{name='DtQDpPfEx', joinedAt=2010-04-21, dateOfBirth=1980-02-29}, 2185-03-17]
Discovered error
Generators
@Property
boolean shouldBeLongerThanTwo(@ForAll("shortStrings") String string, @ForAll("10 to 99") int number) {
return (string + number).length() > 2;
}
@Provide
Arbitrary<String> shortStrings() {
return Arbitraries.strings().withCharRange('a', 'z').filter('q')
.ofMinLength(1).ofMaxLength(8);
}
@Provide("10 to 99")
Arbitrary<Integer> numbers() {
return Arbitraries.integers().between(10, 99);
}Custom generators
Challenges
Difficult questions earlier
Challenge #1
Nondeterminism
Challenge #2
Documenting
Challenge #3
Coming up with properties
Challenge #4
Strategies
1. Examples first, properties later
2. Properties for key components
Summary
- Higher cost, higher value
- Edge cases and misconceptions
- Coming up with properties
- Balance
More
Questions?
@Property
boolean endOfPresentation(@ForAll Attendee attendee) {
Assume.that(attendee.hasQuestions());
return attendee.questionsAsked > 0;
}
Property-based testing [Java, 30 min]
By Magda Stożek
