Magda Stożek

Property-based testing

Let your testing library work for you

BigDecimal calculateDiscount(Customer customer, LocalDate now);

Example: Calculating discount

Years:

0-1

1-2

2 +

0%

30%

10%

20%

birthday

class Customer {
    private final String name;
    private final LocalDate joinedAt;
    private final LocalDate dateOfBirth;
// (...)
}

Example-based tests

@Test
void shouldCalculateNoDiscount() {...}
@Test
void shouldCalculate10percentDiscount() {...}
@Test
void shouldCalculate20percentDiscount() {...}
@Test
void shouldCalculateBirthdayDiscount() {...}
@Test
void shouldCalculate20percentDiscountFor3Years() {...}

Happy path

Edge cases

What about properties?

BigDecimal calculateDiscount(Customer customer, LocalDate now);

Example: Calculating discount

Years:

0-1

1-2

2 +

0%

30%

10%

20%

birthday

class Customer {
    private final String name;
    private final LocalDate joinedAt;
    private final LocalDate dateOfBirth;
// (...)
}

Property-based tests

@Property
void shouldNotBeLowerThan0() {...}
@Property
void shouldNotBeOver30percent() {...}
@Property
void shouldBeProportionalToMembershipYears() {...}

Properties:

@Property
void shouldNotBeGreaterThanBirthdayDiscount() {...}
@Property
boolean shouldNotBeOver30percent(
    @ForAll Customer customer, 
    @ForAll("futureDate") LocalDate now) 
{
    BigDecimal discount = calculator.calculateDiscount(customer, now);
    Assertions.assertThat(discount).compareTo(new BigDecimal("0.3")) <= 0;
}

Property: discount is never larger than 30%

Discovered error

                              |-------------------jqwik-------------------
tries = 374                   | # of calls to property
checks = 374                  | # of not rejected calls
generation-mode = RANDOMIZED  | parameters are randomly generated
after-failure = PREVIOUS_SEED | use the previous seed
seed = -272292185905571017    | random seed to reproduce generated values
sample = 
[Customer{name='AAA', joinedAt=2010-01-01, dateOfBirth=1980-02-29}, 2019-10-06]
original-sample = 
[Customer{name='DtQDpPfEx', joinedAt=2010-04-21, dateOfBirth=1980-02-29}, 2185-03-17]

Challenges

Difficult questions earlier

Challenge #1

Nondeterminism

Challenge #2

fsharpforfunandprofit.com/posts/property-based-testing-2/

Coming up with properties

Challenge #3

Strategies

1. Examples first, properties later

2. Properties for key components

Summary

  1. Higher cost, higher value
  2. Edge cases and misconceptions
  3. Coming up with properties
  4. Balance

More

@Property
boolean endOfPresentation(@ForAll Attendee attendee) {
    Assert.that(attendee.wantsToTryIt);
}

http://bit.ly/properties-15

Thanks!