Property-based testing

Magda Stożek

@magdastozek

Let your testing library work for you

def calculateDiscount(customer: Customer, now: LocalDate): BigDecimal

Example: Calculating discount

Years:

0-1

1-2

2 +

0%

30%

10%

20%

birthday

case class Customer(
    name: String, 
    joinedAt: LocalDate, 
    dateOfBirth: LocalDate
)

Example-based tests

it "should calculate no dicount" in {...}
it should "calculate 10 percent discount" in {...}
it should "calculate 20 percent discount" in {...}
it should "calculate birthday discount" in {...}
it "should calculate 20 percent discount for 3 years" in {...}

Happy path

Edge cases

What about properties?

def calculateDiscount(customer: Customer, now: LocalDate): BigDecimal

Example: Calculating discount

Years:

0-1

1-2

2 +

0%

30%

10%

20%

birthday

case class Customer(
    name: String, 
    joinedAt: LocalDate, 
    dateOfBirth: LocalDate
)

Libraries

  • First:
  • Later:

    • C, C++, C#, Chicken, Clojure, Common Lisp, D, Elm, Elixir, Erlang, F#, Factor, Go, Io, Java, JavaScript, Julia, Kotlin, Logtalk, Lua, Node.js, Objective-C, OCaml, Perl, Prolog, PHP, Pony, Python, R, Racket, Ruby, Rust, Scala, Scheme, Smalltalk, Standard ML, Swift, Typescript, VB.NET, ...

What about Scala?

Also keep an eye on:

Property-based tests

it "should not be lower than 0" in {...}
it "should not be over 30 percent" in {...}
it "should be proportional to membership years" in {...}

Properties:

it "should not be greater than birthday discount" in {...}
property("Discount should not be over 30 percent") {
  forAll { (customer: Customer, now: LocalDate) =>
    val discount = calculator.calculateDiscount(customer, now)
    discount should be <= BigDecimal(0.3)
  }
}

Property: discount is never larger than 30%

ScalaTestFailureLocation: pbt.DiscountCalculatorPropertySpec at (DiscountCalculatorPropertySpec.scala:28)
org.scalatest.exceptions.GeneratorDrivenPropertyCheckFailedException: DateTimeException was thrown during property evaluation.
  Message: Invalid date 'February 29' as '2049' is not a leap year
  Occurred when passed generated values (
    arg0 = Customer(nWtCZLoAQ,2009-04-14,1988-02-29),
    arg1 = Customer(bLNkLdYmCYCjVJlIDRWumkWZceQHyoulrCcietBLPyfBCqbyXqub,2015-12-19,1996-12-13),
    arg2 = 2049-04-09
  )

Discovered error

Generators

Custom generators

val customerGen: Gen[Customer] = for {
    name <- Gen.alphaStr
    age <- Gen.choose(12, 120)
    subscription <- Gen.option(Gen.oneOf("Daily", "Monthly", "Yearly")
    dateJoined <- joinedGen()
  } yield Customer(name, age, subscription, dateJoined)

Challenges

Difficult questions earlier

Challenge #1

Nondeterminism

Challenge #2

Coming up with properties

Challenge #3

fsharpforfunandprofit.com/posts/property-based-testing-2/

Strategies

1. Examples first, properties later

2. Properties for key components

Summary

  1. Higher cost, higher value
  2. Edge cases and misconceptions
  3. Coming up with properties
  4. Balance

More

Questions?

property("End of presentation") {
  forAll { (attendee: Attendee) =>
    whenever(attendee.hasQuestions) {
      attendee.questionsAsked should be > 0
    }
  }
}

@magdastozek

http://bit.ly/magda-scalalove

https://bit.ly/magda-pbt-scala

Property-based testing [Scala, 20 min]

By Magda Stożek

Property-based testing [Scala, 20 min]

  • 27

More from Magda Stożek