Property-based testing

Magda Stożek

Let your testing library work for you

Property-based testing: テストライブラリ活用方法
def calculateDiscount(customer: Customer, now: LocalDate): BigDecimal

Example: Calculating discount

Years:

0-1

1-2

2 +

0%

30%

10%

20%

birthday

case class Customer(
    name: String, 
    joinedAt: LocalDate, 
    dateOfBirth: LocalDate
)
顧客の継続年数によって割引率を計算する

Example-based tests

it "should calculate no didcount" in {...}
it should "calculate 10 percent discount" in {...}
it should "calculate 20 percent discount" in {...}
it should "calculate birthday discount" in {...}
it "should calculate 20 percent discount for 3 years" in {...}

Happy path

Edge cases

例ベースのテストならこんな感じ

What about properties?

プロパティベーステストの世界では、これがどのように異なるか見てみましょう
def calculateDiscount(customer: Customer, now: LocalDate): BigDecimal

Example: Calculating discount

Years:

0-1

1-2

2 +

0%

30%

10%

20%

birthday

case class Customer(
    name: String, 
    joinedAt: LocalDate, 
    dateOfBirth: LocalDate
)
どんなルールがあるか考えてみましょう

Property-based tests

it "should not be lower than 0" in {...}
it "should not be over 30 percent" in {...}
it "should be proportional to membership years" in {...}

Properties:

it "should not be greater than birthday discount" in {...}
四つのルール
property("Discount should not be over 30 percent") {
  forAll { (customer: Customer, now: LocalDate) =>
    val discount = calculator.calculateDiscount(customer, now)
    discount should be <= BigDecimal(0.3)
  }
}

Property: discount is never larger than 30%

プロパティベーステストの書き方
ScalaTestFailureLocation: pbt.DiscountCalculatorPropertySpec at (DiscountCalculatorPropertySpec.scala:28)
org.scalatest.exceptions.GeneratorDrivenPropertyCheckFailedException: DateTimeException was thrown during property evaluation.
  Message: Invalid date 'February 29' as '2049' is not a leap year
  Occurred when passed generated values (
    arg0 = Customer(nWtCZLoAQ,2009-04-14,1988-02-29),
    arg1 = Customer(bLNkLdYmCYCjVJlIDRWumkWZceQHyoulrCcietBLPyfBCqbyXqub,2015-12-19,1996-12-13),
    arg2 = 2049-04-09
  )

Discovered error

見つかったエラーの例

Generators

データを生成する

Custom generators

val customerGen: Gen[Customer] = for {
    name <- Gen.alphaStr
    age <- Gen.choose(12, 120)
    subscription <- Gen.option(Gen.oneOf("Daily", "Monthly", "Yearly")
    dateJoined <- joinedGen()
  } yield Customer(name, age, subscription, dateJoined)
カスタムクラスのジェネレーター

Challenges

プロパティベーステストの課題

Difficult questions earlier

Challenge #1

難題が前倒しになります

Nondeterminism

Challenge #2

ランダム性があるため、バグが後から見つかる可能性があります

Coming up with properties

Challenge #3

どんなルールがあるか考えてみましょう

Strategies

1. Examples first, properties later

2. Properties for key components

(1)例ベーステストから書きます、それからプロパティベーステスト
(2)重要なコンポーネントだけプロパティベーステストを書きます

Summary

  1. Higher cost, higher value
  2. Edge cases and misconceptions
  3. Coming up with properties
  4. Balance
(1)高いコストだが高品質
(2)エッジケースと誤解を見つける
(3)プロパティを思いつけるようにする
(4)二種類のテストのバランスを取る

More

参考資料

Questions?

property("End of presentation") {
  forAll { (attendee: Attendee) =>
    whenever(attendee.hasQuestions) {
      attendee.questionsAsked should be > 0
    }
  }
}
http://bit.ly/magda-scalalove
ご清聴ありがとうございました

Property-based testing [Scala, 20 min]

By Magda Stożek

Property-based testing [Scala, 20 min]

  • 221