Property based testing in Ruby

Abhishek Yadav

@h6165

Property based testing

What is it ?
Code examples
??
Profit

Property based testing

Most testing we do in Ruby is example based
(Rspec tests are called examples)
We try to come up with examples that cover the various usage scenarios. And we verify with literal results.
We generally don't go by the properties
All possible cases are often not covered, obviously

# Example of examples


def add(a, b)
  a + b
end


# Test examples -

expect(add(1,2)).to eq(3)
expect(add(0,0)).to eq(0)
expect(add(-1,1)).to eq(0)

# and so on...

Property based testing

That seems ok

# Pythagoras


def hypotenuse(a, b)
  Math.sqrt(a**2 + b**2)
end


# Test examples -

expect(hypotenuse(3, 4)).to eq(5.0)
expect(hypotenuse(5, 7)).to eq(8.6)
expect(hypotenuse(0, 0)).to eq(0)
expect(hypotenuse(-1, -2))
  .to raise_error

# and so on...

Property based testing

Here,

we desperately feel the urge to express the Pythagoras theorem

To say, that the square of the hypotenuse equals the sum of the squares of the sides

That, is a property

# Pythagoras


def hypotenuse(a, b)
  Math.sqrt(a**2 + b**2)
end


# What we need - 

for_all_possible_values {
  h = hypotenuse(a,b)
  expect(h**2).to eq(a**2 + b**2)
}

Property based testing

The square of the hypotenuse equals the sum of the squares of the sides

Property based testing

Haskells Quickcheck

Haskell folks built a library for just this
(and published a couple of papers too)
What the library does -
- Help generate a random test data
- Help express the properties
- Shrink the failure cases
Quickcheck - http://www.cse.chalmers.se/~rjmh/QuickCheck/manual.html
Quickcheck has been ported to most languages
https://en.wikipedia.org/wiki/QuickCheck

Property based testing

Ruby's Quickcheck

Ruby has a few implementations -
- rubycheck - a direct port
  https://github.com/mcandre/rubycheck
- theft -
  https://rubygems.org/gems/theft
- rantly
  https://github.com/rantly-rb/rantly

Property based testing

Ruby's Rantly

# Pythagoras
# Property Test -

it 'hypotenuse squared is sum of sides squared' do

  property_of {
    a = integer
    b = integer
    [a, b]
  }
  .check(200) { |a, b|
    h2 = a**2 + b**2
    h2x = hypotenuse(a, b) ** 2
    expect(h2x).to eq(h2)
  }


end

Property based testing

Ruby's Rantly

  property_of {
    a = integer
    b = integer
    [a, b]
  }

Generate random test data

.check(200) { |a, b|
  h2 = a**2 + b**2
  h2x = hypotenuse(a, b) ** 2
  expect(h2x).to eq(h2)
}

Verify our function using the test data

(with 200 different data points)

Property based testing

Ruby's Rantly

.check(200) { |a, b|
  h2 = a**2 + b**2
  h2x = hypotenuse(a, b) ** 2
  expect(h2x).to eq(h2)
}

Does that feel a bit wrong ?

We've almost specified the implementation here

Is that acceptable ?

Why or why not ?

Property based testing

Ruby's Rantly

Demo

Property based testing

Conclusion - Example tests vs Property tests

Property tests are more useful with algorithms, formulae etc, where -
- there are lots of test values
- there's a risk of missing edge cases
- functions are pure
Example unit tests - more useful with business logic, where code is mostly impure

Property based testing

Conclusion - Example tests vs Property tests

That's all