Testing non-deterministic code

Part 1:

Determinism

Determinism is the philosophical position that for every event there exist conditions that could cause it.

Wikipedia

Exercise

Exercise

4 + 2 = ?

3

(haha…)

Exercise

x + 2 = 6

x = ?

x = 4

Exercise

x² + 2 = 6

x = ?

x = 2

OR

x = -2

c^2 = a^2 + b^2 - 2ab\ \cos\ \gamma.

Exercise

\frac{d \hat{A}_H (t)}{dt} = \frac{i}{\hbar} \left[ \hat H , \hat A_H(t) \right] + \left(\frac{\partial \hat A_S(t)}{\partial t}\right)_H

Exercise

Part 2:

Determinism in computing

Determinism in computing

    (…) A deterministic system is a system in which no randomness is involved in the development of future states of the system.

Wikipedia

Determinism in computing

Anything that is based on programmatically unpredictable behavior is considered non-deterministic.

Random number generation

Pseudo-random number generators (PRNG)

are deterministic.

Achieving true randomness

Source: https://www.cloudflare.com/learning/ssl/lava-lamp-encryption/

CloudFlare lava lamps wall, "Lavarand"

Part 3:

How to test randomness

How to test randomness

You don't.

Create mocks.

Testing

class Calculator
{
    public function add(float $a, float $b): float
    {
        return $a + $b;
    }
}

use PHPUnit\Framework\TestCase;

class CalculatorTest extends TestCase
{
    public function testAdd(): void
    {
        $math = new Calculator();
        
        $result = $math->add(1, 2);

        static::assertSame(3, $result);
    }
}

Coding randomness

class DiceRoller
{
    public function roll(
    	int $numberOfDice = 1,
        int $diceSides,
        int $bonus = 0
    ): int {
        $result = $bonus;

        for ($i = 0; $i < $numberOfDice; ++$i) {
            $result += random_int(1, $diceSides);
        }

        return $result;
    }
}
// Roll 2d6+3
$diceRoller->roll(2, 6, 3);

Testing randomness:

Potential solutions

Solution 1:

 

Testing a lot of cases.

Testing a lot of cases

namespace Tests\App;

use PHPUnit\Framework\TestCase;

class DiceRollerTest extends TestCase
{
    public function provide dice rolls(): \Generator
    {
        $sidesToTest = [4, 6, 8, 12, 20]; // d4, d6, etc.
        $numberOfDicesToTest = range(1, 10); // Up to 10 dices at the same time.
        $bonuses = [1, 2, 3, 4, 5]; // Not too much, that's already a lot.

        foreach ($sidesToTest as $diceSides) {
            foreach ($numberOfDicesToTest as $numberOfDice) {
                foreach ($bonuses as $bonus) {
                    yield "$diceSides-$numberOfDice-$bonus" => [$diceSides, $numberOfDice, $bonus];
                }
            }
        }
    }
}
namespace Tests\App;

use PHPUnit\Framework\TestCase;

class DiceRollerTest extends TestCase
{
    // ...

    /** @dataProvider provide dice rolls */
    public function test dice roller result is in dice range(
    	int $numberOfDice = 1,
        int $diceSides, int $bonus = 0
    ): void {
        $diceRoller = new DiceRoller();

        $result = $diceRoller->roll($sides, $multiplier, $offset);

        self::assertGreaterThanOrEqual(1 * $multiplier + $offset, $result);
        self::assertLessThanOrEqual($sides * $multiplier + $offset, $result);
    }
}

Testing a lot of cases

Testing a lot of cases

Pros:

  • Easy to set up

Cons:

  • Lots of tests
  • Statistically unstable

Testing a lot of cases

random_int() is not deterministic.

$diceRoller = new App\DiceRoller();

$count = 1000000;
$results = [];

for ($i = 1; $i <= $count; $i++) {
    $results[] = $diceRoller->roll(2, 6, 3);
}

// Average value
echo array_sum($results) / $count, "\n";

$ for i in {1..10}; do php roll.php; done
11.998764
11.997870
12.003618
12.000348
11.999262
11.998068
11.993424
12.000720
12.003618
12.000378

Testing randomness:

Potential solutions

Solution 2:

 

Override the random_int() function

in our tests

Override the

random_int() function

Don't do it.

Testing randomness:

Potential solutions

Solution 3:

 

Create mocks

Create mocks

interface RandomIntProviderInterface
{
    public function randomInt(int $min, int $max): int;
}

Create mocks

class DiceRoller
{





    public function roll(
    	int $numberOfDice = 1,
        int $diceSides,
        int $bonus = 0
    ): int {
        $result = $bonus;

        for ($i = 0; $i < $numberOfDice; ++$i) {
            $result += random_int(1, $diceSides);
        }

        return $result;
    }
}

Use mocks

class DiceRoller
{
    public function __construct(
    	private RandomIntProviderInterface $randomIntProvider
    ) {
    }

    public function roll(
    	int $numberOfDice = 1,
        int $diceSides,
        int $bonus = 0
    ): int {
        $result = $bonus;

        for ($i = 0; $i < $numberOfDice; ++$i) {
            $result += $this->randomIntProvider->randomInt(1, $diceSides);
        }

        return $result;
    }
}

Implement mocks

class NativeRandomIntProvider implements RandomIntProviderInterface
{
    public function randomInt(int $min, int $max): int
    {
        return \random_int($min, $max);
    }
}

Implement mocks

class DeterministicRandomIntProvider implements RandomIntProviderInterface
{
    public int $determinedResult = 0;

    public function randomInt(int $min, int $max): int
    {
        return $this->determinedResult;
    }
}

Using our mock in test

class DiceRollerTest extends TestCase
{
    public function test dice roller result(): void {
        $randomIntProvider = new DeterministicRandomIntProvider();

        $diceRoller = new DiceRoller($randomIntProvider);

        $randomIntProvider->determinedResult = 1;

        $result = $diceRoller->roll(2, 6, 3); // 2d6+3

        self::assertSame(5, $result); // Yay!
    }
}

Conclusion

  • Don't test random input
  • Make randomness a third-party service by using a custom interface
  • Implement this 3rd-party and use native randomness for true usage
  • Mock this 3rd-party service in your tests to have deterministic results

@pierstoval

Alex Rock

Freelance dev, architect, coach & trainer @Orbitaleio

Merci !

Testing randomness

By Alex Rock

Testing randomness

-

  • 409

More from Alex Rock