Level-up Your Testing Toolkit!

Janina

Micha

I am…

👨‍💻 a Software Developer/
    Software Crafting enthusiast

👧🏼 a father to a daughter

🥦 a vegetarian


I like…

🏄‍♂️ my flow to be continuous,

🙅‍♂️ my stress minimized ⇒ QA test automation

🏃‍♂️ running

🇫🇷 learning French

🍺 drinking Craft Beer

I am…

👩‍💻 a Software Developer/Crafter

🥦 a vegetarian


I like…

💡Learning & Improving ⇒ Software Teaming

🔄 Automatization

🎨 Drawing

👩‍🌾 Gardening

🎮 Video Games

🐶 Dogs 💜

Parameteriz.Testing

722-45173-78

€ fewer tests

€ test duplication

€ more values tested

free shipping (OSS)

Join the QAC
community

555722.de/78

Call now! Only one test left!! Secure the parameter!

JUnit

pytest

Parameteriz.Testing

722-45173-78

€ fewer tests

€ test duplication

€ more values tested

free shipping (OSS)

Join the QAC
community

555722.de/78

Call now! Only one test left!! Secure the parameter!

JUnit

pytest

class FizzBuzzTest {

  @Test
  void fizzbuzz_1() {
    assertThat(fizzbuzz(1))
        .isEqualTo("1");
  }

  @Test
  void fizzbuzz_3() {
    assertThat(fizzbuzz(3))
        .isEqualTo("Fizz");
  }

  @Test
  void fizzbuzz_5() {
    assertThat(fizzbuzz(5))
        .isEqualTo("Buzz");
  }

  @Test
  void fizzbuzz_15() {
    assertThat(fizzbuzz(15))
        .isEqualTo("FizzBuzz");
  }
}
class FizzBuzzTest {

  @Test
  void fizzbuzz_1() {
    assertThat(fizzbuzz(1))
        .isEqualTo("1");
  }

  // …
}
class FizzBuzzTest {

  @Test
  void fizzbuzz_1() {
    assertThat(fizzbuzz(1))
        .isEqualTo("1");
  }

  @Test
  void fizzbuzz_2() {
    assertThat(fizzbuzz(2))
        .isEqualTo("2");
  }

  @Test
  void fizzbuzz_4() {
    assertThat(fizzbuzz(4))
        .isEqualTo("4");
  }

  @Test
  void fizzbuzz_7() {
    assertThat(fizzbuzz(7))
        .isEqualTo("7");
  }
  
  // …
}
class FizzBuzzTest {

  @ParameterizedTest(name = "fizzbuzz({0})")
  @ValueSource(ints = {1, 2, 4, 7, 8, 11, 13, 14})
  void fizzbuzz_not_divisible_by_3_or_5(int notBy3Or5) {
    assertThat(fizzbuzz(notBy3Or5))
        .isEqualTo(String.valueOf(notBy3Or5));
  }

  @Test
  void fizzbuzz_1() {
    assertThat(fizzbuzz(1))
        .isEqualTo("1");
  }

  @Test
  void fizzbuzz_2() {
    assertThat(fizzbuzz(2))
        .isEqualTo("2");
  }

  @Test
  void fizzbuzz_4() {
    assertThat(fizzbuzz(4))
        .isEqualTo("4");
  }

  @Test
  void fizzbuzz_7() {
    assertThat(fizzbuzz(7))
        .isEqualTo("7");
  }
  
  // …
}
class FizzBuzzTest {

  @ParameterizedTest(name = "fizzbuzz({0})")
  @ValueSource(ints = {1, 2, 4, 7, 8, 11, 13, 14})
  void fizzbuzz_not_divisible_by_3_or_5(int notBy3Or5) {
    assertThat(fizzbuzz(notBy3Or5))
        .isEqualTo(String.valueOf(notBy3Or5));
  }
  
  // …
}
class FizzBuzzTest {

  @ParameterizedTest(name = "fizzbuzz({0})")
  @ValueSource(ints = {1, 2, 4, 7, 8, 11, 13, 14})
  void fizzbuzz_not_divisible_by_3_or_5(int notBy3Or5) {
    assertThat(fizzbuzz(notBy3Or5))
        .isEqualTo(String.valueOf(notBy3Or5));
  }

  @ParameterizedTest(name = "fizzbuzz({0})")
  @ValueSource(ints = {3, 6, 9, 12, 18, 21, 24, 27})
  void fizzbuzz_divisible_by_3(int divisibleBy3) {
    assertThat(fizzbuzz(divisibleBy3))
        .isEqualTo("Fizz");
  }

  @ParameterizedTest(name = "fizzbuzz({0})")
  @ValueSource(ints = {5, 10, 20, 25, 35, 40, 50, 55})
  void fizzbuzz_divisible_by_5(int divisibleBy5) {
    assertThat(fizzbuzz(divisibleBy5))
        .isEqualTo("Buzz");
  }

  @ParameterizedTest(name = "fizzbuzz({0})")
  @ValueSource(ints = {15, 30, 45, 60, 75, 90, 105, 120})
  void fizzbuzz_divisible_by_3_and_5(int divisibleBy3And5) {
    assertThat(fizzbuzz(divisibleBy3And5))
        .isEqualTo("FizzBuzz");
  }
}

Value

  • Unit testing
  • Coding required
  • Often included in your test framework
  • Many libs available
  • Property-Based Testing
  • Unit Testing

Build

€ more values tested

722-45173-78

€ test duplication

€ fewer tests

Parameterized Testing

  • Less test code duplication
  • Better expression of intent
  • More variants tested

1.000.000s

Application

Buyers also liked…

Energy Label

Property-Based Testing

722-45173-728

€ refactor unit tests

€ find surprises late

€ one at a time

free shipping (OSS)

Join the QAC
community

555722.de/728

I propose to properly test properties

QuickCheck

jqwik, QuickTheories

JSVerify

Hypothesis

Rantly

FsCheck

Property-Based Testing

722-45173-728

€ refactor unit tests

€ find surprises late

€ one at a time

free shipping (OSS)

Join the QAC
community

555722.de/728

I propose to properly test properties

QuickCheck

jqwik, QuickTheories

JSVerify

Hypothesis

Rantly

FsCheck

class FizzBuzzTest {

  @ParameterizedTest(name = "fizzbuzz({0})")
  @ValueSource(ints = {1, 2, 4, 7, 8, 11, 13, 14})
  void fizzbuzz_not_divisible_by_3_or_5(int notBy3Or5) {
    assertThat(fizzbuzz(notBy3Or5))
        .isEqualTo(String.valueOf(notBy3Or5));
  }

  @ParameterizedTest(name = "fizzbuzz({0})")
  @ValueSource(ints = {3, 6, 9, 12, 18, 21, 24, 27})
  void fizzbuzz_divisible_by_3(int divisibleBy3) {
    assertThat(fizzbuzz(divisibleBy3))
        .isEqualTo("Fizz");
  }

  @ParameterizedTest(name = "fizzbuzz({0})")
  @ValueSource(ints = {5, 10, 20, 25, 35, 40, 50, 55})
  void fizzbuzz_divisible_by_5(int divisibleBy5) {
    assertThat(fizzbuzz(divisibleBy5))
        .isEqualTo("Buzz");
  }

  @ParameterizedTest(name = "fizzbuzz({0})")
  @ValueSource(ints = {15, 30, 45, 60, 75, 90, 105, 120})
  void fizzbuzz_divisible_by_3_and_5(int divisibleBy3And5) {
    assertThat(fizzbuzz(divisibleBy3And5))
        .isEqualTo("FizzBuzz");
  }
}

class FizzBuzzTest {

  // …

  @ParameterizedTest(name = "fizzbuzz({0})")
  @ValueSource(ints = {3, 6, 9, 12, 18, 21, 24, 27})
  void fizzbuzz_divisible_by_3(int divisibleBy3) {
    assertThat(fizzbuzz(divisibleBy3))
        .isEqualTo("Fizz");
  }

  // …
}

class FizzBuzzTest {

  // …
  
  @Property
  void fizzbuzz_divisible_by_3_property(@ForAll("divisibleBy3") int divisibleBy3) {
    assertThat(fizzbuzz(divisibleBy3))
        .isEqualTo("Fizz");
  }
  @Provide
  Arbitrary<Integer> divisibleBy3() {
    return Arbitraries
        .integers()
        .filter(i -> i % 3 == 0 && i % 5 != 0);
  }

  @ParameterizedTest(name = "fizzbuzz({0})")
  @ValueSource(ints = {3, 6, 9, 12, 18, 21, 24, 27})
  void fizzbuzz_divisible_by_3(int divisibleBy3) {
    assertThat(fizzbuzz(divisibleBy3))
        .isEqualTo("Fizz");
  }

  // …
}

class FizzBuzzTest {

  // …
  
  @Property
  void fizzbuzz_divisible_by_3(@ForAll("divisibleBy3") int divisibleBy3) {
    assertThat(fizzbuzz(divisibleBy3))
        .isEqualTo("Fizz");
  }
  @Provide
  Arbitrary<Integer> divisibleBy3() {
    return Arbitraries
        .integers()
        .filter(i -> i % 3 == 0 && i % 5 != 0);
  }

  // …
}

timestamp = 2024-07-16T09:50:25.850958, PropertyBasedTest:fizzbuzz divisible by 3 = 
                              |-----------------------jqwik-----------------------
tries = 1000                  | # of calls to property
checks = 1000                 | # of not rejected calls
generation = RANDOMIZED       | parameters are randomly generated
after-failure = SAMPLE_FIRST  | try previously failed sample, then previous seed
when-fixed-seed = ALLOW       | fixing the random seed is allowed
edge-cases#mode = MIXIN       | edge cases are mixed in
edge-cases#total = 1          | # of all combined edge cases
edge-cases#tried = 1          | # of edge cases tried in current run
seed = 7192800264771943696    | random seed to reproduce generated values
class FizzBuzzTest {

  // …

  @Property
  void fizzbuzz_divisible_by_3(@ForAll("divisibleBy3") int divisibleBy3) {
    assertThat(fizzbuzz(divisibleBy3))
        .isEqualTo("Fizz");
  }
  @Provide
  Arbitrary<Integer> divisibleBy3() {
    return Arbitraries
        .integers()
        .filter(i -> i % 3 == 0 && i % 5 != 0);
  }

  @Property
  void fizzbuzz_divisible_by_5(@ForAll("divisibleBy5") int divisibleBy5) {
    assertThat(fizzbuzz(divisibleBy5))
        .isEqualTo("Buzz");
  }
  @Provide
  Arbitrary<Integer> divisibleBy5() {
    return Arbitraries
        .integers()
        .filter(i -> i % 3 != 0 && i % 5 == 0);
  }
  
  // …
}

€ one at a time

722-45173-728

€ find surprises late

€ refactor unit tests

Property-Based Testing

Value

  • Unit testing
  • Somewhat advanced coding skills required
  • Many libs available
  • Parameterized Testing
  • Unit Testing

Build

  • Less test code duplication
  • Better expression of intent
  • More variants tested
  • …

Application

Buyers also liked…

Energy Label

1.000.000s

Approval Testing

722-45173-28

€ one verification

€ long assertions

€ see all differences

free shipping (OSS)

Join the QAC
community

555722.de/28

I approve this tooling!

Approval Tests for Node.js

… ApprovalTests

Approval Testing

722-45173-28

€ one verification

€ long assertions

€ see all differences

free shipping (OSS)

Join the QAC
community

555722.de/28

I approve this tooling!

Approval Tests for Node.js

… ApprovalTests

record Address(
    String id,
    String firstName,
    String lastName,
    String streetName,
    String houseNumber,
    String city,
    String country,
    String phone,
    String latitude,
    String longitude,
    String email,
    String postalCode
) {}

record Order(
    String id,
    int version,
    List<Item> items,
    List<Coupon> coupons,
    LocalDateTime orderTimeStamp,
    LocalDate deliveryDate,
    List<Price> shippingCost,
    Customer customer,
    Address shippingAddress,
    Address billingAddress
) {}

record Item(
    String id,
    String name,
    int amount,
    Price price
) {}

record Coupon(
    String id,
    String description,
    int reducedRateInPercentage
) {}

record Price(
    int value,
    String monetaryUnit,
    String currency
) {}

record Customer(
    String id,
    String firstName,
    String lastName
) {}

class AssertionTest {

  @Test
  void assertionTest() throws JsonProcessingException {
    String orderId = "someOrderId";
    ShopOrder order = aDefaultOrder(orderId);

    anOrderWasProcessed(order);

    // then
    JsonNode result = jsonMapper.readTree(callRestEndpoint(orderId));

    assertThat(result.get("id").asText()).isEqualTo("someOrderId");
    assertThat(result.get("version").asText()).isEqualTo("1");

    JsonNode item = result.get("items").get(0);
    assertThat(item.get("id").asText()).isEqualTo("someItemId");
    assertThat(item.get("name").asText()).isEqualTo("ATD 3 Conf. Days");
    assertThat(item.get("amount").asText()).isEqualTo("2");

    JsonNode itemPrice = item.get("price");
    assertThat(itemPrice.get("value").asText()).isEqualTo("225000");
    assertThat(itemPrice.get("monetaryUnit").asText()).isEqualTo("cent");
    assertThat(itemPrice.get("currency").asText()).isEqualTo("EUR");

    JsonNode coupon = result.get("coupons").get(0);
    assertThat(coupon.get("id").asText()).isEqualTo("someCouponId");
    assertThat(coupon.get("description").asText()).isEqualTo("Speaker Coupon");
    assertThat(coupon.get("reducedRateInPercentage").asText()).isEqualTo("100");

    JsonNode orderTimeStamp = result.get("orderTimeStamp");
    assertThat(orderTimeStamp.get(0).asInt()).isEqualTo(2024);
    assertThat(orderTimeStamp.get(1).asInt()).isEqualTo(7);
    assertThat(orderTimeStamp.get(2).asInt()).isEqualTo(19);
    assertThat(orderTimeStamp.get(3).asInt()).isEqualTo(11);
    assertThat(orderTimeStamp.get(4).asInt()).isEqualTo(45);

    JsonNode deliveryDate = result.get("deliveryDate");
    assertThat(deliveryDate.get(0).asInt()).isEqualTo(2024);
    assertThat(deliveryDate.get(1).asInt()).isEqualTo(11);
    assertThat(deliveryDate.get(2).asInt()).isEqualTo(22);

    JsonNode shippingCost = result.get("shippingCost").get(0);
    assertThat(shippingCost.get("value").asText()).isEqualTo("500");
    assertThat(shippingCost.get("monetaryUnit").asText()).isEqualTo("cent");
    assertThat(shippingCost.get("currency").asText()).isEqualTo("EUR");

    JsonNode customer = result.get("customer");
    assertThat(customer.get("id").asText()).isEqualTo("someCustomerId");
    assertThat(customer.get("firstName").asText()).isEqualTo("REWE");
    assertThat(customer.get("lastName").asText()).isEqualTo("Digital");

    JsonNode shippingAddress = result.get("shippingAddress");
    assertThat(shippingAddress.get("id").asText()).isEqualTo("someShippingAddressId");
    assertThat(shippingAddress.get("firstName").asText()).isEqualTo("Janina");
    assertThat(shippingAddress.get("lastName").asText()).isEqualTo("Nemec");
    assertThat(shippingAddress.get("streetName").asText()).isEqualTo("Schanzenstr.");
    assertThat(shippingAddress.get("houseNumber").asText()).isEqualTo("6-20");
    assertThat(shippingAddress.get("postalCode").asText()).isEqualTo("51063");
    assertThat(shippingAddress.get("city").asText()).isEqualTo("Köln");
    assertThat(shippingAddress.get("country").asText()).isEqualTo("Deutschland");
    assertThat(shippingAddress.get("phone").asText()).isEqualTo("0221 9758420");
    assertThat(shippingAddress.get("latitude").asText()).isEqualTo("50.96490882194811");
    assertThat(shippingAddress.get("longitude").asText()).isEqualTo("7.014472855463499");
    assertThat(shippingAddress.get("email").asText()).isEqualTo("kontakt@rewe-digital.com");

    JsonNode billingAddress = result.get("billingAddress");
    assertThat(billingAddress.get("id").asText()).isEqualTo("someBillingAddressId");
    assertThat(billingAddress.get("firstName").asText()).isEqualTo("Micha");
    assertThat(billingAddress.get("lastName").asText()).isEqualTo("Kutz");
    assertThat(billingAddress.get("streetName").asText()).isEqualTo("Domstr.");
    assertThat(billingAddress.get("houseNumber").asText()).isEqualTo("20");
    assertThat(billingAddress.get("postalCode").asText()).isEqualTo("50668");
    assertThat(billingAddress.get("city").asText()).isEqualTo("Köln");
    assertThat(billingAddress.get("country").asText()).isEqualTo("Deutschland");
    assertThat(billingAddress.get("phone").asText()).isEqualTo("+49 221 1490");
    assertThat(billingAddress.get("latitude").asText()).isEqualTo("50.94603935915518");
    assertThat(billingAddress.get("longitude").asText()).isEqualTo("6.959302840118697");
    assertThat(billingAddress.get("email").asText()).isEqualTo("info@rewe-group.com");
  }
}


record Address(
    String id,
    String firstName,
    String lastName,
    String streetName,
    String houseNumber,
    String city,
    String country,
    String phone,
    String latitude,
    String longitude,
    String email,
    String postalCode
) {}

record Order(
    String id,
    int version,
    List<Item> items,
    List<Coupon> coupons,
    LocalDateTime orderTimeStamp,
    LocalDate deliveryDate,
    List<Price> shippingCost,
    Customer customer,
    Address shippingAddress,
    Address billingAddress
) {}

record Item(
    String id,
    String name,
    int amount,
    Price price
) {}

record Coupon(
    String id,
    String description,
    int reducedRateInPercentage
) {}

record Price(
    int value,
    String monetaryUnit,
    String currency
) {}

record Customer(
    String id,
    String firstName,
    String lastName
) {}

class AddressAssertionTest {
    @Test
    void assertionTest() throws JsonProcessingException {
        String orderId = "someOrderId";
        ShopOrder shopOrder = anyOrder(orderId)
                .billingAddress(anAddress()
                        .id("someBillingAddressId")
                        .firstName("Micha").lastName("Kutz")
                        .streetName("Domstr.").houseNumber("20")
                        .postalCode("50668").city("Köln").country("Deutschland")
                        .phone("+49 221 1490").email("info@rewe-group.com").build()
                ).build();

        anOrderWasProcessed(shopOrder);

        JsonNode result = jsonMapper.readTree(callRestEndpointForBillingAddress(orderId));
        assertThat(result.get("id").asText()).isEqualTo("someBillingAddressId");
        assertThat(result.get("firstName").asText()).isEqualTo("Micha");
        assertThat(result.get("lastName").asText()).isEqualTo("Kutz");
        assertThat(result.get("streetName").asText()).isEqualTo("Domstr.");
        assertThat(result.get("houseNumber").asText()).isEqualTo("20");
        assertThat(result.get("postalCode").asText()).isEqualTo("50668");
        assertThat(result.get("city").asText()).isEqualTo("Köln");
        assertThat(result.get("country").asText()).isEqualTo("Deutschland");
        assertThat(result.get("phone").asText()).isEqualTo("+49 221 1490");
        assertThat(result.get("latitude").asText()).isEqualTo("50.94603935915518");
        assertThat(result.get("longitude").asText()).isEqualTo("6.959302840118697");
        assertThat(result.get("email").asText()).isEqualTo("info@rewe-group.com");
    }
}
class AddressAssertionTest {
    @Test
    void assertionTest() throws JsonProcessingException {
        String orderId = "someOrderId";
        ShopOrder shopOrder = anyOrder(orderId)
                .billingAddress(anAddress()
                        .id("someBillingAddressId")
                        .firstName("Micha").lastName("Kutz")
                        .streetName("Domstr.").houseNumber("20")
                        .postalCode("50668").city("Köln").country("Deutschland")
                        .phone("+49 221 1490").email("info@rewe-group.com").build()
                ).build();

        anOrderWasProcessed(shopOrder);

        JsonNode result = jsonMapper.readTree(callRestEndpointForBillingAddress(orderId));
        assertThat(result.get("id").asText()).isEqualTo("someBillingAddressId");
        assertThat(result.get("firstName").asText()).isEqualTo("Micha");
        assertThat(result.get("lastName").asText()).isEqualTo("Kutz");
        assertThat(result.get("streetName").asText()).isEqualTo("Domstr.");
        assertThat(result.get("houseNumber").asText()).isEqualTo("20");
        assertThat(result.get("postalCode").asText()).isEqualTo("50668");
        assertThat(result.get("city").asText()).isEqualTo("Köln");
        assertThat(result.get("country").asText()).isEqualTo("Deutschland");
        assertThat(result.get("phone").asText()).isEqualTo("+49 221 1490");
        assertThat(result.get("email").asText()).isEqualTo("info@rewe-group.com");
    }
}
{
  "id": "someBillingAddressId",
  "firstName": "Micha",
  "lastName": "Kutz",
  "streetName": "Domstr.",
  "houseNumber": "20",
  "city": "Köln",
  "country": "Deutschland",
  "phone": "+49 221 1490",
  "latitude": "50.94603935915518",
  "longitude": "6.959302840118697",
  "email": "info@rewe-group.com",
  "postalCode": "50668"
}


{
  "id": "someBillingAddressId",
  "firstName": "Micha",
  "lastName": "Kutz",
  "streetName": "Domstr.",
  "houseNumber": "20",
  "city": "Köln",
  "country": "Deutschland",
  "phone": "+49 221 1490",
  "email": "info@rewe-group.com",
  "postalCode": "50668"
}


class AddressAssertionTest {
    @Test
    void assertionTest() throws JsonProcessingException {
        String orderId = "someOrderId";
        ShopOrder shopOrder = anyOrder(orderId)
                .billingAddress(anAddress()
                        .id("someBillingAddressId")
                        .firstName("Micha").lastName("Kutz")
                        .streetName("Domstr.").houseNumber("20")
                        .postalCode("50668").city("Köln").country("Deutschland")
                        .phone("+49 221 1490").email("info@rewe-group.com").build()
                ).build();

        anOrderWasProcessed(shopOrder);

        JsonNode result = jsonMapper.readTree(callRestEndpointForBillingAddress(orderId));
        assertThat(result.get("id").asText()).isEqualTo("someBillingAddressId");
        assertThat(result.get("firstName").asText()).isEqualTo("Micha");
        assertThat(result.get("lastName").asText()).isEqualTo("Kutz");
        assertThat(result.get("streetName").asText()).isEqualTo("Domstr.");
        assertThat(result.get("houseNumber").asText()).isEqualTo("20");
        assertThat(result.get("postalCode").asText()).isEqualTo("50668");
        assertThat(result.get("city").asText()).isEqualTo("Köln");
        assertThat(result.get("country").asText()).isEqualTo("Deutschland");
        assertThat(result.get("phone").asText()).isEqualTo("+49 221 1490");
        assertThat(result.get("latitude").asText()).isEqualTo("50.94603935915518");
        assertThat(result.get("longitude").asText()).isEqualTo("6.959302840118697");
        assertThat(result.get("email").asText()).isEqualTo("info@rewe-group.com");
    }
}
class AddressAssertionTest {
    @Test
    void assertionTest() throws JsonProcessingException {
        String orderId = "someOrderId";
        ShopOrder shopOrder = anyOrder(orderId)
                .billingAddress(anAddress()
                        .id("someBillingAddressId")
                        .firstName("Micha").lastName("Kutz")
                        .streetName("Domstr.").houseNumber("20")
                        .postalCode("50668").city("Köln").country("Deutschland")
                        .phone("+49 221 1490").email("info@rewe-group.com").build()
                ).build();

        anOrderWasProcessed(shopOrder);

        JsonApprovals.verifyJson(callRestEndpointForBillingAddress(orderId));
    }
}

722-45173-28

€ see all differences

€ one verification

Approval Testing

€ long assertions

Value

  • Unit testing, integration testing, (GUI) testing
  • A little coding skills required
  • Many libs available
  • Exploratory Testing

Build

  • More concise result verification
  • More complete result verification

Application

Buyers also liked…

Energy Label

1.000.000s

Mutation Testing

722-45173-68

€ create mutants

€ untested code

€ find missing cases

free shipping (OSS)

Join the QAC
community

555722.de/68

Teenage Mutatant Hero Testing

PIT

Mutant

Stryker Mutator

Mutatest

class FizzBuzzTest {

  @Test
  void fizzbuzz_1() {
    assertThat(fizzbuzz(1)).isEqualTo("1");
  }

  @Test
  void fizzbuzz_3() {
    assertThat(fizzbuzz(3)).isEqualTo("Fizz");
  }

  @Test
  void fizzbuzz_5() {
    assertThat(fizzbuzz(5)).isEqualTo("Buzz");
  }
}
$ ./gradlew pitest

class FizzBuzzTest {

  @Test
  void fizzbuzz_1() {
    assertThat(fizzbuzz(1)).isEqualTo("1");
  }

  @Test
  void fizzbuzz_3() {
    assertThat(fizzbuzz(3)).isEqualTo("Fizz");
  }

  @Test
  void fizzbuzz_5() {
    assertThat(fizzbuzz(5)).isEqualTo("Buzz");
  }
}
$ ./gradlew pitest
…
================================================================================
- Statistics
================================================================================
>> Line Coverage (for mutated classes only): 5/5 (100%)
>> Generated 10 mutations Killed 8 (80%)
>> Mutations with no coverage 1. Test strength 89%
>> Ran 11 tests (1.1 tests per mutation)
class FizzBuzzTest {

  @Test
  void fizzbuzz_1() {
    assertThat(fizzbuzz(1)).isEqualTo("1");
  }

  @Test
  void fizzbuzz_3() {
    assertThat(fizzbuzz(3)).isEqualTo("Fizz");
  }

  @Test
  void fizzbuzz_5() {
    assertThat(fizzbuzz(5)).isEqualTo("Buzz");
  }
}
class FizzBuzzTest {

  @Test
  void fizzbuzz_1() {
    assertThat(fizzbuzz(1)).isEqualTo("1");
  }

  @Test
  void fizzbuzz_3() {
    assertThat(fizzbuzz(3)).isEqualTo("Fizz");
  }

  @Test
  void fizzbuzz_5() {
    assertThat(fizzbuzz(5)).isEqualTo("Buzz");
  }
 
  @Test
  void fizzbuzz_15() {
    assertThat(fizzbuzz(15)).isNotNull();
  }
}
$ ./gradlew pitest
…
================================================================================
- Statistics
================================================================================
>> Line Coverage (for mutated classes only): 4/5 (80%)
>> Generated 10 mutations Killed 8 (80%)
>> Mutations with no coverage 0. Test strength 80%
>> Ran 13 tests (1.3 tests per mutation)
class FizzBuzzTest {

  @Test
  void fizzbuzz_1() {
    assertThat(fizzbuzz(1)).isEqualTo("1");
  }

  @Test
  void fizzbuzz_3() {
    assertThat(fizzbuzz(3)).isEqualTo("Fizz");
  }

  @Test
  void fizzbuzz_5() {
    assertThat(fizzbuzz(5)).isEqualTo("Buzz");
  }
 
  @Test
  void fizzbuzz_15() {
    assertThat(fizzbuzz(15)).isNotNull();
  }
}
class FizzBuzzTest {

  @Test
  void fizzbuzz_1() {
    assertThat(fizzbuzz(1)).isEqualTo("1");
  }

  @Test
  void fizzbuzz_3() {
    assertThat(fizzbuzz(3)).isEqualTo("Fizz");
  }

  @Test
  void fizzbuzz_5() {
    assertThat(fizzbuzz(5)).isEqualTo("Buzz");
  }
 
  @Test
  void fizzbuzz_15() {
    assertThat(fizzbuzz(15)).isEqualTo("FizzBuzz");
  }
}
$ ./gradlew pitest
…
================================================================================
- Statistics
================================================================================
>> Line Coverage (for mutated classes only): 4/5 (80%)
>> Generated 10 mutations Killed 10 (100%)
>> Mutations with no coverage 0. Test strength 100%
>> Ran 11 tests (1.1 tests per mutation)

Value

  • Unit testing
  • Tooling can be tricky
  • Static code analysis

Build

  • Detect holes in your test coverage
  • Much better insight compared to code coverage

Application

Buyers also liked…

Energy Label

10× less

€ find missing cases

722-45173-68

€ untested code

€ create mutants

Mutation Testing

Ensemble Testing

722-45173-38

€ early feedback

€ works on my machine

€ learn & improve

free shipping (OSS)

Join the QAC
community

555722.de/38

Let's test together!!

"All the brilliant people working on the same thing, at the same time, in the same space, and on the same computer"

- Woody Zuill

Ensemble Testing

Why do ensemble testing?

Plan

Understand

Value

  • Great (free) remote tooling available
  • Office setup also possible (hardware separately available)
  • Please adhere to the manual to avoid known side effects
  • Can be applied to manual testing and automation
  • Team programming
  • Exploratory testing

Build

  • Get everyone on the team involved in testing
  • Learn from each other
  • More immediate feedback & problem solving

Application

Buyers also liked…

Energy Label

More insights

€ learn & improve

722-45173-38

€ works on my machine  

€ early feedback

Ensemble Testing

Feature Toggles

722-45173-387

€ canary testing

€ everyone suffers

€ ab-testing

free shipping (OSS)

Join the QAC
community

555722.de/38

Throw in on production like it's 1999 :)

How is it related to testing?

Testing in production

Canary Testing

A/B-Testing

Plan

Understand

Value

  • Great (free) remote tooling available
  • ⚠️ Important to coordinate toggles across the product
  • 🧹 Cleaning required: remove toggles after release!
  • Continuous delivery & deployment (CD)
  • Monitoring & alerting
  • Service/Site Reliability Engineering (SRE)

Build

  • Deployment ≠ Release ⇒ more continuous work
  • Seamless and coordinated rollbacks possible

Application

Buyers also liked…

Energy Label

More insights

722-45173-387

€ a/b testing

€ delayed releases

€ canary releases

Feature Toggles

€ more values tested

722-45173-78

€ test duplication

€ fewer tests

Parameterized Testing

€ one at a time

722-45173-728

€ find surprises late

€ refactor unit tests

Property-Based Testing

€ see all differences

€ one verification

722-45173-28

Approval Testing

€ long assertions

€ one at a time

722-45173-728

€ find surprises late

€ refactor unit tests

Property-Based Testing

€ find missing cases

722-45173-68

€ untested code

€ create mutants

Mutation Testing

€ learn & improve

722-45173-38

€ works on my machine  

€ early feedback

Ensemble Testing

722-45173-387

€ a/b testing

€ delayed releases

€ canary releases

Feature Toggles