Level-up Your Testing Toolkit!

Janina

Micha

I am…

👨‍💻 a Software Developer/
    Software Crafting enthusiast

👧🏼 a father to a daughter

🥦 a vegetarian


I like…

🏄‍♂️ my flow to be continuous,

🙅‍♂️ my stress minimized ⇒ QA test automation

🏃‍♂️ running

🇫🇷 learning French

🍺 drinking Craft Beer

I am…

👩‍💻 a Software Developer/Crafter

💡 an Ambassador for learning

     a co-organizer for a security conference 


I like…

💡Learning & Improving ⇒ Software Teaming

🔄 Automatization

🎨 Drawing

👩‍🌾 Gardening

🎮 Video Games

🐶 Dogs 💜

Parameteriz.Testing

722-45173-78

€ fewer tests

€ test duplication

€ more values tested

free shipping (OSS)

Join the QAC
community

555722.de/78

Call now! Only one test left!! Secure the parameter!

JUnit

pytest

class FizzBuzzTest {

  @Test
  void fizzbuzz_1() {
    assertThat(fizzbuzz(1))
        .isEqualTo("1");
  }

  @Test
  void fizzbuzz_3() {
    assertThat(fizzbuzz(3))
        .isEqualTo("Fizz");
  }

  @Test
  void fizzbuzz_5() {
    assertThat(fizzbuzz(5))
        .isEqualTo("Buzz");
  }

  @Test
  void fizzbuzz_15() {
    assertThat(fizzbuzz(15))
        .isEqualTo("FizzBuzz");
  }
}
class FizzBuzzTest {

  @Test
  void fizzbuzz_1() {
    assertThat(fizzbuzz(1))
        .isEqualTo("1");
  }

  // …
}
class FizzBuzzTest {

  @Test
  void fizzbuzz_1() {
    assertThat(fizzbuzz(1))
        .isEqualTo("1");
  }

  @Test
  void fizzbuzz_2() {
    assertThat(fizzbuzz(2))
        .isEqualTo("2");
  }

  @Test
  void fizzbuzz_4() {
    assertThat(fizzbuzz(4))
        .isEqualTo("4");
  }

  @Test
  void fizzbuzz_7() {
    assertThat(fizzbuzz(7))
        .isEqualTo("7");
  }
  
  // …
}
class FizzBuzzTest {

  @ParameterizedTest(name = "fizzbuzz({0})")
  @ValueSource(ints = {1, 2, 4, 7, 8, 11, 13, 14})
  void fizzbuzz_not_divisible_by_3_or_5(int notBy3Or5) {
    assertThat(fizzbuzz(notBy3Or5))
        .isEqualTo(String.valueOf(notBy3Or5));
  }

  @Test
  void fizzbuzz_1() {
    assertThat(fizzbuzz(1))
        .isEqualTo("1");
  }

  @Test
  void fizzbuzz_2() {
    assertThat(fizzbuzz(2))
        .isEqualTo("2");
  }

  @Test
  void fizzbuzz_4() {
    assertThat(fizzbuzz(4))
        .isEqualTo("4");
  }

  @Test
  void fizzbuzz_7() {
    assertThat(fizzbuzz(7))
        .isEqualTo("7");
  }
  
  // …
}
class FizzBuzzTest {

  @ParameterizedTest(name = "fizzbuzz({0})")
  @ValueSource(ints = {1, 2, 4, 7, 8, 11, 13, 14})
  void fizzbuzz_not_divisible_by_3_or_5(int notBy3Or5) {
    assertThat(fizzbuzz(notBy3Or5))
        .isEqualTo(String.valueOf(notBy3Or5));
  }
  
  // …
}
class FizzBuzzTest {

  @ParameterizedTest(name = "fizzbuzz({0})")
  @ValueSource(ints = {1, 2, 4, 7, 8, 11, 13, 14})
  void fizzbuzz_not_divisible_by_3_or_5(int notBy3Or5) {
    assertThat(fizzbuzz(notBy3Or5))
        .isEqualTo(String.valueOf(notBy3Or5));
  }

  @ParameterizedTest(name = "fizzbuzz({0})")
  @ValueSource(ints = {3, 6, 9, 12, 18, 21, 24, 27})
  void fizzbuzz_divisible_by_3(int divisibleBy3) {
    assertThat(fizzbuzz(divisibleBy3))
        .isEqualTo("Fizz");
  }

  @ParameterizedTest(name = "fizzbuzz({0})")
  @ValueSource(ints = {5, 10, 20, 25, 35, 40, 50, 55})
  void fizzbuzz_divisible_by_5(int divisibleBy5) {
    assertThat(fizzbuzz(divisibleBy5))
        .isEqualTo("Buzz");
  }

  @ParameterizedTest(name = "fizzbuzz({0})")
  @ValueSource(ints = {15, 30, 45, 60, 75, 90, 105, 120})
  void fizzbuzz_divisible_by_3_and_5(int divisibleBy3And5) {
    assertThat(fizzbuzz(divisibleBy3And5))
        .isEqualTo("FizzBuzz");
  }
}

Value

  • Unit testing
  • Coding required
  • Often included in your test framework
  • Many libs available
  • Property-Based Testing
  • Unit Testing

Build

€ more values tested

722-45173-78

€ test duplication

€ fewer tests

Parameterized Testing

  • Less test code duplication
  • Better expression of intent
  • More variants tested

1.000.000s

Application

Buyers also liked…

Energy Label

Property-Based Testing

722-45173-728

€ refactor unit tests

€ find surprises late

€ one at a time

free shipping (OSS)

Join the QAC
community

555722.de/728

I propose to properly test properties

QuickCheck

jqwik, QuickTheories

JSVerify

Hypothesis

Rantly

FsCheck

class FizzBuzzTest {

  @ParameterizedTest(name = "fizzbuzz({0})")
  @ValueSource(ints = {1, 2, 4, 7, 8, 11, 13, 14})
  void fizzbuzz_not_divisible_by_3_or_5(int notBy3Or5) {
    assertThat(fizzbuzz(notBy3Or5))
        .isEqualTo(String.valueOf(notBy3Or5));
  }

  @ParameterizedTest(name = "fizzbuzz({0})")
  @ValueSource(ints = {3, 6, 9, 12, 18, 21, 24, 27})
  void fizzbuzz_divisible_by_3(int divisibleBy3) {
    assertThat(fizzbuzz(divisibleBy3))
        .isEqualTo("Fizz");
  }

  @ParameterizedTest(name = "fizzbuzz({0})")
  @ValueSource(ints = {5, 10, 20, 25, 35, 40, 50, 55})
  void fizzbuzz_divisible_by_5(int divisibleBy5) {
    assertThat(fizzbuzz(divisibleBy5))
        .isEqualTo("Buzz");
  }

  @ParameterizedTest(name = "fizzbuzz({0})")
  @ValueSource(ints = {15, 30, 45, 60, 75, 90, 105, 120})
  void fizzbuzz_divisible_by_3_and_5(int divisibleBy3And5) {
    assertThat(fizzbuzz(divisibleBy3And5))
        .isEqualTo("FizzBuzz");
  }
}

class FizzBuzzTest {

  // …

  @ParameterizedTest(name = "fizzbuzz({0})")
  @ValueSource(ints = {3, 6, 9, 12, 18, 21, 24, 27})
  void fizzbuzz_divisible_by_3(int divisibleBy3) {
    assertThat(fizzbuzz(divisibleBy3))
        .isEqualTo("Fizz");
  }

  // …
}

class FizzBuzzTest {

  // …
  
  @Property
  void fizzbuzz_divisible_by_3_property(@ForAll("divisibleBy3") int divisibleBy3) {
    assertThat(fizzbuzz(divisibleBy3))
        .isEqualTo("Fizz");
  }
  @Provide
  Arbitrary<Integer> divisibleBy3() {
    return Arbitraries
        .integers()
        .filter(i -> i % 3 == 0 && i % 5 != 0);
  }

  @ParameterizedTest(name = "fizzbuzz({0})")
  @ValueSource(ints = {3, 6, 9, 12, 18, 21, 24, 27})
  void fizzbuzz_divisible_by_3(int divisibleBy3) {
    assertThat(fizzbuzz(divisibleBy3))
        .isEqualTo("Fizz");
  }

  // …
}

class FizzBuzzTest {

  // …
  
  @Property
  void fizzbuzz_divisible_by_3(@ForAll("divisibleBy3") int divisibleBy3) {
    assertThat(fizzbuzz(divisibleBy3))
        .isEqualTo("Fizz");
  }
  @Provide
  Arbitrary<Integer> divisibleBy3() {
    return Arbitraries
        .integers()
        .filter(i -> i % 3 == 0 && i % 5 != 0);
  }

  // …
}

timestamp = 2024-07-16T09:50:25.850958, PropertyBasedTest:fizzbuzz divisible by 3 = 
                              |-----------------------jqwik-----------------------
tries = 1000                  | # of calls to property
checks = 1000                 | # of not rejected calls
generation = RANDOMIZED       | parameters are randomly generated
after-failure = SAMPLE_FIRST  | try previously failed sample, then previous seed
when-fixed-seed = ALLOW       | fixing the random seed is allowed
edge-cases#mode = MIXIN       | edge cases are mixed in
edge-cases#total = 1          | # of all combined edge cases
edge-cases#tried = 1          | # of edge cases tried in current run
seed = 7192800264771943696    | random seed to reproduce generated values
class FizzBuzzTest {

  // …

  @Property
  void fizzbuzz_divisible_by_3(@ForAll("divisibleBy3") int divisibleBy3) {
    assertThat(fizzbuzz(divisibleBy3))
        .isEqualTo("Fizz");
  }
  @Provide
  Arbitrary<Integer> divisibleBy3() {
    return Arbitraries
        .integers()
        .filter(i -> i % 3 == 0 && i % 5 != 0);
  }

  @Property
  void fizzbuzz_divisible_by_5(@ForAll("divisibleBy5") int divisibleBy5) {
    assertThat(fizzbuzz(divisibleBy5))
        .isEqualTo("Buzz");
  }
  @Provide
  Arbitrary<Integer> divisibleBy5() {
    return Arbitraries
        .integers()
        .filter(i -> i % 3 != 0 && i % 5 == 0);
  }
  
  // …
}

€ one at a time

722-45173-728

€ find surprises late

€ refactor unit tests

Property-Based Testing

Value

  • Unit testing
  • Somewhat advanced coding skills required
  • Many libs available
  • Parameterized Testing
  • Unit Testing

Build

  • Less test code duplication
  • Better expression of intent
  • More variants tested
  • …

Application

Buyers also liked…

Energy Label

1.000.000s

Approval Testing

722-45173-28

€ one verification

€ long assertions

€ see all differences

free shipping (OSS)

Join the QAC
community

555722.de/28

I approve this tooling!

Approval Tests for Node.js

… ApprovalTests

record Address(
    String id,
    String firstName,
    String lastName,
    String streetName,
    String houseNumber,
    String city,
    String country,
    String phone,
    String latitude,
    String longitude,
    String email,
    String postalCode
) {}

record Order(
    String id,
    int version,
    List<Item> items,
    List<Coupon> coupons,
    LocalDateTime orderTimeStamp,
    LocalDate deliveryDate,
    List<Price> shippingCost,
    Customer customer,
    Address shippingAddress,
    Address billingAddress
) {}

record Item(
    String id,
    String name,
    int amount,
    Price price
) {}

record Coupon(
    String id,
    String description,
    int reducedRateInPercentage
) {}

record Price(
    int value,
    String monetaryUnit,
    String currency
) {}

record Customer(
    String id,
    String firstName,
    String lastName
) {}

class ShopOrderTest {

  @Test
  void assertionTest() throws JsonProcessingException {
  	// given
    String orderId = "someOrderId";
    ShopOrder order = aDefaultOrder(orderId);
        
    // when
    anOrderWasProcessed(order);
	
    // then
    OrderResult orderResult = jsonMapper.readValue(callRestEndpoint(orderId), OrderResult.class);

    assertThat(orderResult.getId()).isEqualTo("someOrderId");
    assertThat(orderResult.getVersion()).isEqualTo(1);

    ItemResult item = orderResult.getItems().getFirst();
    assertThat(item.getId()).isEqualTo("someItemId");
    assertThat(item.getName()).isEqualTo("ATD 3 Conf. Days");
    assertThat(item.getAmount()).isEqualTo(2);

    PriceResult itemPrice = item.getPrice();
    assertThat(itemPrice.getValue()).isEqualTo(225000);
    assertThat(itemPrice.getMonetaryUnit()).isEqualTo("cent");
    assertThat(itemPrice.getCurrency()).isEqualTo("EUR");

    CouponResult coupon = orderResult.getCoupons().getFirst();
    assertThat(coupon.getId()).isEqualTo("someCouponId");
    assertThat(coupon.getDescription()).isEqualTo("Speaker Coupon");
    assertThat(coupon.getReducedRateInPercentage()).isEqualTo(100);

    assertThat(orderResult.getOrderTimeStamp()).isEqualTo(LocalDateTime.of(2024, 7, 19, 11, 45));
    assertThat(orderResult.getDeliveryDate()).isEqualTo(LocalDate.of(2024, 11, 22));

    PriceResult shippingCost = orderResult.getShippingCost().getFirst();
    assertThat(shippingCost.getValue()).isEqualTo(500);
    assertThat(shippingCost.getMonetaryUnit()).isEqualTo("cent");
    assertThat(shippingCost.getCurrency()).isEqualTo("EUR");

    CustomerResult customer = orderResult.getCustomer();
    assertThat(customer.getId()).isEqualTo("someCustomerId");
    assertThat(customer.getFirstName()).isEqualTo("REWE");
    assertThat(customer.getLastName()).isEqualTo("Digital");

    AddressResult shippingAddress = orderResult.getShippingAddress();
    assertThat(shippingAddress.getId()).isEqualTo("someShippingAddressId");
    assertThat(shippingAddress.getFirstName()).isEqualTo("Janina");
    assertThat(shippingAddress.getLastName()).isEqualTo("Nemec");
    assertThat(shippingAddress.getStreetName()).isEqualTo("Schanzenstr.");
    assertThat(shippingAddress.getHouseNumber()).isEqualTo("6-20");
    assertThat(shippingAddress.getPostalCode()).isEqualTo("51063");
    assertThat(shippingAddress.getCity()).isEqualTo("Köln");
    assertThat(shippingAddress.getCountry()).isEqualTo("Deutschland");
    assertThat(shippingAddress.getPhone()).isEqualTo("0221 9758420");
    assertThat(shippingAddress.getLatitude()).isEqualTo("50.96490882194811");
    assertThat(shippingAddress.getLongitude()).isEqualTo("7.014472855463499");
    assertThat(shippingAddress.getEmail()).isEqualTo("kontakt@rewe-digital.com");

    AddressResult billingAddress = orderResult.getBillingAddress();
    assertThat(billingAddress.getId()).isEqualTo("someBillingAddressId");
    assertThat(billingAddress.getFirstName()).isEqualTo("Micha");
    assertThat(billingAddress.getLastName()).isEqualTo("Kutz");
    assertThat(billingAddress.getStreetName()).isEqualTo("Domstr.");
    assertThat(billingAddress.getHouseNumber()).isEqualTo("20");
    assertThat(billingAddress.getPostalCode()).isEqualTo("50668");
    assertThat(billingAddress.getCity()).isEqualTo("Köln");
    assertThat(billingAddress.getCountry()).isEqualTo("Deutschland");
    assertThat(billingAddress.getPhone()).isEqualTo("+49 221 1490");
    assertThat(billingAddress.getLatitude()).isEqualTo("50.94603935915518");
    assertThat(billingAddress.getLongitude()).isEqualTo("6.959302840118697");
    assertThat(billingAddress.getEmail()).isEqualTo("info@rewe-group.com");
  }
}

record Address(
    String id,
    String firstName,
    String lastName,
    String streetName,
    String houseNumber,
    String city,
    String country,
    String phone,
    String latitude,
    String longitude,
    String email,
    String postalCode
) {}

record Order(
    String id,
    int version,
    List<Item> items,
    List<Coupon> coupons,
    LocalDateTime orderTimeStamp,
    LocalDate deliveryDate,
    List<Price> shippingCost,
    Customer customer,
    Address shippingAddress,
    Address billingAddress
) {}

record Item(
    String id,
    String name,
    int amount,
    Price price
) {}

record Coupon(
    String id,
    String description,
    int reducedRateInPercentage
) {}

record Price(
    int value,
    String monetaryUnit,
    String currency
) {}

record Customer(
    String id,
    String firstName,
    String lastName
) {}

record Address(
    String id,
    String firstName,
    String lastName,
    String streetName,
    String houseNumber,
    String city,
    String country,
    String phone,
    String latitude,
    String longitude,
    String email,
    String postalCode
) {}

class AddressAssertionTest {

  @Test
  void assertionTest() throws JsonProcessingException {
    String orderId = "someOrderId";
    ShopOrder shopOrder = anyOrder(orderId)
        .billingAddress(anAddress().id("someBillingAddressId")
            .firstName("Janina").lastName("Nemec")
            .streetName("Domstr.").houseNumber("20")
            .postalCode("50668").city("Köln").country("Deutschland")
            .phone("+49 221 1490").email("info@rewe-group.com").build()
        ).build();

    anOrderWasProcessed(shopOrder);

    AddressResult billingAddress = jsonMapper.readValue(callRestEndpointForBillingAddress(orderId));
    assertThat(billingAddress.getId()).isEqualTo("someBillingAddressId");
    assertThat(billingAddress.getFirstName()).isEqualTo("Janina");
    assertThat(billingAddress.getLastName()).isEqualTo("Nemec");
    assertThat(billingAddress.getStreetName()).isEqualTo("Domstr.");
    assertThat(billingAddress.getHouseNumber()).isEqualTo("20");
    assertThat(billingAddress.getPostalCode()).isEqualTo("50668");
    assertThat(billingAddress.getCity()).isEqualTo("Köln");
    assertThat(billingAddress.getCountry()).isEqualTo("Deutschland");
    assertThat(billingAddress.getPhone()).isEqualTo("+49 221 1490");
    assertThat(billingAddress.getLatitude()).isEqualTo("50.94603935915518");
    assertThat(billingAddress.getLongitude()).isEqualTo("6.959302840118697");
    assertThat(billingAddress.getStatus()).isEqualTo(CustomerStatus.NEW_CUSTOMER);
    assertThat(billingAddress.getEmail()).isEqualTo("info@rewe-group.com");
  }
}

class AddressAssertionTest {

  @Test
  void assertionTest() throws JsonProcessingException {
    String orderId = "someOrderId";
    ShopOrder shopOrder = anyOrder(orderId)
        .billingAddress(anAddress().id("someBillingAddressId")
            .firstName("Janina").lastName("Nemec")
            .streetName("Domstr.").houseNumber("20")
            .postalCode("50668").city("Köln").country("Deutschland")
            .phone("+49 221 1490").email("info@rewe-group.com").build()
        ).build();

    anOrderWasProcessed(shopOrder);

    AddressResult billingAddress = jsonMapper.readValue(callRestEndpointForBillingAddress(orderId));
    assertThat(billingAddress.getId()).isEqualTo("someBillingAddressId");
    assertThat(billingAddress.getFirstName()).isEqualTo("Janina");
    assertThat(billingAddress.getLastName()).isEqualTo("Nemec");
    assertThat(billingAddress.getStreetName()).isEqualTo("Domstr.");
    assertThat(billingAddress.getHouseNumber()).isEqualTo("20");
    assertThat(billingAddress.getPostalCode()).isEqualTo("50668");
    assertThat(billingAddress.getCity()).isEqualTo("Köln");
    assertThat(billingAddress.getCountry()).isEqualTo("Deutschland");
    assertThat(billingAddress.getPhone()).isEqualTo("+49 221 1490");
    assertThat(billingAddress.getStatus()).isEqualTo(CustomerStatus.NEW_CUSTOMER);
    assertThat(billingAddress.getEmail()).isEqualTo("info@rewe-group.com");
  }
}


{
  "id": "someBillingAddressId",
  "firstName": "Janina",
  "lastName": "Nemec",
  "streetName": "Domstr.",
  "houseNumber": "20",
  "city": "Köln",
  "country": "Deutschland",
  "phone": "+49 221 1490",
  "latitude": "50.94603935915518",
  "longitude": "6.959302840118697",
  "email": "info@rewe-group.com",
  "postalCode": "50668",
  "status": "new_customer"
}


{
  "id": "someBillingAddressId",
  "firstName": "Janina",
  "lastName": "Nemec",
  "streetName": "Domstr.",
  "houseNumber": "20",
  "city": "Köln",
  "country": "Deutschland",
  "phone": "+49 221 1490",
  "email": "info@rewe-group.com",
  "postalCode": "50668",
  "status": "new_customer"
}

Expected

Actual

class AddressAssertionTest {

  @Test
  void assertionTest() throws JsonProcessingException {
    String orderId = "someOrderId";
    ShopOrder shopOrder = anyOrder(orderId)
        .billingAddress(anAddress().id("someBillingAddressId")
            .firstName("Janina").lastName("Nemec")
            .streetName("Domstr.").houseNumber("20")
            .postalCode("50668").city("Köln").country("Deutschland")
            .phone("+49 221 1490").email("info@rewe-group.com").build()
        ).build();

    anOrderWasProcessed(shopOrder);

    AddressResult billingAddress = jsonMapper.readValue(callRestEndpointForBillingAddress(orderId));
    assertThat(billingAddress.getId()).isEqualTo("someBillingAddressId");
    assertThat(billingAddress.getFirstName()).isEqualTo("Janina");
    assertThat(billingAddress.getLastName()).isEqualTo("Nemec");
    assertThat(billingAddress.getStreetName()).isEqualTo("Domstr.");
    assertThat(billingAddress.getHouseNumber()).isEqualTo("20");
    assertThat(billingAddress.getPostalCode()).isEqualTo("50668");
    assertThat(billingAddress.getCity()).isEqualTo("Köln");
    assertThat(billingAddress.getCountry()).isEqualTo("Deutschland");
    assertThat(billingAddress.getPhone()).isEqualTo("+49 221 1490");
    assertThat(billingAddress.getStatus()).isEqualTo(CustomerStatus.NEW_CUSTOMER);
    assertThat(billingAddress.getEmail()).isEqualTo("info@rewe-group.com");
  }
}


class AddressAssertionTest {

  @Test
  void assertionTest() throws JsonProcessingException {
    String orderId = "someOrderId";
    ShopOrder shopOrder = anyOrder(orderId)
        .billingAddress(anAddress().id("someBillingAddressId")
            .firstName("Janina").lastName("Nemec")
            .streetName("Domstr.").houseNumber("20")
            .postalCode("50668").city("Köln").country("Deutschland")
            .phone("+49 221 1490").email("info@rewe-group.com").build()
        ).build();

    anOrderWasProcessed(shopOrder);

    JsonApprovals.verifyJson(callRestEndpointForBillingAddress(orderId));
  }
}

Actual

Expected

722-45173-28

€ see all differences

€ one verification

Approval Testing

€ long assertions

Value

  • Unit testing, integration testing, (GUI) testing
  • A little coding skills required
  • Many libs available
  • Exploratory Testing

Build

  • More concise result verification
  • More complete result verification

Application

Buyers also liked…

Energy Label

1.000.000s

Mutation Testing

722-45173-68

€ create mutants

€ untested code

€ find missing cases

free shipping (OSS)

Join the QAC
community

555722.de/68

Teenage Mutatant Hero Testing

PIT

Mutant

Stryker Mutator

Mutatest

class FizzBuzzTest {

  @Test
  void fizzbuzz_1() {
    assertThat(fizzbuzz(1)).isEqualTo("1");
  }

  @Test
  void fizzbuzz_3() {
    assertThat(fizzbuzz(3)).isEqualTo("Fizz");
  }

  @Test
  void fizzbuzz_5() {
    assertThat(fizzbuzz(5)).isEqualTo("Buzz");
  }
}
$ ./gradlew pitest

class FizzBuzzTest {

  @Test
  void fizzbuzz_1() {
    assertThat(fizzbuzz(1)).isEqualTo("1");
  }

  @Test
  void fizzbuzz_3() {
    assertThat(fizzbuzz(3)).isEqualTo("Fizz");
  }

  @Test
  void fizzbuzz_5() {
    assertThat(fizzbuzz(5)).isEqualTo("Buzz");
  }
}
$ ./gradlew pitest
…
================================================================================
- Statistics
================================================================================
>> Line Coverage (for mutated classes only): 4/5 (80%)
>> Generated 10 mutations Killed 8 (80%)
>> Mutations with no coverage 1. Test strength 89%
>> Ran 11 tests (1.1 tests per mutation)
class FizzBuzzTest {

  @Test
  void fizzbuzz_1() {
    assertThat(fizzbuzz(1)).isEqualTo("1");
  }

  @Test
  void fizzbuzz_3() {
    assertThat(fizzbuzz(3)).isEqualTo("Fizz");
  }

  @Test
  void fizzbuzz_5() {
    assertThat(fizzbuzz(5)).isEqualTo("Buzz");
  }
}
class FizzBuzzTest {

  @Test
  void fizzbuzz_1() {
    assertThat(fizzbuzz(1)).isEqualTo("1");
  }

  @Test
  void fizzbuzz_3() {
    assertThat(fizzbuzz(3)).isEqualTo("Fizz");
  }

  @Test
  void fizzbuzz_5() {
    assertThat(fizzbuzz(5)).isEqualTo("Buzz");
  }
 
  @Test
  void fizzbuzz_15() {
    assertThat(fizzbuzz(15)).isNotNull();
  }
}
$ ./gradlew pitest
…
================================================================================
- Statistics
================================================================================
>> Line Coverage (for mutated classes only): 5/5 (100%)
>> Generated 10 mutations Killed 8 (80%)
>> Mutations with no coverage 0. Test strength 80%
>> Ran 13 tests (1.3 tests per mutation)
class FizzBuzzTest {

  @Test
  void fizzbuzz_1() {
    assertThat(fizzbuzz(1)).isEqualTo("1");
  }

  @Test
  void fizzbuzz_3() {
    assertThat(fizzbuzz(3)).isEqualTo("Fizz");
  }

  @Test
  void fizzbuzz_5() {
    assertThat(fizzbuzz(5)).isEqualTo("Buzz");
  }
 
  @Test
  void fizzbuzz_15() {
    assertThat(fizzbuzz(15)).isNotNull();
  }
}
class FizzBuzzTest {

  @Test
  void fizzbuzz_1() {
    assertThat(fizzbuzz(1)).isEqualTo("1");
  }

  @Test
  void fizzbuzz_3() {
    assertThat(fizzbuzz(3)).isEqualTo("Fizz");
  }

  @Test
  void fizzbuzz_5() {
    assertThat(fizzbuzz(5)).isEqualTo("Buzz");
  }
 
  @Test
  void fizzbuzz_15() {
    assertThat(fizzbuzz(15)).isEqualTo("FizzBuzz");
  }
}
$ ./gradlew pitest
…
================================================================================
- Statistics
================================================================================
>> Line Coverage (for mutated classes only): 5/5 (100%)
>> Generated 10 mutations Killed 10 (100%)
>> Mutations with no coverage 0. Test strength 100%
>> Ran 11 tests (1.1 tests per mutation)

Value

  • Unit testing
  • Tooling can be tricky
  • Static code analysis

Build

  • Detect holes in your test coverage
  • Much better insight compared to code coverage

Application

Buyers also liked…

Energy Label

10× less

€ find missing cases

722-45173-68

€ untested code

€ create mutants

Mutation Testing

Ensemble Testing

722-45173-38

€ early feedback

€ works on my machine

€ learn & improve

free shipping (OSS)

Join the QAC
community

555722.de/38

Let's test together!!

"All the brilliant people working on the same thing, at the same time, in the same space, and on the same computer"

- Woody Zuill

Ensemble Testing

Ensemble Testing

Why do ensemble testing?

  • More ideas what to test

  • Better insight into user perspective for developers

  • Developers can help improve test setup

  • The product owner can give direct feedback

  • Can reduce pressure

  • Everyone learns from each other

Plan

Understand

Value

  • Great (free) remote tooling available
  • Office setup also possible (hardware separately available)
  • Please adhere to the manual to avoid known side effects
  • Can be applied to manual testing and automation
  • Team programming
  • Exploratory testing

Build

  • Get everyone on the team involved in testing
  • Learn from each other
  • More immediate feedback & problem solving

Application

Buyers also liked…

Energy Label

More insights

€ learn & improve

722-45173-38

€ works on my machine  

€ early feedback

Ensemble Testing

Feature Toggles

722-45173-387

€ canary testing

€ everyone suffers

€ ab-testing

free shipping (OSS)

Join the QAC
community

555722.de/38

Throw in on production like it's 1999 :)

How are feature toggles related to testing?

Testing in production

Canary Testing

A/B-Testing

Timed release

Easy Rollback

Plan

Understand

Value

  • Great (free) remote tooling available
  • ⚠️ Important to coordinate toggles across the product
  • 🧹 Cleaning required: remove toggles after release!
  • Continuous delivery & deployment (CD)
  • Monitoring & alerting
  • Service/Site Reliability Engineering (SRE)

Build

  • Deployment ≠ Release ⇒ more continuous work
  • Seamless and coordinated rollbacks possible

Application

Buyers also liked…

Energy Label

More insights

722-45173-387

€ a/b testing

€ delayed releases

€ canary releases

Feature Toggles

€ more values tested

722-45173-78

€ test duplication

€ fewer tests

Parameterized Testing

€ one at a time

722-45173-728

€ find surprises late

€ refactor unit tests

Property-Based Testing

€ see all differences

€ one verification

722-45173-28

Approval Testing

€ long assertions

€ one at a time

722-45173-728

€ find surprises late

€ refactor unit tests

Property-Based Testing

€ find missing cases

722-45173-68

€ untested code

€ create mutants

Mutation Testing

€ learn & improve

722-45173-38

€ works on my machine  

€ early feedback

Ensemble Testing

722-45173-387

€ a/b testing

€ delayed releases

€ canary releases

Feature Toggles

Rate the session 🙏

Get the slides 👆

Level-up Your Testing Toolkit!

By Michael Kutz

Level-up Your Testing Toolkit!

Have you ever heard about property-based testing? Do you know about mutation testing? Are you familiar with approval testing? What is your opinion about fuzzing? Many terms, techniques, and tools have come up since the invention of unit testing. But who has time to look into all of them? Thankfully, over the last few years, we were fortunate enough to have that time. In this talk, we want to share the insights we gained with you. Hence, we will lead you through each of the terms, explain their most important characteristics, and in which cases they hold value. This may not make you an expert, but gives you enough of an impression, to judge for yourself if a topic is worth further investigation.

  • 556

More from Michael Kutz