@MichaKutz

@mkutz@mstdn.social

# Why Measure Quality?

improve quality

Goal:

Make better informed decisions about quality

# What Could Possibly Go Wrong?

## Goodhart's Law

When a measure becomes a target, it ceases to be a good measure.

# How to Find Metrics?

## Goal → Question → Metric

Are we going in the right direction?

Where are we?

# Which Quality?

## Code Coverage

@Test
void strike() {
var game = new Game();
var firstRollPins = 10;
var secondRollPins = 5;
var thirdRollPins = 3;
game.roll(firstRollPins);
game.roll(secondRollPins);
game.roll(thirdRollPins);

var score = game.score();

assertThat(score)
.isEqualTo(
firstRollPins +
(secondRollPins + thirdRollPins) * 2);
}
public int score() {
int score = firstRoll + secondRoll;
if (previous != null) {
if (previous.strike) {
score *= 2;
} else if (previous.spare) {
score += firstRoll;
}
}
return score;
}
@Test
void strike() {
var game = new Game();
var firstRollPins = 10;
var secondRollPins = 5;
var thirdRollPins = 3;
game.roll(firstRollPins);
game.roll(secondRollPins);
game.roll(thirdRollPins);

var score = game.score();

// assertThat(score)
//  .isEqualTo(
//    firstRollPins +
//    (secondRollPins + thirdRollPins) * 2);
}

## Mutation Testing: Surviving Mutations

@Test
void strike() {
var game = new Game();
var firstRollPins = 10;
var secondRollPins = 5;
var thirdRollPins = 3;
game.roll(firstRollPins);
game.roll(secondRollPins);
game.roll(thirdRollPins);

var score = game.score();

assertThat(score
.isEqualTo(
firstRollPins +
(secondRollPins + thirdRollPins) * 2);
}
public int score() {
int score = firstRoll + secondRoll;
if (previous != null) {
if (previous.strike) {
score *= 2;
} else if (previous.spare) {
score += firstRoll;
}
}
return score;
}
AssertionFailedError:
expected: 26
but was: 14

## Team Surveys

### How confident is the team?

How effective can you work with the code?

How confident are you to deploy to production?

What would you need to improve the above answers?

## Static Code Analysis: Complexity

public int score() {
return firstRoll + secondRoll;
}
public int score() {
int score = firstRoll + secondRoll;
if (previous != null && previous.spare) {
score += firstRoll;
}
return score;
}
public int score() {
int score = firstRoll + secondRoll;
if (previous != null) {
if (previous.strike) {
score *= 2;
} else if (previous.spare) {
score += firstRoll;
}
}
return score;
}
previous
.spare
score *= 2;
previous
.strike
score += firstRoll;
previous != null
  && previous.spare
int score = firstRoll
+ secondRoll;
return score;

## Static Code Analysis: Code Smells

# Code Smells

- long methods,
- huge classes,
- many parameters,
- code duplicates,
- methods with complexity > 7,
- …

## Velocity

v = \frac{P_{estimate}}{t_{delivery} - t_{commit1}}

## Delivery Lead Time

\Delta t_{delivery} = t_{delivery} - t_{commit1}

## Change Fail Rate

08:07:23 Deploy basket-service ✓
09:06:11 Migrate checkout-service DB ✓
09:56:54 Deploy checkout-service ✓


08:07:23 Deploy basket-service ✓
09:06:11 Migrate checkout-service DB ✓
09:56:54 Deploy checkout-service ✓
10:19:44 Deploy order-managment-service ✗
10:39:27 Rollback order-managemet-service ✓


08:07:23 Deploy basket-service ✓
09:06:11 Migrate checkout-service DB ✓
09:56:54 Deploy checkout-service ✓
10:19:44 Deploy order-managment-service ✗
10:39:27 Rollback order-managemet-service ✓
11:09:59 Update database cluster ✓
12:27:32 Migrate order-managemet-service DB ✓
13:19:22 Deploy order-managemet-service ✓


08:07:23 Deploy basket-service ✓
09:06:11 Migrate checkout-service DB ✓
09:56:54 Deploy checkout-service ✓
10:19:44 Deploy order-managment-service ✗
10:39:27 Rollback order-managemet-service ✓
11:09:59 Update database cluster ✓
12:27:32 Migrate order-managemet-service DB ✓
13:19:22 Deploy order-managemet-service ✓
14:45:55 Update database-cluster ✗
15:50:49 Update database-cluster ✓


08:07:23 Deploy basket-service ✓
09:06:11 Migrate checkout-service DB ✓
09:56:54 Deploy checkout-service ✓
10:19:44 Deploy order-managment-service ✗
10:39:27 Rollback order-managemet-service ✓
11:09:59 Update database cluster ✓
12:27:32 Migrate order-managemet-service DB ✓
13:19:22 Deploy order-managemet-service ✓
14:45:55 Update database-cluster ✗
15:50:49 Update database-cluster ✓
16:39:11 Deploy product-service ✓
17:44:56 Deploy customer-data-service ✓

## Mean Time to Restore Service

08:07:23 Deploy basket-service ✓
09:06:11 Migrate checkout-service DB ✓
09:56:54 Deploy checkout-service ✓
10:19:44 Deploy order-managment-service ✗
10:39:27 Rollback order-managemet-service ✓
11:09:59 Update database cluster ✓
12:27:32 Migrate order-managemet-service DB ✓
13:19:22 Deploy order-managemet-service ✓
14:45:55 Update database-cluster ✗
15:50:49 Update database-cluster ✓
16:39:11 Deploy product-service ✓
17:44:56 Deploy customer-data-service ✓

## How fast?

### How fast?

@MichaKutz

@mkutz@mstdn.social

Not everything that counts can be counted,
and not everything that can be counted counts.