# Do Not Fight Your Testing Tools

### Sr Director of Engineering, Mercari US

Heavy smoke fills the air as people cross 34th Street in Herald Square in New York City on June 6, 2023.

## survival is possible* but we need to act now

• change your life
• join an organization

# {testing DX}

• JavaScript vs The World
• JavaScript vs JavaScript vs JavaScript
• Testing is broken
• Testing will be alright

## Speaker: Gleb Bahmutov PhD

C / C++ / C# / Java / CoffeeScript / JavaScript / Node / Angular / Vue / Cycle.js / functional programming / testing

# Lots Of Testing

Edit and view HTML documents, 1991

JavaScript!

Problem: given an array of numbers, multiply each number by a constant and print the result.

``````var numbers = [3, 1, 7];
var constant = 2;
// 6 2 14``````

# procedural / imperative JS

``````var numbers = [3, 1, 7];
var constant = 2;
var k = 0;
for(k = 0; k < numbers.length; k += 1) {
console.log(numbers[k] * constant);
}
// 6 2 14``````

Arrays: object-oriented + functional

``````var numbers = [3, 1, 7];
var constant = 2;
function mul(a, b) {
return a * b;
}
function print(n) {
console.log(n);
}
var mulBy = mul.bind(null, constant);
numbers
.map(mulBy)
.forEach(print);
// 6 2 14``````
• clear "multiply then print" semantics

functional bits

Object-oriented methods

Arrays: object-oriented + functional

``````numbers
.map(mulBy)
.forEach(print);``````

JS was influenced by Self and Scheme

``````numbers
.map(mulBy)
.forEach(print);
// same as
const a1 = numbers.map(mulBy);
a1.forEach(print)``````
``````numbers
.map(mulBy)
.filter(predicate)
.map(...)
.reduce(...)``````

Fluent and chained

I want a value!

``````const X = numbers
.map(mulBy)
.filter(predicate)
.map(...)
.reduce(...)
// use x``````

"But what if my computation is asynchronous?"

``````compute((x) => {
// x is the value
})
// runs before callback
// cannot access x``````

callback

``````const X = compute()
expect(X).to.deep.equal([6, 2, 14])``````

# What about testing?

"But what if my computation is asynchronous?"

``````it('computes', (done) => {
compute((x) => {
expect(x).to.deep.equal(...)
done()
})
})``````

# Promises

Single async operation with a value or an error

``````const p = new Promise((resolve, reject) => {
...
resolve(42)
})
// after promise successfully completes
p.then(...)
.catch(...)``````
``````var sleepSecond = _.partial(Q.delay, 1000);

var pauseMulAndPrint = function (n) {
return function () {
return sleepSecond()
.then(_.partial(byConstant, n))
.then(print);
};
};``````

### Sleep, mul then print using promises

``````numbers.map(pauseMulAndPrint)
.reduce(Q.when, Q())
.done();
// ... 6 ... 2 ... 14``````
``````it('computes', () => {
return compute().then(x => {
expect(x).to.deep.equal(...)
})
})``````

# Async to the rescue

``````it('computes', async () => {
const x = await compute()
expect(x).to.deep.equal(...)
})``````

# Async to the rescue

``````<body>
<h1>Fruits</h1>
<ul>
<li>Apples</li>
<li>Grapes</li>
<li>Kiwi</li>
</ul>
</body>``````

# Static page

``````test('has 3 fruits', async ({ page }) => {
await page.goto(index)
await expect(page.getByRole('listitem')).toHaveText([
'Apples',
'Grapes',
'Kiwi',
])
})``````

Playwright (pw)

# Dynamic page

``````test('has 3 fruits', async ({ page }) => {
await page.goto(index)
await expect(page.getByRole('listitem')).toHaveText([
'Apples',
'Grapes',
'Kiwi',
])
})``````

Playwright (pw)

Microsoft Playwright automatically retries the assertion

# With prices

``````test('has 3 fruits', async ({ page }) => {
await page.goto(index)
const elements = page.getByRole('listitem')
await expect(async () => {
const strings = await elements.allInnerTexts()
const prices = strings.map((s) => s.split('\$')[1])
.map(Number)
expect(prices).toEqual([1, 2, 3])
}).toPass()
})``````

Playwright (pw)

``````test('has 3 fruits', async ({ page }) => {
await page.goto(index)
const elements = page.getByRole('listitem')
await expect(async () => {
const strings = await elements.allInnerTexts()
const prices = strings.map((s) => s.split('\$')[1])
.map(Number)
expect(prices).toEqual([1, 2, 3])
}).toPass()
})``````

Playwright (pw)

``````test('has 3 fruits', async ({ page }) => {

await expect(async () => {

}).toPass()
})``````

retry async fn

Playwright (pw)

# I want a value!

``````const X = elements
const strings = X.innerText
const prices = ...
expect(prices).to.deep.equal(...)``````

## Promises were wrapped values

``````getElements()
.then(els => els.innerText)
.then(parsePrices)
.then(prices => expect(prices)...)
.catch(e => ...)``````

# In functional programming

``````const make = (input) =>
Option.fromNullable(input)
.map(parseInput)
.flatMap((input) => Option.fromNullable(transform(input)))
.map(print)
.map(prettify)
.getWithDefault("fallback");``````

# In functional reactive programming

• Everything is a source of asynchronous events.
• Your code is a pipeline reacting to one or more input streams and outputting another stream of events

# In functional reactive programming

User click events

``````var Rx = require('rx');

var clickEvents = Rx.Observable
.fromEvent(document.querySelector('#btn'), 'click')

var numberEvents = Rx.Observable
.fromArray(numbers);

function pickSecond(c, n) { return n; }

Rx.Observable.zip(clickEvents, numberEvents,
pickSecond)
.map(byConstant)
.subscribe(print);
// prints a number on each button click``````

Promise

Observable

async / await

🛑

# I want a value!

``````test('has 3 fruits', async ({ page }) => {
await page.goto(index)
const elements = page.getByRole('listitem')
await expect(async () => {
const strings = await elements.allInnerTexts()
const prices = strings.map((s) => s.split('\$')[1])
.map(Number)
expect(prices).toEqual([1, 2, 3])
}).toPass()
})``````

Retry

• What if we accept that almost everything is async?
• What if we optimize for data transformations and assertions?
``````import 'cypress-map'

it('has 3 fruits', () => {
cy.visit('public/index.html')
cy.get('#fruits li')
.map('innerText')
.mapInvoke('split', '\$')
.mapInvoke('at', 1)
.map(Number)
.should('deep.equal', [1, 2, 3])
})``````

Cypress (cy)

``````import 'cypress-map'

it('has 3 fruits', () => {
cy.visit('public/index.html')
cy.get('#fruits li')
.map('innerText')
.mapInvoke('split', '\$')
.mapInvoke('at', 1)
.map(Number)
.should('deep.equal', [1, 2, 3])
})``````
``````import 'cypress-map'

it('has 3 fruits', () => {
cy.visit('public/index.html')
cy.get('#fruits li')
.map('innerText')
.mapInvoke('split', '\$')
.mapInvoke('at', 1)
.map(Number)
.should('deep.equal', [1, 2, 3])
})``````

Elements to numbers

pipeline

retry

``````import 'cypress-map'

it('has 3 fruits', () => {
cy.visit('public/index.html')
cy.get('#fruits li')
.map('innerText')
.mapInvoke('split', '\$')
.mapInvoke('at', 1)
.map(Number)
.should('deep.equal', [1, 2, 3])
})``````

Elements to numbers

pipeline

``````test('has 3 fruits', async ({ page }) => {
await page.goto(index)
const elements = page.getByRole('listitem')
await expect(async () => {
const strings = await elements.allInnerTexts()
const prices = strings.map((s) => s.split('\$')[1])
.map(Number)
expect(prices).toEqual([1, 2, 3])
}).toPass()
})``````

Retry

Vs

retry

cy

pw

``````it('has 3 fruits', () => {
cy.visit('public/index.html')
})``````
``````test('has 3 fruits', async ({ page }) => {
await page.goto(index)
})``````

Of course the test is async

Of course loading the page is async

Of course the test must wait for the command to finish

cy

pw

``````import { test, expect } from '@playwright/test'

test('adds todos', async ({ page, request }) => {
await request.post('/reset', { data: { todos: [] } })
await page.goto('/')
// create a new todo locator
const newTodo = page.getByPlaceholder('What needs to be done?')
await newTodo.fill('one')
await newTodo.press('Enter')
await newTodo.fill('two')
await newTodo.press('Enter')
const todoItems = page.locator('li.todo')
await expect(todoItems).toHaveCount(2)
})``````
``````it('adds todos', () => {
cy.request('POST', '/reset', { todos: [] })
cy.visit('/')
cy.get('.new-todo').type('one{enter}').type('two{enter}')
cy.get('li.todo').should('have.length', 2)
})
``````

// cy

## The Syntax

// pw

• imperative syntax
• promise-based
• declarative syntax
• reactive stream
``````import { test, expect } from '@playwright/test'

test('adds todos', async ({ page, request }) => {
await request.post('/reset', { data: { todos: [] } })
await page.goto('/')
// create a new todo locator
const newTodo = page.getByPlaceholder('What needs to be done?')
await newTodo.fill('one')
await newTodo.press('Enter')
await newTodo.fill('two')
await newTodo.press('Enter')
const todoItems = page.locator('li.todo')
await expect(todoItems).toHaveCount(2)
})``````
``````it('adds todos', () => {
cy.request('POST', '/reset', { todos: [] })
cy.visit('/')
cy.get('.new-todo').type('one{enter}').type('two{enter}')
cy.get('li.todo').should('have.length', 2)
})
``````

// cy

## The "Tinder" data flow

// pw

• imperative syntax
• promise-based
• declarative syntax
• reactive stream

Swipe right

(data flows to the right)

Swipe left

(data is assigned to the left)

``````var k = 0;
for(k = 0; k < numbers.length; k += 1) {
console.log(numbers[k] * constant);
}
// 6 2 14``````
``````// _ is Lodash / Ramda
_(numbers)
.map(_.partial(mul, constant))
.forEach(print);
// 6 2 14``````

// functional

## for-loop vs Array.forEach

// imperative

Swipe right

(data flows to the right)

Swipe left

(data is assigned to the left)

``````const n = Number(await locator.getText())
// n is set``````
``````cy.get('#count').then(n =>
// n is set
)``````

// right

## Data flows left vs right

// left

Swipe right

(data flows to the right)

Swipe left

(data is assigned to the left)

# 🤯 🤬

``````const n = Number(await locator.getText())
// n is set``````
``````cy.get('#count').then(n =>
// n is set
)``````

// right

// left

# 🤯 🤬

What do you want to check on the page?

``cy.get('li.todo').should('have.length', 2)``
There should be 2 items
``````cy.get('li.todo')
.should('satisfy', \$li => \$li.length % 2 === 0)``````
There should be an even number of items
``````cy.wait('@load').its('response.body.length')
.then(n => {
cy.get('li.todo').should('have.length', n)
})``````
There should be the same number of items as returned by the server
``````cy.intercept('GET', '/todos', { fixture: 'three.json' })
cy.get('li.todo').should('have.length', 3)``````

If you can control the data in your tests, then the test syntax collapses into a simple and elegant fluent chain

\$todo

``````test('has 3 fruits', async ({ page }) => {
await page.goto(index)
await expect(page.getByRole('listitem')).toHaveText([
'Apples',
'Grapes',
'Kiwi',
])
})``````
``````<body>
<h1>Fruits</h1>
<ul>
<li>Apples</li>
<li>Grapes</li>
<li>Kiwi</li>
</ul>
</body>``````

# Static page

``````test('has 3 fruits', async ({ page }) => {
await page.goto(index)
await expect(page.getByRole('listitem')).toHaveText([
'Apples',
'Grapes',
'Kiwi',
])
})``````
``````import 'cypress-map'
it('has 3 fruits', () => {
cy.visit(index)
cy.get('#fruits li')
.map('innerText')
.should('deep.equal', ['Apples', 'Grapes', 'Kiwi'])
})``````

functional reactive bits

if you do not have "toHaveText" assertion...

If the test looks weird or complicated...

``````it('has 3 fruits', () => {
try {
cy.visit('public/index.html')
} catch (e) {
...
}
})``````
``````test('has 3 fruits', async ({ page }) => {
try {
await page.goto(index)
} catch (err) {
...
}
})``````

# Side Rant: Error handling

Nope, does not work

just like it does not with

event emitters or callbacks

Ok, now what?!

cy

pw

``````import { defineConfig } from '@playwright/test';

export default defineConfig({
// Rest of your config...

// Run your local dev server before starting the tests
webServer: {
command: 'npm run start',
url: 'http://127.0.0.1:3000',
reuseExistingServer: !process.env.CI,
timeout: 120 * 1000,
},
});``````

playwright.config.ts

Tests should follow a single deterministic path

I do not care how long the tests take to run.

I care how long it takes to debug a failing test

Time-travel to inspect the result of each command ⏳

``\$ npx playwright test --trace on``
``\$ npx playwright test --ui``

Jest + RTL + MSW vs Cypress component testing

``````const handlers = [
rest.get(`\${process.env.REACT_APP_API_URL}/heroes`, async (_req, res, ctx) =>
res(ctx.status(200), ctx.json(heroes))
),
rest.delete(
`\${process.env.REACT_APP_API_URL}/heroes/\${heroes[0].id}`, // use /.*/ for all requests
async (_req, res, ctx) => res(ctx.status(400), ctx.json("expected error"))
),
];
const server = setupServer(...handlers);
beforeAll(() => {
server.listen({
onUnhandledRequest: "warn",
});
});
afterEach(server.resetHandlers);
afterAll(server.close);

it("should display the hero list on render, and go through hero add & refresh flow", async () => {
expect(await screen.findByTestId("hero-list")).toBeVisible();

await userEvent.click(await screen.findByTestId("refresh-button"));
expect(window.location.pathname).toBe("/heroes");
});
``````
``````beforeEach(() => {
cy.intercept('GET', `\${Cypress.env('API_URL')}/heroes`, {
fixture: 'heroes.json',
}).as('getHeroes')

cy.wrappedMount(<Heroes />)
})

it('should display the hero list on render, and go through hero add & refresh flow', () => {
cy.wait('@getHeroes')

cy.getByCy('hero-list').should('be.visible')

cy.getByCy('refresh-button').click()
cy.location('pathname').should('eq', '/heroes')
})``````

"Cypress Component Testing Vs RTL and MSW: The Missing Comparison Part"

Jest

dumps

the current

HTML

"Cypress Component Testing Vs RTL and MSW: The Missing Comparison Part"

Cypress

shows what

each test

command

did or did not do

# Time-Travel From The Future

Let me take a look. Is this why my test has failed?

# Past Predictions

• Component testing in Cypress will be big
• Capturing everything during tests will be big

Gleb at AssertJS 2018

https://slides.com/bahmutov/assertjs

• Playwright needs UI mode
• Cypress needs trace capture

Gleb at Global Test Summit 2022

https://slides.com/bahmutov/the-debate-cy-and-pw

# Predictions

• Diffing captured information >>> capture all information
• The line between monitoring and testing tools will become blurry

Gleb at Refactor DX 2023

https://slides.com/bahmutov/no-fighting

*

* past performance does not guarantee future results. For entertainment purposes only

👏 Thank You 👏

By Gleb Bahmutov

# Do Not Fight Your Testing Tools

We are truly living in the golden age of web testing tools. Node.js got its own built-in test runner, web developers got Cypress and Playwright, and everyone got new debugging tools like Replay.io that replay the entire browser session statement by statement. In this talk, I will show how to take advantage of these new testing and debugging tools. Shown at RefactorDX 2023, watch at https://youtu.be/b87e7k3RjOg

• 1,468