Building confidence in times of change
using automation
By
Shadi Sharaf
About me
<role>
Software engineer</role>
<ul id="experience">
<li id="years">
17+ years of experience 👨🏼🦳</li>
<li id="passion">
Software engineering, standards, and high performing teams.</li>
<li id="worked-for">
The Times, Human Made, XWP, and more.</li>
<li id="worked-with">
Google, Siemens, Sony, Disney, and more.</li>
<li id="worked-on">
Stream, WordPress Coding Standards</li>
</ul>
Agenda
- The challenge of Regression
- How does it happen
- What does that mean
- What can we do about
- Introduction to Playwright
- Generating automated tests
- Performance benchmarking
Regression {noun}
A change in the behavior of a system that is not expected.

Challenge: Regression
How does that happen?
Code changes can have unexpected consequences
Challenge: Regression

Why is that dangerous?
What you don’t know can still burn.
Challenge: Regression

What can we do about it?
Don’t wait for the smoke
Challenge: Regression

Active testing
aka QA
TEST ALL THE THINGS!
Challenge: Regression

Active testing
aka QA
BUT....
Challenge: Regression

Passive monitoring
Don’t leave problems in the dark
Challenge: Regression

Playwright {noun}
A person who writes plays.

Solution: Playwright
Why Playwright?
- Cross-browser testing
- Modern architecture
- Powerful API
- Flexible Emulation
- Great developer experience
- WordPress integration

Solution: Playwright
Setting up
$ yarn create playwright # OR npm init playwright
✔ Do you want to use TypeScript or JavaScript? · JavaScript
✔ Where to put your end-to-end tests? · tests
✔ Add a GitHub Actions workflow? (y/N) · false
✔ Install Playwright browsers (can be done manually via 'yarn playwright install')? (Y/n) · true
Writing playwright.config.js.
Writing tests/example.spec.js.
Writing tests-examples/demo-todo-app.spec.js.
Writing package.json.
Downloading browsers (yarn playwright install)…
✔ Success! Created a Playwright Test project at /playwright/stage
Happy hacking! 🎭
Solution: Playwright
Running tests
$ yarn playwright test
Running 6 tests using 4 workers
6 passed (5.7s)
To open last HTML report run:
yarn playwright show-report
Solution: Playwright
Viewing reports

Solution: Playwright
Test {noun}
The ultimate prank you play on your code—surprising it with tricky questions, goofy conditions, and unexpected hurdles just to see if it slips on a banana peel or stands tall and proud

Solution: Playwright - Generating tests
Anatomy of a test
import { test, expect } from '@playwright/test';
test('has title', async ({ page }) => {
// Go to the URL
await page.goto('https://playwright.dev/');
// Expect a title "to contain" a substring.
await expect(page).toHaveTitle(/Playwright/);
});
test('get started link', async ({ page }) => {
// Go to the URL
await page.goto('https://playwright.dev/');
// Click the get started link.
await page.getByRole('link', { name: 'Get started' }).click();
// Expects page to have a heading with the name of Installation.
await expect(page.getByRole('heading', { name: 'Installation' })).toBeVisible();
});
Solution: Playwright - Generating tests
1. Arrange
2. Act
3. Assert
Solution: Playwright - Generating tests
Locators 🔍
// Locate an element by explicit accessibility role
const element = await page.getByRole('link', { name: 'Get started' });
// Locate an element by text content
const element = await page.getByText('Get started');
// Locate an element by aria-label
const element = await page.getByLabel('Search');
// Locate an element by placeholder
const element = await page.getByPlaceholder('Search');
// Locate an element by alt text
const element = await page.getByAltText('Playwright logo');
// Locate an element by title
const element = await page.getByTitle('Playwright logo');
// Locate an element by data-testid
const element = await page.getByTestId('my-element');
Solution: Playwright - Generating tests
Actions 🔨
// Click the element
element.click();
// Focus the element
element.focus();
// Fill the form field
element.fill();
// Check and uncheck a checkbox
element.check();
element.uncheck();
// Select option in the dropdown
element.selectOption();
// Press single key
element.press();
// Upload files
element.setInputFiles();
Solution: Playwright - Generating tests
Assertions 🧐
// Check if the element is visible
await expect(element).toBeVisible();
// Check if the element is disabled
await expect(element).toBeDisabled();
// Check if the checkbox is checked
await expect(element).toBeChecked();
// Check if the element is focused
await expect(element).toBeFocused();
// Check if the element contains text
await expect(element).toContainText('Hello');
// Check if the element has an attribute
await expect(element).toHaveAttribute('href', 'https://playwright.dev/');
// Check if the element has a value
await expect(element).toHaveValue('Hello');
Solution: Playwright - Generating tests
Visual regressions 🤯
// Generate OR check the screenshot
await expect(element).toHaveScreenshot();
Solution: Playwright - Generating tests
Too much?
Too technical?
Solution: Playwright - Generating tests
Let's
Go to
The Fun part!

Solution: Playwright - Generating tests
Generating tests
Solution: Playwright - Generating tests
Generating tests
$ yarn playwright codegen https://playwright.dev/
Solution: Playwright - Generating tests
Generating tests

Solution: Playwright - Generating tests
Generating tests

import { test, expect } from '@playwright/test';
test('test', async ({ page }) => {
await page.goto('https://playwright.dev/');
await page.getByRole('link', { name: 'Get started' }).click();
await page.getByRole('link', { name: 'Generating tests' }).click();
await expect(page.locator('h1')).toContainText('Generating tests');
await expect(page.getByRole('article')).toMatchAriaSnapshot(`
- list:
- listitem:
- link "How to record a test"
- listitem:
- link "How to generate locators"
`);
});
Solution: Playwright - Generating tests
Running Tests
Solution: Playwright - Generating tests
Running Tests
$ yarn playwright test
Solution: Playwright - Generating tests
Running Tests
$ yarn playwright test
Running 6 tests using 4 workers
6 passed (6.7s)
To open last HTML report run:
yarn playwright show-report
Solution: Playwright - Generating tests
Running Tests (UI mode)
$ yarn playwright test --ui
Solution: Playwright - Generating tests
Running Tests (UI mode)
Solution: Playwright - Generating tests
Running Tests (Debug mode)
$ yarn playwright test --debug
Solution: Playwright - Generating tests
Running Tests (Debug mode)
Solution: Playwright - Generating tests
import { test, expect } from '@playwright/test';
test('test', async ({ page }) => {
await page.goto('https://playwright.dev/');
await page.getByRole('link', { name: 'Get started' }).click();
await page.getByRole('link', { name: 'Generating tests' }).click();
await expect(page.locator('h1')).toContainText('Generating toasts');
await expect(page.getByRole('article')).toMatchAriaSnapshot(`
- list:
- listitem:
- link "How to record a test"
- listitem:
- link "How to generate locators"
`);
});
Running Tests (Debug mode) 🐞
Solution: Playwright - Generating tests
Running Tests (Debug mode) 🐞
Solution: Playwright - Generating tests
Traces
Solution: Playwright - Generating tests
Traces
$ yarn playwright test --trace on
Solution: Playwright - Generating tests
Traces
Solution: Playwright - Generating tests
Questions?
Solution: Playwright - Generating tests
Performance
{noun}
When your code takes center stage and delivers a flawless show—no hiccups, just pure applause

Solution: Playwright - Performance
Why measure Performance?
less time = more users
more users = more money
more money = less work
less work = more ✈️🏖️ time

Solution: Playwright - Performance
Why measure Performance
with Playwright?
- Native Performance APIs
- CDP APIs

Solution: Playwright - Performance
Navigation
Resources
Paint
Solution: Playwright - Performance
Navigation
const pageTime = performance.getEntriesByType('navigation');
{
// ..
"duration": 1854.7000000029802, // The difference between startTime and responseEnd
"fetchStart": 11.200000002980232, // Time at which the fetch starts
"domainLookupStart": 13.100000008940697, // Time at which the domain lookup starts
"domainLookupEnd": 13.100000008940697, // Time after the domain lookup
"connectStart": 13.100000008940697, // Time before establishing a server connection
"secureConnectionStart": 44.80000001192093, // Time before the SSL handshake
"connectEnd": 269.80000001192093, // Time after establishing a server connection
"requestStart": 269.90000000596046, // Time before the browser requests the resource
"responseStart": 1293.4000000059605, // Time when the browser receives the first byte of data
"responseEnd": 1298, // Time after receiving the last byte or closing the connection
"transferSize": 10698, // The resource size in bytes including the header and compressed body
"domInteractive": 1475, // Time at which the document becomes interactive
"domContentLoadedEventStart": 1475.1000000089407, // Time before document’s DOMContentLoaded event fires
"domContentLoadedEventEnd": 1520, // Time after document’s DOMContentLoaded event completes
"domComplete": 1851.9000000059605, // Time at which the document becomes complete
"loadEventStart": 1851.9000000059605, // Time at which the window.onload event fires
"loadEventEnd": 1854.7000000029802, // Time after the window.onload event completes
// ..
}
Solution: Playwright - Performance
Resources
const resourceTime = performance.getEntriesByType('resource');
{
connectEnd: 195,
connectStart: 195,
decodedBodySize: 0,
domainLookupEnd: 195,
domainLookupStart: 195,
duration: 2,
encodedBodySize: 0,
entryType: "resource",
fetchStart: 195,
initiatorType: "script",
name: "https://test.com/script.js",
requestStart: 195,
responseEnd: 197,
responseStart: 197,
secureConnectionStart: 195,
serverTiming: [],
startTime: 195,
transferSize: 0,
workerStart: 195
}
Solution: Playwright - Performance
Paint
const webVitals = performance.getEntriesByType('paint');
[
{
"name": "first-paint",
"entryType": "paint",
"startTime": 812,
"duration": 0
},
{
"name": "first-contentful-paint",
"entryType": "paint",
"startTime": 856,
"duration": 0
}
]
Solution: Playwright - Performance
Emulation

Test it all, break nothing: Emulation for every situation!
Solution: Playwright - Emulation
Emulalation
// custom.config.js
export {
viewport: { width: 360, height: 640 },
devices: [playwright.devices['iPhone 12']],
geolocation: { latitude: 37.7749, longitude: -122.4194 },
locale: 'en-US',
timezone: 'America/Los_Angeles',
permissions: ['geolocation', 'notifications'],
colorScheme: 'dark',
offline: true,
javascriptEnabled: false
}
// Use it in playwright.config.js
import { defineConfig } from '@playwright/test';
export default defineConfig({ use: { /*...*/ } })
// Use it in test file
import { test } from '@playwright/test'
test.use({ /*...*/ })
Solution: Playwright - Emulation
Multiple configs
import { defineConfig } from '@playwright/test';
export default defineConfig({ projects: [
{
name: 'desktop',
use {
// ...
}
},
{
name: 'mobile',
use {
// ...
}
},
] })
Solution: Playwright - Emulation
Chrome DevTool Protocol
Solution: Playwright - Emulation
Chrome DevTool Protocol
import { test } from '@playwright/test'
test('basic performance emulation', async ({ page }) => {
const cdp = await page.context().newCDPSession(page)
await cdp.send('Network.enable')
await cdp.send('Network.emulateNetworkConditions', {
offline: false,
downloadThroughput: (4 * 1024 * 1024) / 8,
uploadThroughput: (3 * 1024 * 1024) / 8,
latency: 20
})
await page.goto('https://playwright.dev/docs')
})
Solution: Playwright - Emulation
Chrome DevTool Protocol
async function measurePerformance(executeSteps, stepName, page) {
// Add listeners
page.on('request', request => { /* .. collect some metrics .. */ });
page.on('response', async response => { /* .. collect some more metrics .. */ } );
// Execute the steps with performance markers
const start = performance.now();
await executeSteps();
const end = performance.now();
// Record the performance metrics
const timings = await page.evaluate(() => performance.timing );
const navigationTime = await page.evaluate(() => performance.getEntriesByType('navigation') );
perfMetrics[step] = {
stepName,
timeElapsed: end - start,
visualComplete: timings.loadEventEnd - timing.navigationStart,
navigationTime: navigationTime[0].duration,
// mooooaaarrr metrics
};
step++; // Step Counter
}
Solution: Playwright - Emulation
Further readings
Documentation
- Playwright - https://playwright.dev/
- Playwright emulation - https://playwright.dev/docs/emulation
- Performance API - https://developer.mozilla.org/en-US/docs/Web/API/Performance_API
- CDP APIs - https://chromedevtools.github.io/devtools-protocol/
- Gutenberg Playwright utils - https://developer.wordpress.org/block-editor/reference-guides/packages/packages-e2e-test-utils-playwright/
Articles
- Kinsta - Performance API - https://kinsta.com/blog/performance-api/
- Checkly - Playwright and Performance APIs - https://www.checklyhq.com/learn/playwright/performance/
- Performance testing using Playwright - https://medium.com/@sahajk13/performance-testing-using-playwright-ea8f372e76f0
Thank you!
Questions ?
linkedin: shady.sharaf
wp.org: shadyvb
github: shadyvb

Playwright
By Shadi Sharaf
Playwright
- 38