Gleb Bahmutov PRO
JavaScript ninja, image processing expert, software quality fanatic
The Debate Between Cypress and Playwright
Gleb Bahmutov
@bahmutov
Global
Testing
Summit
Join others to fight the climate crisis
Speaker: Gleb Bahmutov PhD
C / C++ / C# / Java / CoffeeScript / JavaScript / Node / Angular / Vue / Cycle.js / functional programming / testing,
former VP of Engineering at Cypress.io
Gleb Bahmutov
Sr Director of Engineering
I ❤️ Cypress
- used it for 1 year before joining Cypress.io
- VP of Eng, Distinguished Engineer at Cypress.io
- implemented Cypress on Windows, parallelization, the paywall, component testing, API testing, and lots of other things
- taught 100s of people at Cypress workshops
- 500+ examples and recipes at https://glebbahmutov.com/cypress-examples/
- 400+ videos about Cypress at https://www.youtube.com/glebbahmutov
- I answer Cypress questions and help users
Do not compare yourself to other tools
CY vs PW
Book "A Frontend Web Developer's Guide to Testing" by Eran Kinsbruner
Cypress vs Playwright – Clash of the Titans
https://www.zeljkovic.sh/cypress-vs-playwright-clash-of-the-titans-introduction/
On Migrating from Cypress to Playwright
Playwright vs Cypress: A Comparison
Playwright vs. Cypress: Which Cross-Browser Testing Solution Is Right for You?
Playwright vs Cypress: Which Framework to Choose For E2E Testing?
https://labs.eleks.com/2022/07/playwright-vs-cypress-e2e-testing.html
Five reasons why Playwright is better than Cypress
https://alisterbscott.com/2021/10/27/five-reasons-why-playwright-is-better-than-cypress/
Playwright vs Cypress
Cypress vs Playwright: Let the Code Speak
Playwright vs Cypress E2E Testing
https://github.com/cliffordfajardo/cypress-vs-playwright-comparison
Cypress vs Playwright: What’s the Best Test Automation Framework for Your Project?
Cypress vs. Playwright: end-to-end testing showdown
https://silvenon.com/blog/e2e-testing-with-cypress-vs-playwright
CY vs PW
On Migrating from Cypress to Playwright
Playwright vs Cypress: A Comparison
Playwright vs. Cypress: Which Cross-Browser Testing Solution Is Right for You?
Playwright vs Cypress: Which Framework to Choose For E2E Testing?
https://labs.eleks.com/2022/07/playwright-vs-cypress-e2e-testing.html
Five reasons why Playwright is better than Cypress
https://alisterbscott.com/2021/10/27/five-reasons-why-playwright-is-better-than-cypress/
Playwright vs Cypress
Cypress vs Playwright: Let the Code Speak
Playwright vs Cypress E2E Testing
https://github.com/cliffordfajardo/cypress-vs-playwright-comparison
Cypress vs Playwright: What’s the Best Test Automation Framework for Your Project?
Cypress vs. Playwright: end-to-end testing showdown
https://silvenon.com/blog/e2e-testing-with-cypress-vs-playwright
Book "A Frontend Web Developer's Guide to Testing" by Eran Kinsbruner
Cypress vs Playwright – Clash of the Titans
https://www.zeljkovic.sh/cypress-vs-playwright-clash-of-the-titans-introduction/
👍
👍
Cypress vs Playwright: What’s the Best Test Automation Framework for Your Project?
-
Out of date
Any comparison older than one month
-
Pineapples to orange soda
/* Run your local dev server before starting the tests */
webServer: {
command: 'npm run start',
port: 3000,
}// pw
cy.visit('index.html')// cy
In Cypress
// DOES NOT WORK
page.goto('index.html')
// pw
Is this accurate?
how important is this for us?
should it be a core feature?
Cypress design principle
Be a cohesive testing tool, allow new features to be implemented as plugins
core features
experimental features
plugins
external tools
Cypress design principle
core features
experimental features
plugins
external tools
start-server-and-test
Component
testing
Cypress design principle
core features
experimental features
plugins
external tools
start-server-and-test
Component
testing
Cypress design principle
core features
experimental features
plugins
external tools
start-server-and-test
Component
testing
Cypress design principle
core features
experimental features
plugins
external tools
start-server-and-test
Component
testing
cy.ng()
Plugins everywhere (145+ at https://on.cypress.io/plugins)
"My favorite Cypress plugins"
https://cypresstips.substack.com/p/my-favorite-cypress-plugins
"My favorite Cypress plugins, part II"
https://cypresstips.substack.com/p/my-favorite-cypress-plugins-part
"Cypress Plugins" paid course 🎓 💵
import { test } from '@playwright/test'
test('logs hello', async () => {
console.log('hello, world')
})it('logs hello', () => {
console.log('hello, world')
})// pw
// cy
// pw
// cy
Cypress iframes the app and its specs
Node.js
Playwright test code
Browser
application
Chrome Debugger Protocol
Node.js
Config / plugins file
Browser
application
Cypress test code
WebSocket
for
cy.task
Test runners architecture
it('listens to the window.postMessage events', () => {
cy.visit('index.html', {
onBeforeLoad(win) {
cy.spy(win, 'postMessage').as('postMessage')
},
})
cy.get('@postMessage')
.should('have.been.calledTwice')
.and('have.been.calledWithExactly', 'one')
.and('have.been.calledWithExactly', 'two')
})Direct access to the application's objects and browser APIs from the test code
💪
it('collects the window.postMessage events', () => {
const messages = []
cy.visit('index.html', {
onBeforeLoad(win) {
win.addEventListener('message', (e) => {
messages.push(e.data)
})
},
})
cy.wrap(messages).should('have.length', 2)
.its(0).should('equal', 'one')
})Direct access to the application's objects and browser APIs from the test code
💪
Bypass problematic methods from
the test
import { todos } from '../fixtures/three.json'
it('copies the todos to clipboard', () => {
cy.request('POST', '/reset', { todos })
cy.visit('/')
cy.get('li.todo').should('have.length', todos.length)
cy.get('[title="Copy todos to clipboard"]').click()
})
async copyTodos({ state }) {
const markdown =
state.todos
.map((todo) => {
const mark = todo.completed ? 'x' : ' '
return `- [${mark}] ${todo.title}`
})
.join('\n') + '\n'
await navigator.clipboard.writeText(markdown)
}// app.js
cy.window()
.its('navigator.clipboard')
.then((clipboard) => {
cy.stub(clipboard, 'writeText').as('writeText')
})
cy.get('[title="Copy todos to clipboard"]').click()
cy.get('@writeText').should(
'have.been.calledOnceWith',
Cypress.sinon.match.string,
)// cy
cy.window()
.its('navigator.clipboard')
.then((clipboard) => {
cy.stub(clipboard, 'writeText').as('writeText')
})
cy.get('[title="Copy todos to clipboard"]').click()
cy.get('@writeText').should(
'have.been.calledOnceWith',
Cypress.sinon.match.string,
)// cy
Fast Visual Useful Feedback
import { test, expect } from '@playwright/test'
test('adds todos', async ({ page, request }) => {
await page.goto('/')
// create a new todo locator
const newTodo = page.getByPlaceholder('What needs to be done?')
await newTodo.fill('one')
await newTodo.press('Enter')
await newTodo.fill('two')
await newTodo.press('Enter')
const todoItems = page.locator('li.todo')
await expect(todoItems).toHaveCount(2)
})// pw
npx cypress runHeadless Cypress test run
npx cypress runHeadless Cypress test run
npx cypress runHeadless Cypress test run
npx playwright test --trace onGet useful information about the Playwright test run
💪👏🎉
Playwright watch mode discussion
Playwright VSCode extension
playwright-watch wrapper
The Syntax
import { test, expect } from '@playwright/test'
test('adds todos', async ({ page, request }) => {
await request.post('/reset', { data: { todos: [] } })
await page.goto('/')
// create a new todo locator
const newTodo = page.getByPlaceholder('What needs to be done?')
await newTodo.fill('one')
await newTodo.press('Enter')
await newTodo.fill('two')
await newTodo.press('Enter')
const todoItems = page.locator('li.todo')
await expect(todoItems).toHaveCount(2)
})it('adds todos', () => {
cy.request('POST', '/reset', { todos: [] })
cy.visit('/')
cy.get('.new-todo').type('one{enter}').type('two{enter}')
cy.get('li.todo').should('have.length', 2)
})
// cy
The Syntax
// pw
- imperative syntax
- promise-based
- declarative syntax
- reactive stream
import { test, expect } from '@playwright/test'
test('adds todos', async ({ page, request }) => {
await request.post('/reset', { data: { todos: [] } })
await page.goto('/')
// create a new todo locator
const newTodo = page.getByPlaceholder('What needs to be done?')
await newTodo.fill('one')
await newTodo.press('Enter')
await newTodo.fill('two')
await newTodo.press('Enter')
const todoItems = page.locator('li.todo')
await expect(todoItems).toHaveCount(2)
})it('adds todos', () => {
cy.request('POST', '/reset', { todos: [] })
cy.visit('/')
cy.get('.new-todo').type('one{enter}').type('two{enter}')
cy.get('li.todo').should('have.length', 2)
})
// cy
The Syntax
// pw
- imperative syntax
- promise-based
- declarative syntax
- reactive stream
import { test, expect } from '@playwright/test'
test('adds todos', async ({ page, request }) => {
await request.post('/reset', { data: { todos: [] } })
await page.goto('/')
// create a new todo locator
const newTodo = page.getByPlaceholder('What needs to be done?')
await newTodo.fill('one')
await newTodo.press('Enter')
await newTodo.fill('two')
await newTodo.press('Enter')
const todoItems = page.locator('li.todo')
await expect(todoItems).toHaveCount(2)
})it('adds todos', () => {
cy.request('POST', '/reset', { todos: [] })
cy.visit('/')
cy.get('.new-todo').type('one{enter}').type('two{enter}')
cy.get('li.todo').should('have.length', 2)
})
// cy
The Syntax
// pw
- imperative syntax
- promise-based
- declarative syntax
- reactive stream
value (subject) flows through the commands and assertions
import { test, expect } from '@playwright/test'
test('adds todos', async ({ page, request }) => {
await request.post('/reset', { data: { todos: [] } })
await page.goto('/')
// create a new todo locator
const newTodo = page.getByPlaceholder('What needs to be done?')
await newTodo.fill('one')
await newTodo.press('Enter')
await newTodo.fill('two')
await newTodo.press('Enter')
const todoItems = page.locator('li.todo')
await expect(todoItems).toHaveCount(2)
})it('adds todos', () => {
cy.request('POST', '/reset', { todos: [] })
cy.visit('/')
cy.get('.new-todo').type('one{enter}').type('two{enter}')
cy.get('li.todo').should('have.length', 2)
})
// cy
The "Tinder" data flow
// pw
- imperative syntax
- promise-based
- declarative syntax
- reactive stream
Swipe right
(data flows to the right)
Swipe left
(data is assigned to the left)
var k = 0;
for(k = 0; k < numbers.length; k += 1) {
console.log(numbers[k] * constant);
}
// 6 2 14// _ is Lodash / Ramda
_(numbers)
.map(_.partial(mul, constant))
.forEach(print);
// 6 2 14// functional
for-loop vs Array.forEach
// imperative
Swipe right
(data flows to the right)
Swipe left
(data is assigned to the left)
const n = Number(await locator.getText())
// n is setcy.get('#count').then(n =>
// n is set
)// right
Data flows left vs right
// left
Swipe right
(data flows to the right)
Swipe left
(data is assigned to the left)
🤯 🤬
const n = Number(await locator.getText())
// n is setcy.get('#count').then(n =>
// n is set
)// right
Data flows left vs right
// left
🤯 🤬
What do you want to check on the page?
cy.get('li.todo').should('have.length', 2)There should be 2 items
cy.get('li.todo')
.should('satisfy', $li => $li.length % 2 === 0)There should be an even number of items
cy.wait('@load').its('response.body.length')
.then(n => {
cy.get('li.todo').should('have.length', n)
})There should be the same number of items as returned by the server
cy.intercept('GET', '/todos', { fixture: 'three.json' })
cy.get('li.todo').should('have.length', 3)If you can control the data in your tests, then the test syntax collapses into a simple and elegant fluent chain
$todo
"Good Cypress Test Syntax" https://www.youtube.com/watch?v=X8iIoTxu_8k
import 'cypress-aliases'
cy.wait('@load').its('response.body.length').as('n')
cy.get('li.todo').should('have.length', '@n')Tip: Simple declarative access to the aliased values via https://github.com/bahmutov/cypress-aliases plugin
Dear user,
- start with no todos
- visit the application page
- get the input field and type "one" + enter, "two" + enter
- there should be two Todo items
Dear user,
- cy.request('POST', '/reset', { todos: [] })
- visit the application page
- get the input field and type "one" + enter, "two" + enter
- there should be two Todo items
Dear user,
- cy.request('POST', '/reset', { todos: [] })
- cy.visit('/')
- get the input field and type "one" + enter, "two" + enter
- there should be two Todo items
Dear user,
- cy.request('POST', '/reset', { todos: [] })
- cy.visit('/')
- cy.get('.new-todo').type('one{enter}').type('two{enter}')
- there should be two Todo items
Dear user,
- cy.request('POST', '/reset', { todos: [] })
- cy.visit('/')
- cy.get('.new-todo').type('one{enter}').type('two{enter}')
- cy.get('li.todo').should('have.length', 2)
Dear cy,
- cy.request('POST', '/reset', { todos: [] })
- cy.visit('/')
- cy.get('.new-todo').type('one{enter}').type('two{enter}')
- cy.get('li.todo').should('have.length', 2)
Dear cy,
- cy.request('POST', '/reset', { todos: [] })
- cy.visit('/')
- cy.get('.new-todo').type('one{enter}').type('two{enter}')
- cy.get('li.todo').should('have.length', 2)
The tests should read naturally, almost like English instructions to a human tester
If the test looks weird or complicated...
cy.get('[title="Copy todos to clipboard"]').click()import 'cypress-real-events'
cy.get('[title="Copy todos to clipboard"]').realClick()
Node.js
Playwright test code
Browser
application
Chrome Debugger Protocol
Node.js
Config / plugins file
Browser
application
Cypress test code
WebSocket
for
cy.task
Test runners architecture
Chrome Debugger Protocol
// realClick
Cypress.automation("remote:debugger:protocol", {
command: 'Input.dispatchMouseEvent'
params: ...
})Use any Chrome Debugger Protocol command easily today https://github.com/bahmutov/cypress-cdp
⚠️ missing subscribing to Chrome events
"Cypress Automation"
Just for fun...
The Future
Playwright
- better E2E watch mode
- component testing (?)
- spying and stubbing methods
Playwright
- better E2E watch mode
- component testing (?)
- spying and stubbing methods
Cypress
- bring CDP and alias shortcuts into the core
core features
experimental features
plugins
external tools
cypress-real-events, cypress-aliases
Playwright
- better E2E watch mode
- component testing (?)
- spying and stubbing methods
Cypress
- bring CDP and alias shortcuts into the core
- saving traces
Playwright
- better E2E watch mode
- component testing (?)
- spying and stubbing methods
Cypress
- bring CDP and alias shortcuts into the core
- saving traces
- automatic traces comparison
X vs Y
People testing with test runner X
Cypress users?
No E2E tests
How do you pick a testing tool?
Study many
Pick the one that solves your problem
👏 Thank You 👏
Global
Testing
Summit
The Debate Between Cypress and Playwright
By Gleb Bahmutov
The Debate Between Cypress and Playwright
This session will discuss the two most popular modern web application testing tools in Cypress and Playwright. The two approach the same problems in very different ways — learn how to write end-to-end, API, and component tests using both. You'll come away understanding how to execute them on a continuous integration system. While similar comparisons of Cypress and Playwright focus on finding a winner, this session will go in-depth on the advantages of each tool to help you make an informed decision depending on your use case. Watch the video at https://youtu.be/lT9eif7YqUs
