The Debate Between Cypress and Playwright
Gleb Bahmutov
@bahmutov
Global
Testing
Summit
Join others to fight the climate crisis
Speaker: Gleb Bahmutov PhD
C / C++ / C# / Java / CoffeeScript / JavaScript / Node / Angular / Vue / Cycle.js / functional programming / testing,
former VP of Engineering at Cypress.io
Gleb Bahmutov
Sr Director of Engineering
I ❤️ Cypress
- used it for 1 year before joining Cypress.io
- VP of Eng, Distinguished Engineer at Cypress.io
- implemented Cypress on Windows, parallelization, the paywall, component testing, API testing, and lots of other things
- taught 100s of people at Cypress workshops
- 500+ examples and recipes at https://glebbahmutov.com/cypress-examples/
- 400+ videos about Cypress at https://www.youtube.com/glebbahmutov
- I answer Cypress questions and help users
Do not compare yourself to other tools
CY vs PW
Book "A Frontend Web Developer's Guide to Testing" by Eran Kinsbruner
Cypress vs Playwright – Clash of the Titans
https://www.zeljkovic.sh/cypress-vs-playwright-clash-of-the-titans-introduction/
On Migrating from Cypress to Playwright
Playwright vs Cypress: A Comparison
Playwright vs. Cypress: Which Cross-Browser Testing Solution Is Right for You?
Playwright vs Cypress: Which Framework to Choose For E2E Testing?
https://labs.eleks.com/2022/07/playwright-vs-cypress-e2e-testing.html
Five reasons why Playwright is better than Cypress
https://alisterbscott.com/2021/10/27/five-reasons-why-playwright-is-better-than-cypress/
Playwright vs Cypress
Cypress vs Playwright: Let the Code Speak
Playwright vs Cypress E2E Testing
https://github.com/cliffordfajardo/cypress-vs-playwright-comparison
Cypress vs Playwright: What’s the Best Test Automation Framework for Your Project?
Cypress vs. Playwright: end-to-end testing showdown
https://silvenon.com/blog/e2e-testing-with-cypress-vs-playwright
CY vs PW
On Migrating from Cypress to Playwright
Playwright vs Cypress: A Comparison
Playwright vs. Cypress: Which Cross-Browser Testing Solution Is Right for You?
Playwright vs Cypress: Which Framework to Choose For E2E Testing?
https://labs.eleks.com/2022/07/playwright-vs-cypress-e2e-testing.html
Five reasons why Playwright is better than Cypress
https://alisterbscott.com/2021/10/27/five-reasons-why-playwright-is-better-than-cypress/
Playwright vs Cypress
Cypress vs Playwright: Let the Code Speak
Playwright vs Cypress E2E Testing
https://github.com/cliffordfajardo/cypress-vs-playwright-comparison
Cypress vs Playwright: What’s the Best Test Automation Framework for Your Project?
Cypress vs. Playwright: end-to-end testing showdown
https://silvenon.com/blog/e2e-testing-with-cypress-vs-playwright
Book "A Frontend Web Developer's Guide to Testing" by Eran Kinsbruner
Cypress vs Playwright – Clash of the Titans
https://www.zeljkovic.sh/cypress-vs-playwright-clash-of-the-titans-introduction/
👍
👍
Cypress vs Playwright: What’s the Best Test Automation Framework for Your Project?
-
Out of date
Any comparison older than one month
-
Pineapples to orange soda
/* Run your local dev server before starting the tests */
webServer: {
command: 'npm run start',
port: 3000,
}// pw
cy.visit('index.html')// cy
In Cypress
// DOES NOT WORK
page.goto('index.html')
// pw
Is this accurate?
how important is this for us?
should it be a core feature?
Cypress design principle
Be a cohesive testing tool, allow new features to be implemented as plugins
core features
experimental features
plugins
external tools
Cypress design principle
core features
experimental features
plugins
external tools
start-server-and-test
Component
testing
Cypress design principle
core features
experimental features
plugins
external tools
start-server-and-test
Component
testing
Cypress design principle
core features
experimental features
plugins
external tools
start-server-and-test
Component
testing
Cypress design principle
core features
experimental features
plugins
external tools
start-server-and-test
Component
testing
cy.ng()
Plugins everywhere (145+ at https://on.cypress.io/plugins)
"My favorite Cypress plugins"
https://cypresstips.substack.com/p/my-favorite-cypress-plugins
"My favorite Cypress plugins, part II"
https://cypresstips.substack.com/p/my-favorite-cypress-plugins-part
"Cypress Plugins" paid course 🎓 💵
import { test } from '@playwright/test'
test('logs hello', async () => {
console.log('hello, world')
})it('logs hello', () => {
console.log('hello, world')
})// pw
// cy
// pw
// cy
Cypress iframes the app and its specs
Node.js
Playwright test code
Browser
application
Chrome Debugger Protocol
Node.js
Config / plugins file
Browser
application
Cypress test code
WebSocket
for
cy.task
Test runners architecture
it('listens to the window.postMessage events', () => {
cy.visit('index.html', {
onBeforeLoad(win) {
cy.spy(win, 'postMessage').as('postMessage')
},
})
cy.get('@postMessage')
.should('have.been.calledTwice')
.and('have.been.calledWithExactly', 'one')
.and('have.been.calledWithExactly', 'two')
})Direct access to the application's objects and browser APIs from the test code
💪
it('collects the window.postMessage events', () => {
const messages = []
cy.visit('index.html', {
onBeforeLoad(win) {
win.addEventListener('message', (e) => {
messages.push(e.data)
})
},
})
cy.wrap(messages).should('have.length', 2)
.its(0).should('equal', 'one')
})Direct access to the application's objects and browser APIs from the test code
💪
Bypass problematic methods from
the test
import { todos } from '../fixtures/three.json'
it('copies the todos to clipboard', () => {
cy.request('POST', '/reset', { todos })
cy.visit('/')
cy.get('li.todo').should('have.length', todos.length)
cy.get('[title="Copy todos to clipboard"]').click()
})
async copyTodos({ state }) {
const markdown =
state.todos
.map((todo) => {
const mark = todo.completed ? 'x' : ' '
return `- [${mark}] ${todo.title}`
})
.join('\n') + '\n'
await navigator.clipboard.writeText(markdown)
}// app.js
cy.window()
.its('navigator.clipboard')
.then((clipboard) => {
cy.stub(clipboard, 'writeText').as('writeText')
})
cy.get('[title="Copy todos to clipboard"]').click()
cy.get('@writeText').should(
'have.been.calledOnceWith',
Cypress.sinon.match.string,
)// cy
cy.window()
.its('navigator.clipboard')
.then((clipboard) => {
cy.stub(clipboard, 'writeText').as('writeText')
})
cy.get('[title="Copy todos to clipboard"]').click()
cy.get('@writeText').should(
'have.been.calledOnceWith',
Cypress.sinon.match.string,
)// cy
Fast Visual Useful Feedback
import { test, expect } from '@playwright/test'
test('adds todos', async ({ page, request }) => {
await page.goto('/')
// create a new todo locator
const newTodo = page.getByPlaceholder('What needs to be done?')
await newTodo.fill('one')
await newTodo.press('Enter')
await newTodo.fill('two')
await newTodo.press('Enter')
const todoItems = page.locator('li.todo')
await expect(todoItems).toHaveCount(2)
})// pw
npx cypress runHeadless Cypress test run
npx cypress runHeadless Cypress test run
npx cypress runHeadless Cypress test run
npx playwright test --trace onGet useful information about the Playwright test run
💪👏🎉
Playwright watch mode discussion
Playwright VSCode extension
playwright-watch wrapper
The Syntax
import { test, expect } from '@playwright/test'
test('adds todos', async ({ page, request }) => {
await request.post('/reset', { data: { todos: [] } })
await page.goto('/')
// create a new todo locator
const newTodo = page.getByPlaceholder('What needs to be done?')
await newTodo.fill('one')
await newTodo.press('Enter')
await newTodo.fill('two')
await newTodo.press('Enter')
const todoItems = page.locator('li.todo')
await expect(todoItems).toHaveCount(2)
})it('adds todos', () => {
cy.request('POST', '/reset', { todos: [] })
cy.visit('/')
cy.get('.new-todo').type('one{enter}').type('two{enter}')
cy.get('li.todo').should('have.length', 2)
})
// cy
The Syntax
// pw
- imperative syntax
- promise-based
- declarative syntax
- reactive stream
import { test, expect } from '@playwright/test'
test('adds todos', async ({ page, request }) => {
await request.post('/reset', { data: { todos: [] } })
await page.goto('/')
// create a new todo locator
const newTodo = page.getByPlaceholder('What needs to be done?')
await newTodo.fill('one')
await newTodo.press('Enter')
await newTodo.fill('two')
await newTodo.press('Enter')
const todoItems = page.locator('li.todo')
await expect(todoItems).toHaveCount(2)
})it('adds todos', () => {
cy.request('POST', '/reset', { todos: [] })
cy.visit('/')
cy.get('.new-todo').type('one{enter}').type('two{enter}')
cy.get('li.todo').should('have.length', 2)
})
// cy
The Syntax
// pw
- imperative syntax
- promise-based
- declarative syntax
- reactive stream
import { test, expect } from '@playwright/test'
test('adds todos', async ({ page, request }) => {
await request.post('/reset', { data: { todos: [] } })
await page.goto('/')
// create a new todo locator
const newTodo = page.getByPlaceholder('What needs to be done?')
await newTodo.fill('one')
await newTodo.press('Enter')
await newTodo.fill('two')
await newTodo.press('Enter')
const todoItems = page.locator('li.todo')
await expect(todoItems).toHaveCount(2)
})it('adds todos', () => {
cy.request('POST', '/reset', { todos: [] })
cy.visit('/')
cy.get('.new-todo').type('one{enter}').type('two{enter}')
cy.get('li.todo').should('have.length', 2)
})
// cy
The Syntax
// pw
- imperative syntax
- promise-based
- declarative syntax
- reactive stream
value (subject) flows through the commands and assertions
import { test, expect } from '@playwright/test'
test('adds todos', async ({ page, request }) => {
await request.post('/reset', { data: { todos: [] } })
await page.goto('/')
// create a new todo locator
const newTodo = page.getByPlaceholder('What needs to be done?')
await newTodo.fill('one')
await newTodo.press('Enter')
await newTodo.fill('two')
await newTodo.press('Enter')
const todoItems = page.locator('li.todo')
await expect(todoItems).toHaveCount(2)
})it('adds todos', () => {
cy.request('POST', '/reset', { todos: [] })
cy.visit('/')
cy.get('.new-todo').type('one{enter}').type('two{enter}')
cy.get('li.todo').should('have.length', 2)
})
// cy
The "Tinder" data flow
// pw
- imperative syntax
- promise-based
- declarative syntax
- reactive stream
Swipe right
(data flows to the right)
Swipe left
(data is assigned to the left)
var k = 0;
for(k = 0; k < numbers.length; k += 1) {
console.log(numbers[k] * constant);
}
// 6 2 14// _ is Lodash / Ramda
_(numbers)
.map(_.partial(mul, constant))
.forEach(print);
// 6 2 14// functional
for-loop vs Array.forEach
// imperative
Swipe right
(data flows to the right)
Swipe left
(data is assigned to the left)
const n = Number(await locator.getText())
// n is setcy.get('#count').then(n =>
// n is set
)// right
Data flows left vs right
// left
Swipe right
(data flows to the right)
Swipe left
(data is assigned to the left)
🤯 🤬
const n = Number(await locator.getText())
// n is setcy.get('#count').then(n =>
// n is set
)// right
Data flows left vs right
// left
🤯 🤬
What do you want to check on the page?
cy.get('li.todo').should('have.length', 2)There should be 2 items
cy.get('li.todo')
.should('satisfy', $li => $li.length % 2 === 0)There should be an even number of items
cy.wait('@load').its('response.body.length')
.then(n => {
cy.get('li.todo').should('have.length', n)
})There should be the same number of items as returned by the server
cy.intercept('GET', '/todos', { fixture: 'three.json' })
cy.get('li.todo').should('have.length', 3)If you can control the data in your tests, then the test syntax collapses into a simple and elegant fluent chain
$todo
"Good Cypress Test Syntax" https://www.youtube.com/watch?v=X8iIoTxu_8k
import 'cypress-aliases'
cy.wait('@load').its('response.body.length').as('n')
cy.get('li.todo').should('have.length', '@n')Tip: Simple declarative access to the aliased values via https://github.com/bahmutov/cypress-aliases plugin
Dear user,
- start with no todos
- visit the application page
- get the input field and type "one" + enter, "two" + enter
- there should be two Todo items
Dear user,
- cy.request('POST', '/reset', { todos: [] })
- visit the application page
- get the input field and type "one" + enter, "two" + enter
- there should be two Todo items
Dear user,
- cy.request('POST', '/reset', { todos: [] })
- cy.visit('/')
- get the input field and type "one" + enter, "two" + enter
- there should be two Todo items
Dear user,
- cy.request('POST', '/reset', { todos: [] })
- cy.visit('/')
- cy.get('.new-todo').type('one{enter}').type('two{enter}')
- there should be two Todo items
Dear user,
- cy.request('POST', '/reset', { todos: [] })
- cy.visit('/')
- cy.get('.new-todo').type('one{enter}').type('two{enter}')
- cy.get('li.todo').should('have.length', 2)
Dear cy,
- cy.request('POST', '/reset', { todos: [] })
- cy.visit('/')
- cy.get('.new-todo').type('one{enter}').type('two{enter}')
- cy.get('li.todo').should('have.length', 2)
Dear cy,
- cy.request('POST', '/reset', { todos: [] })
- cy.visit('/')
- cy.get('.new-todo').type('one{enter}').type('two{enter}')
- cy.get('li.todo').should('have.length', 2)
The tests should read naturally, almost like English instructions to a human tester
If the test looks weird or complicated...
cy.get('[title="Copy todos to clipboard"]').click()import 'cypress-real-events'
cy.get('[title="Copy todos to clipboard"]').realClick()
Node.js
Playwright test code
Browser
application
Chrome Debugger Protocol
Node.js
Config / plugins file
Browser
application
Cypress test code
WebSocket
for
cy.task
Test runners architecture
Chrome Debugger Protocol
// realClick
Cypress.automation("remote:debugger:protocol", {
command: 'Input.dispatchMouseEvent'
params: ...
})Use any Chrome Debugger Protocol command easily today https://github.com/bahmutov/cypress-cdp
⚠️ missing subscribing to Chrome events
"Cypress Automation"
Just for fun...
The Future
Playwright
- better E2E watch mode
- component testing (?)
- spying and stubbing methods
Playwright
- better E2E watch mode
- component testing (?)
- spying and stubbing methods
Cypress
- bring CDP and alias shortcuts into the core
core features
experimental features
plugins
external tools
cypress-real-events, cypress-aliases
Playwright
- better E2E watch mode
- component testing (?)
- spying and stubbing methods
Cypress
- bring CDP and alias shortcuts into the core
- saving traces
Playwright
- better E2E watch mode
- component testing (?)
- spying and stubbing methods
Cypress
- bring CDP and alias shortcuts into the core
- saving traces
- automatic traces comparison
X vs Y
People testing with test runner X
Cypress users?
No E2E tests