The Debate Between Cypress and Playwright

Gleb Bahmutov

@bahmutov

Global

Testing

Summit

Join others to fight the climate crisis

Speaker: Gleb Bahmutov PhD

C / C++ / C# / Java / CoffeeScript / JavaScript / Node / Angular / Vue / Cycle.js / functional programming / testing,

former VP of Engineering at Cypress.io

Gleb Bahmutov

Sr Director of Engineering

I ❤️ Cypress

  • used it for 1 year before joining Cypress.io
  • VP of Eng, Distinguished Engineer at Cypress.io
  • implemented Cypress on Windows, parallelization, the paywall, component testing, API testing, and lots of other things
  • taught 100s of people at Cypress workshops
  • 500+ examples and recipes at https://glebbahmutov.com/cypress-examples/
  • 400+ videos about Cypress at https://www.youtube.com/glebbahmutov
  • I answer Cypress questions and help users

Do not compare yourself to other tools

CY vs PW

Book "A Frontend Web Developer's Guide to Testing" by Eran Kinsbruner

On Migrating from Cypress to Playwright

https://mtlynch.io/notes/cypress-vs-playwright/

Playwright vs Cypress: A Comparison

https://www.browserstack.com/guide/playwright-vs-cypress

Playwright vs. Cypress: Which Cross-Browser Testing Solution Is Right for You?

https://www.perfecto.io/blog/playwright-vs-cypress

Playwright vs Cypress: Which Framework to Choose For E2E Testing?

https://labs.eleks.com/2022/07/playwright-vs-cypress-e2e-testing.html

Cypress vs Playwright: Let the Code Speak

https://applitools.com/blog/cypress-vs-playwright/

Cypress vs Playwright: What’s the Best Test Automation Framework for Your Project?

https://www.kellton.com/kellton-tech-blog/cypress-vs-playwright-what-is-the-best-test-automation-framework

Cypress vs. Playwright: end-to-end testing showdown

https://silvenon.com/blog/e2e-testing-with-cypress-vs-playwright

CY vs PW

On Migrating from Cypress to Playwright

https://mtlynch.io/notes/cypress-vs-playwright/

Playwright vs Cypress: A Comparison

https://www.browserstack.com/guide/playwright-vs-cypress

Playwright vs. Cypress: Which Cross-Browser Testing Solution Is Right for You?

https://www.perfecto.io/blog/playwright-vs-cypress

Playwright vs Cypress: Which Framework to Choose For E2E Testing?

https://labs.eleks.com/2022/07/playwright-vs-cypress-e2e-testing.html

Cypress vs Playwright: Let the Code Speak

https://applitools.com/blog/cypress-vs-playwright/

Cypress vs Playwright: What’s the Best Test Automation Framework for Your Project?

https://www.kellton.com/kellton-tech-blog/cypress-vs-playwright-what-is-the-best-test-automation-framework

Cypress vs. Playwright: end-to-end testing showdown

https://silvenon.com/blog/e2e-testing-with-cypress-vs-playwright

Book "A Frontend Web Developer's Guide to Testing" by Eran Kinsbruner

👍

👍

Cypress vs Playwright: What’s the Best Test Automation Framework for Your Project?

https://www.kellton.com/kellton-tech-blog/cypress-vs-playwright-what-is-the-best-test-automation-framework

  • Out of date

Any comparison older than one month

  • Pineapples to orange soda

/* Run your local dev server before starting the tests */
webServer: {
  command: 'npm run start',
  port: 3000,
}

// pw

cy.visit('index.html')

// cy

In Cypress

// DOES NOT WORK
page.goto('index.html')

// pw

Is this accurate?

how important is this for us?

should it be a core feature?

Cypress design principle

Be a cohesive testing tool, allow new features to be implemented as plugins

core features

experimental features

plugins

external tools

Cypress design principle

core features

experimental features

plugins

external tools

start-server-and-test

Component

testing

Cypress design principle

core features

experimental features

plugins

external tools

start-server-and-test

Component

testing

Cypress design principle

core features

experimental features

plugins

external tools

start-server-and-test

Component

testing

Cypress design principle

core features

experimental features

plugins

external tools

start-server-and-test

Component

testing

cy.ng()

Plugins everywhere (145+ at https://on.cypress.io/plugins)

"Cypress Plugins" paid course 🎓 💵

https://cypress.tips/courses/cypress-plugins

import { test } from '@playwright/test'

test('logs hello', async () => {
  console.log('hello, world')
})
it('logs hello', () => {
  console.log('hello, world')
})

// pw

// cy

// pw

// cy

Cypress iframes the app and its specs

Node.js

Playwright test code

Browser

application

Chrome Debugger Protocol

Node.js

Config / plugins file

Browser

application

Cypress test code

WebSocket

for

cy.task

Test runners architecture

it('listens to the window.postMessage events', () => {
  cy.visit('index.html', {
    onBeforeLoad(win) {
      cy.spy(win, 'postMessage').as('postMessage')
    },
  })
  cy.get('@postMessage')
    .should('have.been.calledTwice')
    .and('have.been.calledWithExactly', 'one')
    .and('have.been.calledWithExactly', 'two')
})

Direct access to the application's objects and browser APIs from the test code

💪

it('collects the window.postMessage events', () => {
  const messages = []
  cy.visit('index.html', {
    onBeforeLoad(win) {
      win.addEventListener('message', (e) => {
        messages.push(e.data)
      })
    },
  })
  cy.wrap(messages).should('have.length', 2)
  	.its(0).should('equal', 'one')
})

Direct access to the application's objects and browser APIs from the test code

💪

Bypass problematic methods from

the test

import { todos } from '../fixtures/three.json'

it('copies the todos to clipboard', () => {
  cy.request('POST', '/reset', { todos })
  cy.visit('/')
  cy.get('li.todo').should('have.length', todos.length)
  cy.get('[title="Copy todos to clipboard"]').click()
})
async copyTodos({ state }) {
  const markdown =
    state.todos
      .map((todo) => {
        const mark = todo.completed ? 'x' : ' '
        return `- [${mark}] ${todo.title}`
      })
      .join('\n') + '\n'
  await navigator.clipboard.writeText(markdown)
}

// app.js

cy.window()
  .its('navigator.clipboard')
  .then((clipboard) => {
    cy.stub(clipboard, 'writeText').as('writeText')
  })
cy.get('[title="Copy todos to clipboard"]').click()
cy.get('@writeText').should(
  'have.been.calledOnceWith',
  Cypress.sinon.match.string,
)

// cy

cy.window()
  .its('navigator.clipboard')
  .then((clipboard) => {
    cy.stub(clipboard, 'writeText').as('writeText')
  })
cy.get('[title="Copy todos to clipboard"]').click()
cy.get('@writeText').should(
  'have.been.calledOnceWith',
  Cypress.sinon.match.string,
)

// cy

Fast Visual Useful Feedback

import { test, expect } from '@playwright/test'

test('adds todos', async ({ page, request }) => {
  await page.goto('/')
  // create a new todo locator
  const newTodo = page.getByPlaceholder('What needs to be done?')
  await newTodo.fill('one')
  await newTodo.press('Enter')
  await newTodo.fill('two')
  await newTodo.press('Enter')
  const todoItems = page.locator('li.todo')
  await expect(todoItems).toHaveCount(2)
})

// pw

npx cypress run

Headless Cypress test run

npx cypress run

Headless Cypress test run

npx cypress run

Headless Cypress test run

npx playwright test --trace on

Get useful information about the Playwright test run

💪👏🎉

Playwright watch mode discussion

https://github.com/microsoft/playwright/issues/7035

Playwright VSCode extension

playwright-watch wrapper

The Syntax

import { test, expect } from '@playwright/test'

test('adds todos', async ({ page, request }) => {
  await request.post('/reset', { data: { todos: [] } })
  await page.goto('/')
  // create a new todo locator
  const newTodo = page.getByPlaceholder('What needs to be done?')
  await newTodo.fill('one')
  await newTodo.press('Enter')
  await newTodo.fill('two')
  await newTodo.press('Enter')
  const todoItems = page.locator('li.todo')
  await expect(todoItems).toHaveCount(2)
})
it('adds todos', () => {
  cy.request('POST', '/reset', { todos: [] })
  cy.visit('/')
  cy.get('.new-todo').type('one{enter}').type('two{enter}')
  cy.get('li.todo').should('have.length', 2)
})

// cy

The Syntax

// pw

  • imperative syntax
  • promise-based
  • declarative syntax
  • reactive stream
import { test, expect } from '@playwright/test'

test('adds todos', async ({ page, request }) => {
  await request.post('/reset', { data: { todos: [] } })
  await page.goto('/')
  // create a new todo locator
  const newTodo = page.getByPlaceholder('What needs to be done?')
  await newTodo.fill('one')
  await newTodo.press('Enter')
  await newTodo.fill('two')
  await newTodo.press('Enter')
  const todoItems = page.locator('li.todo')
  await expect(todoItems).toHaveCount(2)
})
it('adds todos', () => {
  cy.request('POST', '/reset', { todos: [] })
  cy.visit('/')
  cy.get('.new-todo').type('one{enter}').type('two{enter}')
  cy.get('li.todo').should('have.length', 2)
})

// cy

The Syntax

// pw

  • imperative syntax
  • promise-based
  • declarative syntax
  • reactive stream
import { test, expect } from '@playwright/test'

test('adds todos', async ({ page, request }) => {
  await request.post('/reset', { data: { todos: [] } })
  await page.goto('/')
  // create a new todo locator
  const newTodo = page.getByPlaceholder('What needs to be done?')
  await newTodo.fill('one')
  await newTodo.press('Enter')
  await newTodo.fill('two')
  await newTodo.press('Enter')
  const todoItems = page.locator('li.todo')
  await expect(todoItems).toHaveCount(2)
})
it('adds todos', () => {
  cy.request('POST', '/reset', { todos: [] })
  cy.visit('/')
  cy.get('.new-todo').type('one{enter}').type('two{enter}')
  cy.get('li.todo').should('have.length', 2)
})

// cy

The Syntax

// pw

  • imperative syntax
  • promise-based
  • declarative syntax
  • reactive stream

value (subject) flows through the commands and assertions

import { test, expect } from '@playwright/test'

test('adds todos', async ({ page, request }) => {
  await request.post('/reset', { data: { todos: [] } })
  await page.goto('/')
  // create a new todo locator
  const newTodo = page.getByPlaceholder('What needs to be done?')
  await newTodo.fill('one')
  await newTodo.press('Enter')
  await newTodo.fill('two')
  await newTodo.press('Enter')
  const todoItems = page.locator('li.todo')
  await expect(todoItems).toHaveCount(2)
})
it('adds todos', () => {
  cy.request('POST', '/reset', { todos: [] })
  cy.visit('/')
  cy.get('.new-todo').type('one{enter}').type('two{enter}')
  cy.get('li.todo').should('have.length', 2)
})

// cy

The "Tinder" data flow

// pw

  • imperative syntax
  • promise-based
  • declarative syntax
  • reactive stream

Swipe right

(data flows to the right)

Swipe left

(data is assigned to the left)

var k = 0;
for(k = 0; k < numbers.length; k += 1) {
  console.log(numbers[k] * constant);
}
// 6 2 14
// _ is Lodash / Ramda
_(numbers)
  .map(_.partial(mul, constant))
  .forEach(print);
// 6 2 14

// functional

for-loop vs Array.forEach

// imperative

Swipe right

(data flows to the right)

Swipe left

(data is assigned to the left)

const n = Number(await locator.getText())
// n is set
cy.get('#count').then(n => 
  // n is set
)

// right

Data flows left vs right

// left

Swipe right

(data flows to the right)

Swipe left

(data is assigned to the left)

🤯 🤬

const n = Number(await locator.getText())
// n is set
cy.get('#count').then(n => 
  // n is set
)

// right

Data flows left vs right

// left

🤯 🤬

What do you want to check on the page?

cy.get('li.todo').should('have.length', 2)
There should be 2 items
cy.get('li.todo')
  .should('satisfy', $li => $li.length % 2 === 0)
There should be an even number of items
cy.wait('@load').its('response.body.length')
  .then(n => {
    cy.get('li.todo').should('have.length', n)
  })
There should be the same number of items as returned by the server
cy.intercept('GET', '/todos', { fixture: 'three.json' })
cy.get('li.todo').should('have.length', 3)

If you can control the data in your tests, then the test syntax collapses into a simple and elegant fluent chain

$todo

"Good Cypress Test Syntax"   https://www.youtube.com/watch?v=X8iIoTxu_8k

import 'cypress-aliases'

cy.wait('@load').its('response.body.length').as('n')
cy.get('li.todo').should('have.length', '@n')

Tip: Simple declarative access to the aliased values via https://github.com/bahmutov/cypress-aliases plugin

Dear user,

  • start with no todos
  • visit the application page
  • get the input field and type "one" + enter, "two" + enter
  • there should be two Todo items

Dear user,

  • cy.request('POST', '/reset', { todos: [] })
  • visit the application page
  • get the input field and type "one" + enter, "two" + enter
  • there should be two Todo items

Dear user,

  • cy.request('POST', '/reset', { todos: [] })
  • cy.visit('/')
  • get the input field and type "one" + enter, "two" + enter
  • there should be two Todo items

Dear user,

  • cy.request('POST', '/reset', { todos: [] })
  • cy.visit('/')
  • cy.get('.new-todo').type('one{enter}').type('two{enter}')
  • there should be two Todo items

Dear user,

  • cy.request('POST', '/reset', { todos: [] })
  • cy.visit('/')
  • cy.get('.new-todo').type('one{enter}').type('two{enter}')
  • cy.get('li.todo').should('have.length', 2)

Dear cy,

  • cy.request('POST', '/reset', { todos: [] })
  • cy.visit('/')
  • cy.get('.new-todo').type('one{enter}').type('two{enter}')
  • cy.get('li.todo').should('have.length', 2)

Dear cy,

  • cy.request('POST', '/reset', { todos: [] })
  • cy.visit('/')
  • cy.get('.new-todo').type('one{enter}').type('two{enter}')
  • cy.get('li.todo').should('have.length', 2)

The tests should read naturally, almost like English instructions to a human tester

If the test looks weird or complicated...

cy.get('[title="Copy todos to clipboard"]').click()
import 'cypress-real-events'

cy.get('[title="Copy todos to clipboard"]').realClick()

Node.js

Playwright test code

Browser

application

Chrome Debugger Protocol

Node.js

Config / plugins file

Browser

application

Cypress test code

WebSocket

for

cy.task

Test runners architecture

Chrome Debugger Protocol

// realClick
Cypress.automation("remote:debugger:protocol", {
  command: 'Input.dispatchMouseEvent'
  params: ...
})

Use any Chrome Debugger Protocol command easily today https://github.com/bahmutov/cypress-cdp

⚠️ missing subscribing to Chrome events

Just for fun...

The Future

Playwright

  • better E2E watch mode
  • component testing (?)
  • spying and stubbing methods

Playwright

  • better E2E watch mode
  • component testing (?)
  • spying and stubbing methods

Cypress

  • bring CDP and alias shortcuts into the core

core features

experimental features

plugins

external tools

cypress-real-events, cypress-aliases

Playwright

  • better E2E watch mode
  • component testing (?)
  • spying and stubbing methods

Cypress

  • bring CDP and alias shortcuts into the core
  • saving traces

Playwright

  • better E2E watch mode
  • component testing (?)
  • spying and stubbing methods

Cypress

  • bring CDP and alias shortcuts into the core
  • saving traces
  • automatic traces comparison

X vs Y

People testing with test runner X

Cypress users?

No E2E tests

How do you pick a testing tool?

Study many

Pick the one that solves your problem

👏 Thank You 👏

Global

Testing

Summit

The Debate Between Cypress and Playwright

By Gleb Bahmutov

The Debate Between Cypress and Playwright

This session will discuss the two most popular modern web application testing tools in Cypress and Playwright. The two approach the same problems in very different ways — learn how to write end-to-end, API, and component tests using both. You'll come away understanding how to execute them on a continuous integration system. While similar comparisons of Cypress and Playwright focus on finding a winner, this session will go in-depth on the advantages of each tool to help you make an informed decision depending on your use case. Watch the video at https://youtu.be/lT9eif7YqUs

  • 623