Cypress: Beyond the "Hello World" Test

Gleb Bahmutov

VP of Engineering Distinguished Engineer

Cypress.io 

@bahmutov

https://slides.com/bahmutov/cypress-beyond-the-hello-world

Q&A at Slido.com #beyond-hello

our planet is in imminent danger

https://lizkeogh.com/2019/07/02/off-the-charts/

+3 degrees Celsius will be the end.

@bahmutov

survival is possible* but we need to act now

  • change your life
  • join an organization

@bahmutov

rebellion.global          350.org

How long have you used Cypress?

If I were learning Cypress ...

  1. Read https://on.cypress.io/intro
  2. Watch Cypress videos https://on.cypress.io/tutorials
  3. Watch free courses at https://on.cypress.io/courses
  4. Watch Cypress webinars https://youtube.com/cypress_io
  5. Read the docs for every command used and linked resources
  6. Read the blog posts

Agenda

Q & A at Slido.com event code #beyond-hello

Cypress Tests

Run

In The

Browser!

// cypress/integration/spec.js
// NOT GOING TO WORK
const fs = require('fs')
fs.readFileSync(...)

⛔️ Cannot simply access the file system

Cypress Architecture

Access Node and OS

  • cy.readFile
  • cy.writeFile
  • cy.task
  • cy.exec

retries

does not retry

most powerful

most OS-specific

cy.readFile

it('retries reading the JSON file', () => {
  cy.get('.new-todo')
    .type('todo A{enter}')
    .type('todo B{enter}')
    .type('todo C{enter}')
    .type('todo D{enter}')
  cy.readFile('./todomvc/data.json').should(data => {
    expect(data).to.have.property('todos')
    expect(data.todos).to.have.length(4, '4 saved items')
    expect(data.todos[0], 'first item').to.include({
      title: 'todo A',
      completed: false
    })
  })
})

cy.readFile

it('retries reading the JSON file', () => {
  cy.get('.new-todo')
    .type('todo A{enter}')
    .type('todo B{enter}')
    .type('todo C{enter}')
    .type('todo D{enter}')
  cy.readFile('./todomvc/data.json').should(data => {
    expect(data).to.have.property('todos')
    expect(data.todos).to.have.length(4, '4 saved items')
    expect(data.todos[0], 'first item').to.include({
      title: 'todo A',
      completed: false
    })
  })
})

cy.readFile is retried until .should(cb) passes

cy.readFile

it('retries reading the JSON file', () => {
  cy.get('.new-todo')
    .type('todo A{enter}')
    .type('todo B{enter}')
    .type('todo C{enter}')
    .type('todo D{enter}')
  cy.readFile('./todomvc/data.json').should(data => {
    expect(data).to.have.property('todos')
    expect(data.todos).to.have.length(4, '4 saved items')
    expect(data.todos[0], 'first item').to.include({
      title: 'todo A',
      completed: false
    })
  })
})

fast app

cy.readFile

it('retries reading the JSON file', () => {
  cy.get('.new-todo')
    .type('todo A{enter}')
    .type('todo B{enter}')
    .type('todo C{enter}')
    .type('todo D{enter}')
  cy.readFile('./todomvc/data.json').should(data => {
    expect(data).to.have.property('todos')
    expect(data.todos).to.have.length(4, '4 saved items')
    expect(data.todos[0], 'first item').to.include({
      title: 'todo A',
      completed: false
    })
  })
})

Text

slow app

Let's look at cy.task

How do I reset or query a database from my test?

it('starts with an empty list', () => {
  cy.request('/persons').its('body').should('deep.equal', [])
})

Passes every time

Q & A at Slido.com event code #beyond-hello

it('starts with an empty list', () => {
  cy.request('/persons').its('body').should('deep.equal', [])
})

Passes every time

it('adds an actor without any movies', () => {
  cy.request('POST', '/persons', {
    firstName: 'Joe',
    lastName: 'Smith',
  })
  // now there should be 1 actor
  cy.request('/persons')
    .its('body')
    .should('have.length', 1)
    .its('0')
    .should('include', {
      firstName: 'Joe',
      lastName: 'Smith',
    })
})

fails after the second time

Problems:

  • The order of tests
  • The initial state

Reset the DB table

// cypress/plugins/index.js
module.exports = (on, config) => {
  on('task', {
    resetPeopleTable() {
      // TODO: clear the table
    },
  })
}

Tip: connect to DB using same code and logic as the app

// cypress/plugins/index.js
const Knex = require('knex')
const knexConfig = require('../../knexfile')
const { Model } = require('objection')
const knex = Knex(knexConfig.development)
Model.knex(knex)
const Person = require('../../models/Person')

/**
 * @type {Cypress.PluginConfig}
 */
module.exports = (on, config) => {
  on('task', {
    resetPeopleTable() {
      console.log('reset People table')
      return Person.query().truncate()
    },
  })
}

Application code

// cypress/plugins/index.js
const Knex = require('knex')
const knexConfig = require('../../knexfile')
const { Model } = require('objection')
const knex = Knex(knexConfig.development)
Model.knex(knex)
const Person = require('../../models/Person')

/**
 * @type {Cypress.PluginConfig}
 */
module.exports = (on, config) => {
  on('task', {
    resetPeopleTable() {
      console.log('reset People table')
      return Person.query().truncate()
    },
  })
}
$ node -v
v12.14.1
$ npm install
...
$ ls node_modules/sqlite3/lib/binding
node-v72-darwin-x64

Database module installed native DB binding for Node v12.14.1

$ npx cypress version
Cypress package version: 6.2.1
Cypress binary version: 6.2.1
Electron version: 11.1.1
Bundled Node version: 12.18.3

By default plugins file runs using the Node bundled with Cypress

Solution: set Cypress to use the system Node to run the plugins file

// cypress/plugins/index.js
const Knex = require('knex')
const knexConfig = require('../../knexfile')
const { Model } = require('objection')
const knex = Knex(knexConfig.development)
Model.knex(knex)
const Person = require('../../models/Person')

/**
 * @type {Cypress.PluginConfig}
 */
module.exports = (on, config) => {
  on('task', {
    resetPeopleTable() {
      console.log('reset People table')
      return Person.query().truncate()
    },
  })
}

terminal

Passing Tests

Fetch Person by ID

// api.js
router.get('/', ...)
router.post('/persons', ...)
router.get('/persons', ...)
router.patch('/persons/:id', ...)
router.delete('/persons/:id', ...)
...

REST API does not have a method "GET /persons/:id" 🧐

Q & A at Slido.com event code #beyond-hello

it('updates Person information', () => {
  cy.request('POST', '/persons', {
    firstName: 'Joe',
    lastName: 'Smith',
  })
    .its('body.id')
    .then((id) => {
      expect(id, 'created id').to.be.a('number')
      cy.request({
        method: 'PATCH',
        url: `/persons/${id}`,
        body: {
          lastName: 'Pesci',
        },
      })
        .its('body')
        .should('deep.equal', {
          success: true,
        })
      // how to check the full record with id: 1?
    })
})

⛔️ Cannot add more methods to the REST API

✅ Can query the database using the cy.task

const Person = require('../../models/Person')

module.exports = (on, config) => {
  on('task', {
    resetPeopleTable() {
      console.log('reset People table')
      return Person.query().truncate()
    },
    findPerson(id) {
      console.log('looking for person with id %d', id)
      return Person.query().findById(id)
    },
  })
}

⛔️ Cannot add more methods to the REST API

✅ Can query the database using the cy.task

// how to check the full record with id: 1?
cy.task('findPerson', id)

⛔️ Cannot add more methods to the REST API

✅ Can query the database using the cy.task

// how to check the full record with id: 1?
cy.task('findPerson', id).should('include', {
  id,
  firstName: 'Joe',
  lastName: 'Pesci',
})

💡 Tip: pass multiple arguments to cy.task

// ⛔️ cannot pass multiple arguments to the task
cy.task('addNumbers', 2, 3)
// ✅ combine arguments into an options object
cy.task('addNumbers', {a: 2, b: 3})
on('task', {
  addNumbers (opts) {
    const {a, b} = opts
    return a + b
  }
})

💡 Tip: use async / await inside cy.task

// ✅ to use modern syntax
on('task', {
  async findPerson (...) {
    const p = await Person.query()...
    const t = await p.children()...
    return t.name
  }
})

Make sure to return a Promise or null to let the Cypress know the code is really finished

// ✅ to use modern syntax
on('task', {
  async findPerson(id) {
    console.log('looking for person with id %d', id)
    const p = await Person.query().findById(id)
    return p || null
  }
})

avoids return undefined if not found

💡 Tip: bring assertions into the cy.task

on('task', {
  findPerson(id) {
    console.log('looking for person with id %d', id)
    return Person.query().findById(id)
  }
})
on('task', {
  findPerson(id) {
    if (typeof id !== 'number') {
      throw new Error('Invalid person id')
    }
    console.log('looking for person with id %d', id)
    return Person.query().findById(id)
  }
})

let's validate the ID before using it

💡 Tip: bring assertions into the cy.task

const { expect } = require('chai')

on('task', {
  findPerson(id) {
    expect(id, 'valid id').to.be.a('number').above(0)
    console.log('looking for person with id %d', id)
    return Person.query().findById(id)
  }
})
$ npm i -D chai
+ chai@4.2.0

Because Cypress comes with bundled Chai assertions, let's use them inside plugins too

💡 Tip: bring assertions into the cy.task

https://glebbahmutov.com/blog/powerful-cy-task/

💡 Tip: cy.task does NOT retry

You need to write your own retry logic inside cy.task

💡 Start App & Run Tests

https://github.com/bahmutov/start-server-and-test

$ npm i -D start-server-and-test
{
  "scripts": {
    "migrate": "knex migrate:latest",
    "start": "npm run migrate && node app",
    "cy:open": "cypress open",
    "cy:run": "cypress run",
    "dev": "start-test 8641 cy:open",
    "local": "start-test 8641 cy:run"
  }
}

App commands

Cypress commands

local TDD

local CI run

Back to The Database!

Set up data before spec

// cypress/integration/matt-damon-spec.js
describe('Matt Damon', () => {
  it('has family', () => {
    // the database already has the initial family for Matt Damon
    cy.request('/persons').its('body').should('have.length', 3)
  })
})

How to set up a complex database state?

  • set up task in plugin file
  • call the task from the spec

Q & A at Slido.com event code #beyond-hello

https://on.cypress.io/before-spec-api

  • before:run
  • before:spec
  • after:spec
  • after:run

Cypress v6.2.0+

Set up data before spec

https://on.cypress.io/before-spec-api

// cypress/plugins/index.js
const { insertPersons } = require('../../utils')
module.exports = (on, config) => {
  on('before:spec', async (spec) => {
    console.log('before:spec %s', spec.name)
    
    if (spec.name === 'matt-damon-spec.js') {
      const mattDamonFamily = require('../fixtures/matt-damon.json')
      await Person.query().truncate()
      await insertPersons(mattDamonFamily)
    }
  })
 }

Instead of setting DB to empty and building up state

🧐 Problem: "before:spec" only works during "cypress run"

// cypress/support/index.js
before(() => {
  if (Cypress.config('isInteractive')) {
    cy.task('beforeSpec', Cypress.spec)
  }
})

true for "cypress open"

{ name, path, ... }

// cypress/plugins/index.js
const onBeforeSpec = async (spec) => {
  console.log('before:spec %s', spec.name)
  if (spec.name === 'matt-damon-spec.js') {
    ...
  }
  return null
}
  
module.exports = (on, config) => {
  on('before:spec', onBeforeSpec) // for "cypress run"

  on('task', {
    beforeSpec: onBeforeSpec // for "cypress open"
  }
}

Code executes in both "cypress run" and "cypress open" modes

Reuse Code

// api.js production code
const { insertPersons } = require('./utils')
router.post('/persons', async (ctx) => {
  const insertedGraph = await insertPersons(ctx.request.body)
  ctx.body = insertedGraph
})

api code

// cypress/plugins/index.js
const { insertPersons } = require('../../utils')
on('task', {
  insertPeople (people) {
    return insertPersons(people)
  }
})

test plugins code

the same database code

Q & A at Slido.com event code #beyond-hello

Fun: re-encode video in after:spec event

https://github.com/bahmutov/cypress-realworld-app/pull/14

// cypress/plugins/index.js
import { toVintageVideo } from "../../code-video"
module.exports = (on, config) => {
  on("after:spec", (spec, results) => {
    if (!results.video) {
      // nothing to process
      return;
    }

    return toVintageVideo(results.video);
  });
}
  • cy.readFile, cy.task
  • Plugins file vs spec files
  • Browser APIs
    • window.open, second tab
    • disabling service worker
  • tips & tricks
  • Cypress documentation
  • how to report a Cypress bug
  • current work: recorder
  • Q&A

Agenda

Cypress Plugins (Node)

Cypress Spec (Browser)

Q & A at Slido.com event code #beyond-hello

context('navigator.battery', () => {
  it('shows battery status of 50%', function () {
    cy.visit('/', {
      onBeforeLoad (win) {
        // mock "navigator.battery" property
        // returning mock charge object
        win.navigator.battery = {
          level: 0.5,
          charging: false,
          chargingTime: Infinity,
          dischargingTime: 3600, // seconds
          addEventListener: () => {}
        }
      }
    })

    // now we can assert actual text - we are charged at 50%
    cy.get('.battery-percentage')
      .should('be.visible')
      .and('have.text', '50%')

    // not charging means running on battery
    cy.contains('.battery-status', 'Battery').should('be.visible')
    // and has enough juice for 1 hour
    cy.contains('.battery-remaining', '1:00').should('be.visible')
  })
})

https://glebbahmutov.com/blog/stub-navigator-api/

✅ Can stub Browser APIs

Example: target=_blank

<a href="/about.html" target="_blank">About</a>

https://glebbahmutov.com/blog/cypress-tips-and-tricks/#deal-with-target_blank

Example: target=_blank

it('loads the about page', () => {
  cy.visit('index.html')
  cy.get('a').should($a => {
    expect($a.attr('href'), 'href').to.equal('/about.html')
    expect($a.attr('target'), 'target').to.equal('_blank')
  })
})

https://glebbahmutov.com/blog/cypress-tips-and-tricks/#deal-with-target_blank

Example: target=_blank

it('loads the about page', () => {
  cy.visit('index.html')
  cy.get('a').should($a => {
    expect($a.attr('href'), 'href').to.equal('/about.html')
    expect($a.attr('target'), 'target').to.equal('_blank')
    $a.attr('target', '_self')
  }).click()
  cy.location('pathname').should('equal', '/about.html')
})

Example: window.open

<a href="/about.html" target="_blank">About</a>
<script>
  document.querySelector('a').addEventListener('click', (e) => {
    e.preventDefault()
    window.open('/about.html')
  })
</script>

https://glebbahmutov.com/blog/cypress-tips-and-tricks/#deal-with-windowopen

Example: window.open

it('opens the about page', () => {
  cy.visit('index.html')
  cy.window().then(win => {
    cy.stub(win, 'open').as('open')
  })
  cy.get('a').click()
  cy.get('@open').should('have.been.calledOnceWithExactly', '/about.html')
})

https://glebbahmutov.com/blog/cypress-tips-and-tricks/#deal-with-windowopen

Example: window.open

it('opens the about page', () => {
  cy.visit('index.html')
  cy.window().then(win => {
    cy.stub(win, 'open').callsFake((url, target) => {
      expect(target).to.be.undefined
      return win.open.wrappedMethod.call(win, url, '_self')
    }).as('open')
  })
  cy.get('a').click()
  cy.get('@open').should('have.been.calledOnceWithExactly', '/about.html')
})

https://glebbahmutov.com/blog/cypress-tips-and-tricks/#deal-with-windowopen

Want to see how your app behaves without ServiceWorker support?

https://glebbahmutov.com/blog/cypress-tips-and-tricks/#disable-serviceworker

cy.visit('index.html', {
  onBeforeLoad (win) {
    delete win.navigator.__proto__.serviceWorker
  }
})

https://on.cypress.io/catalog-of-events

💡 Tip: run the same code for every window load

it('works without SW', () => {
  cy.on('window:before:load', (win) => {
    delete win.navigator.__proto__.serviceWorker
  })  
  
  cy.visit('index.html')
})
Cypress.on('window:before:load', (win) => {
  delete win.navigator.__proto__.serviceWorker
})

it('works without SW', () => {    
  cy.visit('index.html')
})

https://on.cypress.io/writing-and-organizing-tests

💡 Tip: use the support file

// cypress/support/index.js
Cypress.on('window:before:load', (win) => {
  delete win.navigator.__proto__.serviceWorker
})

// cypress/integration/spec.js
it('works without SW', () => {    
  cy.visit('index.html')
})
<script src="cypress/support/index.js"></script>
<script src="cypress/integration/spec.js"></script>

💡 Be careful with negative assertions

// positive
cy.get('.todo-item')
  .should('have.length', 2)
  .and('have.class', 'completed')
// negative
cy.contains('first todo').should('not.have.class', 'completed')
cy.get('#loading').should('not.be.visible')

https://glebbahmutov.com/blog/negative-assertions/

Loading element goes away (wrong)

it('hides the loading element', () => {
  cy.visit('/')
  cy.get('.loading').should('not.be.visible')
})

https://glebbahmutov.com/blog/negative-assertions/

Red 🚩: negative assertion passes

before the XHR

Loading element goes away (wrong)

it('uses negative assertion and passes for the wrong reason', () => {
  cy.visit('/?delay=3000')
  cy.get('.loading').should('not.be.visible')
})

Negative assertions passes even before the Ajax request starts

https://glebbahmutov.com/blog/negative-assertions/

💡 Positive then negative assertion

it('use positive then negative assertion (flakey)', () => {
  cy.visit('/?delay=3000')
  // first, make sure the loading indicator shows up
  cy.get('.loading').should('be.visible')
  // then assert it goes away (negative assertion)
  cy.get('.loading').should('not.be.visible')
})

https://glebbahmutov.com/blog/negative-assertions/

💡 Slow down Ajax request

it('slows down the network response (works)', () => {
  cy.intercept('/todos', {
    body: [],
    delayMs: 5000
  })
  cy.visit('/?delay=3000')
  // first, make sure the loading indicator shows up
  cy.get('.loading').should('be.visible')
  // then assert it goes away (negative assertion)
  cy.get('.loading').should('not.be.visible')
})

https://glebbahmutov.com/blog/negative-assertions/

https://twitter.com/goofballLogic/status/1348003348822302725

Not recommended: installing Cypress globally

Problem: every Cypress version takes 500MB

Solution: trim unused Cypress versions

$ npx cypress cache list --size
┌─────────┬─────────────┬─────────┐
│ version │ last used   │ size    │
├─────────┼─────────────┼─────────┤
│ 6.1.0   │ 11 days ago │ 516.2MB │
├─────────┼─────────────┼─────────┤
│ 6.2.0   │ 13 days ago │ 516.1MB │
├─────────┼─────────────┼─────────┤
│ 6.2.1   │ 4 days ago  │ 515.4MB │
└─────────┴─────────────┴─────────┘

$ npx cypress cache prune

💡 Tip: prune all Cypress versions

💡 Tip: IntelliSense for cypress.json file

https://on.cypress.io/intellisense

"$schema": "https://on.cypress.io/cypress.schema.json"
/// <reference types="cypress" />

📚 Where to find docs

Every command has its documentation at https://on.cypress.io/<command>

https://on.cypress.io/type
https://on.cypress.io/click

Almost every concept has a short link https://on.cypress.io/<concept>

https://on.cypress.io/ci
https://on.cypress.io/docker

💡 Use the doc search

https://docs.cypress.io/

💡 Search from CLI

https://github.com/bahmutov/cy-search

@bahmutov

💡 Advanced topics

https://glebbahmutov.com/blog/tags/cypress/

@bahmutov

https://glebbahmutov.com/blog/cypress-tips-and-tricks/

😡 Why don't you solve my problem!

(how to open a good GitHub issue)

https://github.com/cypress-io/cypress/pulse/monthly

😡 Why don't you solve my problem!

(how to open a good GitHub issue)

https://github.com/cypress-io/cypress/pulse/daily

⛲️ Every day I'm hustlin'

⁉️ If you have a problem:

  • Search our documentation
  • Search open GitHub issues
  • Look at the topics
    • is there a topic matching your problem? 
    • are there open or closed issues matching your issue?

🎁 If opening a new GH issue

You do not have to have fixtures / support / plugins

💡 Tip: remove what is not needed

📹 Cypress Studio Recorder

(under development https://github.com/cypress-io/cypress/pull/9542)

📹 Cypress Studio Recorder

(under development https://github.com/cypress-io/cypress/pull/9542)

super low GitHub issue number = it was hard to do, but the users really want it

Thank you 👏

@bahmutov

https://slides.com/bahmutov/cypress-beyond-the-hello-world

https://docs.cypress.io/

https://glebbahmutov.com/

Cypress: Beyond the "Hello World" Test

By Gleb Bahmutov

Cypress: Beyond the "Hello World" Test

In this presentation, Gleb Bahmutov, a Distinguished Engineer at Cypress will show how to write realistic Cypress tests beyond a simple "Hello World" test. We will look at Node vs Browser boundary, cy.task, setting up the database state, stubbing browser APIs, documentation, GitHub issues, Cypress Studio, and other advanced topics. Everyone who is just starting with Cypress or is an advanced user will benefit from attending this free meetup. Video at https://www.youtube.com/watch?v=Q5djbLL1Pjg

  • 3,125

More from Gleb Bahmutov