Cypress: Beyond the "Hello World" Test

Gleb Bahmutov

VP of Engineering Distinguished Engineer

Cypress.io 

@bahmutov

https://slides.com/bahmutov/cypress-beyond-the-hello-world

Q&A at Slido.com #beyond-hello

our planet is in imminent danger

https://lizkeogh.com/2019/07/02/off-the-charts/

+3 degrees Celsius will be the end.

@bahmutov

survival is possible* but we need to act now

  • change your life
  • join an organization

@bahmutov

rebellion.global          350.org

How long have you used Cypress?

If I were learning Cypress ...

  1. Read https://on.cypress.io/intro
  2. Watch Cypress videos https://on.cypress.io/tutorials
  3. Watch free courses at https://on.cypress.io/courses
  4. Watch Cypress webinars https://youtube.com/cypress_io
  5. Read the docs for every command used and linked resources
  6. Read the blog posts

Agenda

Q & A at Slido.com event code #beyond-hello

Cypress Tests

Run

In The

Browser!

// cypress/integration/spec.js
// NOT GOING TO WORK
const fs = require('fs')
fs.readFileSync(...)

⛔️ Cannot simply access the file system

Cypress Architecture

Access Node and OS

  • cy.readFile
  • cy.writeFile
  • cy.task
  • cy.exec

retries

does not retry

most powerful

most OS-specific

cy.readFile

it('retries reading the JSON file', () => {
  cy.get('.new-todo')
    .type('todo A{enter}')
    .type('todo B{enter}')
    .type('todo C{enter}')
    .type('todo D{enter}')
  cy.readFile('./todomvc/data.json').should(data => {
    expect(data).to.have.property('todos')
    expect(data.todos).to.have.length(4, '4 saved items')
    expect(data.todos[0], 'first item').to.include({
      title: 'todo A',
      completed: false
    })
  })
})

cy.readFile

it('retries reading the JSON file', () => {
  cy.get('.new-todo')
    .type('todo A{enter}')
    .type('todo B{enter}')
    .type('todo C{enter}')
    .type('todo D{enter}')
  cy.readFile('./todomvc/data.json').should(data => {
    expect(data).to.have.property('todos')
    expect(data.todos).to.have.length(4, '4 saved items')
    expect(data.todos[0], 'first item').to.include({
      title: 'todo A',
      completed: false
    })
  })
})

cy.readFile is retried until .should(cb) passes

cy.readFile

it('retries reading the JSON file', () => {
  cy.get('.new-todo')
    .type('todo A{enter}')
    .type('todo B{enter}')
    .type('todo C{enter}')
    .type('todo D{enter}')
  cy.readFile('./todomvc/data.json').should(data => {
    expect(data).to.have.property('todos')
    expect(data.todos).to.have.length(4, '4 saved items')
    expect(data.todos[0], 'first item').to.include({
      title: 'todo A',
      completed: false
    })
  })
})

fast app

cy.readFile

it('retries reading the JSON file', () => {
  cy.get('.new-todo')
    .type('todo A{enter}')
    .type('todo B{enter}')
    .type('todo C{enter}')
    .type('todo D{enter}')
  cy.readFile('./todomvc/data.json').should(data => {
    expect(data).to.have.property('todos')
    expect(data.todos).to.have.length(4, '4 saved items')
    expect(data.todos[0], 'first item').to.include({
      title: 'todo A',
      completed: false
    })
  })
})

Text

slow app

Let's look at cy.task

How do I reset or query a database from my test?

it('starts with an empty list', () => {
  cy.request('/persons').its('body').should('deep.equal', [])
})

Passes every time

Q & A at Slido.com event code #beyond-hello

it('starts with an empty list', () => {
  cy.request('/persons').its('body').should('deep.equal', [])
})

Passes every time

it('adds an actor without any movies', () => {
  cy.request('POST', '/persons', {
    firstName: 'Joe',
    lastName: 'Smith',
  })
  // now there should be 1 actor
  cy.request('/persons')
    .its('body')
    .should('have.length', 1)
    .its('0')
    .should('include', {
      firstName: 'Joe',
      lastName: 'Smith',
    })
})

fails after the second time

Problems:

  • The order of tests
  • The initial state

Reset the DB table

// cypress/plugins/index.js
module.exports = (on, config) => {
  on('task', {
    resetPeopleTable() {
      // TODO: clear the table
    },
  })
}

Tip: connect to DB using same code and logic as the app

// cypress/plugins/index.js
const Knex = require('knex')
const knexConfig = require('../../knexfile')
const { Model } = require('objection')
const knex = Knex(knexConfig.development)
Model.knex(knex)
const Person = require('../../models/Person')

/**
 * @type {Cypress.PluginConfig}
 */
module.exports = (on, config) => {
  on('task', {
    resetPeopleTable() {
      console.log('reset People table')
      return Person.query().truncate()
    },
  })
}

Application code

// cypress/plugins/index.js
const Knex = require('knex')
const knexConfig = require('../../knexfile')
const { Model } = require('objection')
const knex = Knex(knexConfig.development)
Model.knex(knex)
const Person = require('../../models/Person')

/**
 * @type {Cypress.PluginConfig}
 */
module.exports = (on, config) => {
  on('task', {
    resetPeopleTable() {
      console.log('reset People table')
      return Person.query().truncate()
    },
  })
}
$ node -v
v12.14.1
$ npm install
...
$ ls node_modules/sqlite3/lib/binding
node-v72-darwin-x64

Database module installed native DB binding for Node v12.14.1

$ npx cypress version
Cypress package version: 6.2.1
Cypress binary version: 6.2.1
Electron version: 11.1.1
Bundled Node version: 12.18.3

By default plugins file runs using the Node bundled with Cypress

Solution: set Cypress to use the system Node to run the plugins file

// cypress/plugins/index.js
const Knex = require('knex')
const knexConfig = require('../../knexfile')
const { Model } = require('objection')
const knex = Knex(knexConfig.development)
Model.knex(knex)
const Person = require('../../models/Person')

/**
 * @type {Cypress.PluginConfig}
 */
module.exports = (on, config) => {
  on('task', {
    resetPeopleTable() {
      console.log('reset People table')
      return Person.query().truncate()
    },
  })
}

terminal

Passing Tests

Fetch Person by ID

// api.js
router.get('/', ...)
router.post('/persons', ...)
router.get('/persons', ...)
router.patch('/persons/:id', ...)
router.delete('/persons/:id', ...)
...

REST API does not have a method "GET /persons/:id" 🧐

Q & A at Slido.com event code #beyond-hello

it('updates Person information', () => {
  cy.request('POST', '/persons', {
    firstName: 'Joe',
    lastName: 'Smith',
  })
    .its('body.id')
    .then((id) => {
      expect(id, 'created id').to.be.a('number')
      cy.request({
        method: 'PATCH',
        url: `/persons/${id}`,
        body: {
          lastName: 'Pesci',
        },
      })
        .its('body')
        .should('deep.equal', {
          success: true,
        })
      // how to check the full record with id: 1?
    })
})

⛔️ Cannot add more methods to the REST API

✅ Can query the database using the cy.task

const Person = require('../../models/Person')

module.exports = (on, config) => {
  on('task', {
    resetPeopleTable() {
      console.log('reset People table')
      return Person.query().truncate()
    },
    findPerson(id) {
      console.log('looking for person with id %d', id)
      return Person.query().findById(id)
    },
  })
}

⛔️ Cannot add more methods to the REST API

✅ Can query the database using the cy.task

// how to check the full record with id: 1?
cy.task('findPerson', id)

⛔️ Cannot add more methods to the REST API

✅ Can query the database using the cy.task

// how to check the full record with id: 1?
cy.task('findPerson', id).should('include', {
  id,
  firstName: 'Joe',
  lastName: 'Pesci',
})

💡 Tip: pass multiple arguments to cy.task

// ⛔️ cannot pass multiple arguments to the task
cy.task('addNumbers', 2, 3)
// ✅ combine arguments into an options object
cy.task('addNumbers', {a: 2, b: 3})
on('task', {
  addNumbers (opts) {
    const {a, b} = opts
    return a + b
  }
})

💡 Tip: use async / await inside cy.task

// ✅ to use modern syntax
on('task', {
  async findPerson (...) {
    const p = await Person.query()...
    const t = await p.children()...
    return t.name
  }
})

Make sure to return a Promise or null to let the Cypress know the code is really finished

// ✅ to use modern syntax
on('task', {
  async findPerson(id) {
    console.log('looking for person with id %d', id)
    const p = await Person.query().findById(id)
    return p || null
  }
})

avoids return undefined if not found

💡 Tip: bring assertions into the cy.task

on('task', {
  findPerson(id) {
    console.log('looking for person with id %d', id)
    return Person.query().findById(id)
  }
})
on('task', {
  findPerson(id) {
    if (typeof id !== 'number') {
      throw new Error('Invalid person id')
    }
    console.log('looking for person with id %d', id)
    return Person.query().findById(id)
  }
})

let's validate the ID before using it

💡 Tip: bring assertions into the cy.task

const { expect } = require('chai')

on('task', {
  findPerson(id) {
    expect(id, 'valid id').to.be.a('number').above(0)
    console.log('looking for person with id %d', id)
    return Person.query().findById(id)
  }
})
$ npm i -D chai
+ chai@4.2.0

Because Cypress comes with bundled Chai assertions, let's use them inside plugins too

💡 Tip: bring assertions into the cy.task

https://glebbahmutov.com/blog/powerful-cy-task/

💡 Tip: cy.task does NOT retry

You need to write your own retry logic inside cy.task

💡 Start App & Run Tests

https://github.com/bahmutov/start-server-and-test

$ npm i -D start-server-and-test
{
  "scripts": {
    "migrate": "knex migrate:latest",
    "start": "npm run migrate && node app",
    "cy:open": "cypress open",
    "cy:run": "cypress run",
    "dev": "start-test 8641 cy:open",
    "local": "start-test 8641 cy:run"
  }
}

App commands

Cypress commands

local TDD

local CI run

Back to The Database!

Set up data before spec

// cypress/integration/matt-damon-spec.js
describe('Matt Damon', () => {
  it('has family', () => {
    // the database already has the initial family for Matt Damon
    cy.request('/persons').its('body').should('have.length', 3)
  })
})

How to set up a complex database state?

  • set up task in plugin file
  • call the task from the spec

Q & A at Slido.com event code #beyond-hello

https://on.cypress.io/before-spec-api

  • before:run
  • before:spec
  • after:spec
  • after:run

Cypress v6.2.0+

Set up data before spec

https://on.cypress.io/before-spec-api

// cypress/plugins/index.js
const { insertPersons } = require('../../utils')
module.exports = (on, config) => {
  on('before:spec', async (spec) => {
    console.log('before:spec %s', spec.name)
    
    if (spec.name === 'matt-damon-spec.js') {
      const mattDamonFamily = require('../fixtures/matt-damon.json')
      await Person.query().truncate()
      await insertPersons(mattDamonFamily)
    }
  })
 }

Instead of setting DB to empty and building up state

🧐 Problem: "before:spec" only works during "cypress run"

// cypress/support/index.js
before(() => {
  if (Cypress.config('isInteractive')) {
    cy.task('beforeSpec', Cypress.spec)
  }
})

true for "cypress open"

{ name, path, ... }

// cypress/plugins/index.js
const onBeforeSpec = async (spec) => {
  console.log('before:spec %s', spec.name)
  if (spec.name === 'matt-damon-spec.js') {
    ...
  }
  return null
}
  
module.exports = (on, config) => {
  on('before:spec', onBeforeSpec) // for "cypress run"

  on('task', {
    beforeSpec: onBeforeSpec // for "cypress open"
  }
}

Code executes in both "cypress run" and "cypress open" modes

Reuse Code

// api.js production code
const { insertPersons } = require('./utils')
router.post('/persons', async (ctx) => {
  const insertedGraph = await insertPersons(ctx.request.body)
  ctx.body = insertedGraph
})

api code

// cypress/plugins/index.js
const { insertPersons } = require('../../utils')
on('task', {
  insertPeople (people) {
    return insertPersons(people)
  }
})

test plugins code

the same database code

Q & A at Slido.com event code #beyond-hello

Fun: re-encode video in after:spec event

https://github.com/bahmutov/cypress-realworld-app/pull/14

// cypress/plugins/index.js
import { toVintageVideo } from "../../code-video"
module.exports = (on, config) => {
  on("after:spec", (spec, results) => {
    if (!results.video) {
      // nothing to process
      return;
    }

    return toVintageVideo(results.video);
  });
}
  • cy.readFile, cy.task
  • Plugins file vs spec files
  • Browser APIs
    • window.open, second tab
    • disabling service worker
  • tips & tricks
  • Cypress documentation
  • how to report a Cypress bug
  • current work: recorder
  • Q&A

Agenda

Cypress Plugins (Node)

Cypress Spec (Browser)

Q & A at Slido.com event code #beyond-hello

context('navigator.battery', () => {
  it('shows battery status of 50%', function () {
    cy.visit('/', {
      onBeforeLoad (win) {
        // mock "navigator.battery" property
        // returning mock charge object
        win.navigator.battery = {
          level: 0.5,
          charging: false,
          chargingTime: Infinity,
          dischargingTime: 3600, // seconds
          addEventListener: () => {}
        }
      }
    })

    // now we can assert actual text - we are charged at 50%
    cy.get('.battery-percentage')
      .should('be.visible')
      .and('have.text', '50%')

    // not charging means running on battery
    cy.contains('.battery-status', 'Battery').should('be.visible')
    // and has enough juice for 1 hour
    cy.contains('.battery-remaining', '1:00').should('be.visible')
  })
})

https://glebbahmutov.com/blog/stub-navigator-api/

✅ Can stub Browser APIs

Example: target=_blank

<a href="/about.html" target="_blank">About</a>

https://glebbahmutov.com/blog/cypress-tips-and-tricks/#deal-with-target_blank

Example: target=_blank

it('loads the about page', () => {
  cy.visit('index.html')
  cy.get('a').should($a => {
    expect($a.attr('href'), 'href').to.equal('/about.html')
    expect($a.attr('target'), 'target').to.equal('_blank')
  })
})

https://glebbahmutov.com/blog/cypress-tips-and-tricks/#deal-with-target_blank

Example: target=_blank

it('loads the about page', () => {
  cy.visit('index.html')
  cy.get('a').should($a => {
    expect($a.attr('href'), 'href').to.equal('/about.html')
    expect($a.attr('target'), 'target').to.equal('_blank')
    $a.attr('target', '_self')
  }).click()
  cy.location('pathname').should('equal', '/about.html')
})

Example: window.open

<a href="/about.html" target="_blank">About</a>
<script>
  document.querySelector('a').addEventListener('click', (e) => {
    e.preventDefault()
    window.open('/about.html')
  })
</script>

https://glebbahmutov.com/blog/cypress-tips-and-tricks/#deal-with-windowopen

Example: window.open

it('opens the about page', () => {
  cy.visit('index.html')
  cy.window().then(win => {
    cy.stub(win, 'open').as('open')
  })
  cy.get('a').click()
  cy.get('@open').should('have.been.calledOnceWithExactly', '/about.html')
})

https://glebbahmutov.com/blog/cypress-tips-and-tricks/#deal-with-windowopen

Example: window.open

it('opens the about page', () => {
  cy.visit('index.html')
  cy.window().then(win => {
    cy.stub(win, 'open').callsFake((url, target) => {
      expect(target).to.be.undefined
      return win.open.wrappedMethod.call(win, url, '_self')
    }).as('open')
  })
  cy.get('a').click()
  cy.get('@open').should('have.been.calledOnceWithExactly', '/about.html')
})

https://glebbahmutov.com/blog/cypress-tips-and-tricks/#deal-with-windowopen

Want to see how your app behaves without ServiceWorker support?

https://glebbahmutov.com/blog/cypress-tips-and-tricks/#disable-serviceworker

cy.visit('index.html', {
  onBeforeLoad (win) {
    delete win.navigator.__proto__.serviceWorker
  }
})

https://on.cypress.io/catalog-of-events

💡 Tip: run the same code for every window load

it('works without SW', () => {
  cy.on('window:before:load', (win) => {
    delete win.navigator.__proto__.serviceWorker
  })  
  
  cy.visit('index.html')
})
Cypress.on('window:before:load', (win) => {
  delete win.navigator.__proto__.serviceWorker
})

it('works without SW', () => {    
  cy.visit('index.html')
})

https://on.cypress.io/writing-and-organizing-tests

💡 Tip: use the support file

// cypress/support/index.js
Cypress.on('window:before:load', (win) => {
  delete win.navigator.__proto__.serviceWorker
})

// cypress/integration/spec.js
it('works without SW', () => {    
  cy.visit('index.html')
})
<script src="cypress/support/index.js"></script>
<script src="cypress/integration/spec.js"></script>

💡 Be careful with negative assertions

// positive
cy.get('.todo-item')
  .should('have.length', 2)
  .and('have.class', 'completed')
// negative
cy.contains('first todo').should('not.have.class', 'completed')
cy.get('#loading').should('not.be.visible')

https://glebbahmutov.com/blog/negative-assertions/

Loading element goes away (wrong)

it('hides the loading element', () => {
  cy.visit('/')
  cy.get('.loading').should('not.be.visible')
})

https://glebbahmutov.com/blog/negative-assertions/

Red 🚩: negative assertion passes

before the XHR

Loading element goes away (wrong)

it('uses negative assertion and passes for the wrong reason', () => {
  cy.visit('/?delay=3000')
  cy.get('.loading').should('not.be.visible')
})

Negative assertions passes even before the Ajax request starts

https://glebbahmutov.com/blog/negative-assertions/

💡 Positive then negative assertion

it('use positive then negative assertion (flakey)', () => {
  cy.visit('/?delay=3000')
  // first, make sure the loading indicator shows up
  cy.get('.loading').should('be.visible')
  // then assert it goes away (negative assertion)
  cy.get('.loading').should('not.be.visible')
})

https://glebbahmutov.com/blog/negative-assertions/

💡 Slow down Ajax request

it('slows down the network response (works)', () => {
  cy.intercept('/todos', {
    body: [],
    delayMs: 5000
  })
  cy.visit('/?delay=3000')
  // first, make sure the loading indicator shows up
  cy.get('.loading').should('be.visible')
  // then assert it goes away (negative assertion)
  cy.get('.loading').should('not.be.visible')
})

https://glebbahmutov.com/blog/negative-assertions/

https://twitter.com/goofballLogic/status/1348003348822302725

Not recommended: installing Cypress globally

Problem: every Cypress version takes 500MB

Solution: trim unused Cypress versions

$ npx cypress cache list --size
┌─────────┬─────────────┬─────────┐
│ version │ last used   │ size    │
├─────────┼─────────────┼─────────┤
│ 6.1.0   │ 11 days ago │ 516.2MB │
├─────────┼─────────────┼─────────┤
│ 6.2.0   │ 13 days ago │ 516.1MB │
├─────────┼─────────────┼─────────┤
│ 6.2.1   │ 4 days ago  │ 515.4MB │
└─────────┴─────────────┴─────────┘

$ npx cypress cache prune

💡 Tip: prune all Cypress versions

💡 Tip: IntelliSense for cypress.json file

https://on.cypress.io/intellisense

"$schema": "https://on.cypress.io/cypress.schema.json"
/// <reference types="cypress" />

📚 Where to find docs

Every command has its documentation at https://on.cypress.io/<command>

https://on.cypress.io/type
https://on.cypress.io/click

Almost every concept has a short link https://on.cypress.io/<concept>

https://on.cypress.io/ci
https://on.cypress.io/docker

💡 Use the doc search

https://docs.cypress.io/

💡 Search from CLI

https://github.com/bahmutov/cy-search

@bahmutov

💡 Advanced topics

https://glebbahmutov.com/blog/tags/cypress/

@bahmutov

https://glebbahmutov.com/blog/cypress-tips-and-tricks/

😡 Why don't you solve my problem!

(how to open a good GitHub issue)

https://github.com/cypress-io/cypress/pulse/monthly

😡 Why don't you solve my problem!

(how to open a good GitHub issue)

https://github.com/cypress-io/cypress/pulse/daily

⛲️ Every day I'm hustlin'

⁉️ If you have a problem:

  • Search our documentation
  • Search open GitHub issues
  • Look at the topics
    • is there a topic matching your problem? 
    • are there open or closed issues matching your issue?

🎁 If opening a new GH issue

You do not have to have fixtures / support / plugins

💡 Tip: remove what is not needed

📹 Cypress Studio Recorder

(under development https://github.com/cypress-io/cypress/pull/9542)

📹 Cypress Studio Recorder

(under development https://github.com/cypress-io/cypress/pull/9542)

super low GitHub issue number = it was hard to do, but the users really want it

Thank you 👏

@bahmutov

https://slides.com/bahmutov/cypress-beyond-the-hello-world

https://docs.cypress.io/

https://glebbahmutov.com/