Gleb Bahmutov, PhD

VP of Engineering, Cypress.io

🛣

End-to-end Testing

with

Why is all software broken?

Will Klein

Quality software behaves the way users expect it to behave

We going to need some tests

E2E

integration

unit

Smallest pieces

Testing Pyramid △

Unit tests pass...

E2E

integration

unit

Component

E2E

integration

unit

Website / API

E2E

integration

unit

Really important to users

Really important to developers

$ npm install -D cypress
// ui-spec.js
it('loads the app', () => {
  cy.visit('http://localhost:3030')
  cy.get('.todoapp').should('be.visible')
})

Mocha BDD syntax

Chai assertions

it('adds 2 todos', () => {
  cy.visit('http://localhost:3000')
  cy.get('.new-todo')
    .type('learn testing{enter}')
    .type('be cool{enter}')
  cy.get('.todo-list li')
    .should('have.length', 2)
})
it('adds 2 todos', () => {
  const user = cy
  user.visit('http://localhost:3000')
  user.get('.new-todo')
    .type('learn testing{enter}')
    .type('be cool{enter}')
  user.get('.todo-list li')
    .should('have.length', 2)
})

Cypress demo

  • Typical test
  • Failing test
  • Recording tests
  • CI setup
  • Cypress dashboard

Example: testing how app handles failed XHR call

// app loading todos on start
loadTodos ({ commit }) {
  commit('SET_LOADING', true)
  axios
    .get('/todos')
    .then(r => r.data)
    .then(todos => {
      console.log('got %d todos', todos.length)
      commit('SET_TODOS', todos)
      commit('SET_LOADING', false)
    })
    .catch(e => {
      console.error('could not load todos')
      console.error(e.message)
      console.error(e.response.data)
    })
},

external behavior we want to  test

Application

it('handles 404 when loading todos', () => {
  cy.server()
  cy.route({
    url: '/todos',
    response: 'test does not allow it',
    status: 404
  })
  cy.visit('/', {
    onBeforeLoad: win => {
      cy.spy(win.console, 'error').as('console-error')
    }
  })
  // observe external effect from the app
  cy.get('@console-error')
    .should('have.been.calledWithExactly', 
      'test does not allow it')
})

control network

Test

it('handles 404 when loading todos', () => {
  cy.server()
  cy.route({
    url: '/todos',
    response: 'test does not allow it',
    status: 404
  })
  cy.visit('/', {
    onBeforeLoad: win => {
      cy.spy(win.console, 'error').as('console-error')
    }
  })
  // observe external effect from the app
  cy.get('@console-error')
    .should('have.been.calledWithExactly', 
      'test does not allow it')
})

spy on app's console

Test

it('handles 404 when loading todos', () => {
  cy.server()
  cy.route({
    url: '/todos',
    response: 'test does not allow it',
    status: 404
  })
  cy.visit('/', {
    onBeforeLoad: win => {
      cy.spy(win.console, 'error').as('console-error')
    }
  })
  // observe external effect from the app
  cy.get('@console-error')
    .should('have.been.calledWithExactly', 
      'test does not allow it')
})

assert it happens

Test

But how does it FEEL?

When I test with Cypress

DOM

Network

storage

If you can write E2E tests in a framework-agnostic way

You can replace framework X with Y

(without breaking things)

But how is this different from tool X?

Hint: browsers are complicated

Node to Browser actions are hard

Node

Browser

Node

Cy backend

cy.task(name, ...args)
on('task', {
  name: (...args) => ...
})

Browser to Node is easy

cy.task
cy.exec
cy.request

(and you only send data, not code)

Browser to Node is easy

cy.task

Browser to Node is easy

it('finds record in the database', () => {
  // random text to avoid confusion
  const id = Cypress._.random(1, 1e6)
  const title = `todo ${id}`
  cy.get('.new-todo').type(`${title}{enter}`)

})

runs in the browser

drive via DOM

Observe Database Effect

it('finds record in the database', () => {
  // random text to avoid confusion
  const id = Cypress._.random(1, 1e6)
  const title = `todo ${id}`
  cy.get('.new-todo').type(`${title}{enter}`)
  cy.task('hasSavedRecord', title).should('equal', true)
})

runs in the browser

Observe Database Effect

const hasRecordAsync = (title, ms) => {
  // use promise-retry or convergence
  ...
}

module.exports = (on, config) => {
  on('task', {
    hasSavedRecord (title, ms = 3000) {
      return hasRecordAsync(title, ms)
    }
  })
}

runs in Node in cypress/plugins/index.js

Observe Database Effect

task completes as soon as the server gets POST from the app and saves record to DB

task checks for wrong title and eventually times out

Tutorials

Api

Examples

Video course

Full workshop

  • "cypress open" - GUI interactive mode

  • "cypress run" - headless mode

Cypress CLI has 2 main commands

full video of the run, screenshots of every  failure

  • Every CI should be good

  • Or use a Docker image we provide

Running E2E on CI

Other Demos

many presentations and videos about Cypress

Paid Features 💵

Paid Features 💵: artifacts

test output, video, screenshots

Paid Features 💵 - load balancing

cypress run --record
cypress run --record --group single
cypress run --record --group parallel --parallel

Most CIs should just work 🙏

Paid Features 💵 - load balancing

cypress-dashboard parallelization

# machines run duration time savings
1 22:50 ~
2 11:47 48%
3 7:51 65%
4 5:56 74%
6 3:50 83%
8 3:00 87%
10 2:19 90%

cypress-dashboard parallelization

10 machines = 10x speed up

Why developers love Cypress

  • Fast, flake-free

  • GUI, time travel

  • Test recording

  • Documentation

Why developers love Cypress

  • Fast, flake-free

  • GUI, time travel

  • Test recording

  • Documentation

100% FREE & OSS

Cool Things You Can do With Cypress

Test components from frameworks

Test components from these frameworks with ease

Cool Things You Can do With Cypress

Control your application's behavior

Cool Things You Can do With Cypress

Change Cypress color theme

Coming soon

  • Retries / flake factor

  • Cross-browser

  • Full network stubbing

  • so many more ideas ...

IE11

Gleb Bahmutov, PhD

VP of Engineering

🛣

Thank you

@cypress_io

@bahmutov

slides.com/bahmutov/cypress-reactive-roadshow

github.com/cypress-io/cypress ⭐️

End-to-end Testing with Cypress

By Gleb Bahmutov

End-to-end Testing with Cypress

Testing is hard. End-to-end testing is really hard. But have you ever wondered if maybe our tools are the ones to blame? What if we could completely rethink testing tools to be fast, useful and effective? Let me show you how to quickly test any web application using Cypress.io - a modern open source test runner designed to make you more productive. Don't believe the hype? Come see a Cypress demo for yourself and prepare to be amazed.

  • 4,176