Gleb Bahmutov, PhD

VP of Engineering

Automated Testing with

Software Crafters Montréal

our planet is in imminent danger

https://lizkeogh.com/2019/07/02/off-the-charts/

+3 degrees Celsius will be the end.

we have to act today

ME

you

you

you

you

vote & lobby & rebel

if I can help I will

@bahmutov

gleb.bahmutov@gmail.com

Dr Gleb Bahmutov PhD

C / C++ / C# / Java / CoffeeScript / JavaScript / Node / Angular / Vue / Cycle.js / functional

(these slides)

30 people. Atlanta, Philly, Boston, NYC, the world

Fast, easy and reliable testing for anything that runs in a browser

Quality software behaves the way users expect it to behave

We going to need some tests

E2E

integration

unit

Testing Pyramid △

E2E

integration

unit

Smallest pieces

Testing Pyramid △

Unit tests pass...

E2E

integration

unit

Component

E2E

integration

unit

Website / API

E2E

integration

unit

Really important to users

Really important to developers

E2E

integration

unit

Web application

  • Open real browser
  • Load actual app
  • Interact with app like a real user
  • See if it works
$ npm install -D cypress

Cypress demo

  • Typical test
  • Failing test
  • Recording tests
  • CI setup
  • Cypress dashboard
// ui-spec.js
it('loads the app', () => {
  cy.visit('http://localhost:3000')
  cy.get('.todoapp').should('be.visible')
})

Mocha BDD syntax

Chai assertions

it('adds 2 todos', () => {
  cy.visit('http://localhost:3000')
  cy.get('.new-todo')
    .type('learn testing{enter}')
    .type('be cool{enter}')
  cy.get('.todo-list li')
    .should('have.length', 2)
})
$ npx cypress open

DOM

storage

location

cookies

Cypress tests run in the same browser

DOM

storage

location

cookies

Cypress acts as a proxy for your app

Cypress Philosophy

  • Test your app

  • Stub 3rd party services

  • Refactor, while tests are passing

  • Strangle old code 😃

Tutorials

Api

Examples

  • "cypress open" - GUI interactive mode

  • "cypress run" - headless mode

Cypress CLI has 2 main commands

full video of the run, screenshots of every  failure

  • Every CI should be good

  • Or use a Docker image we provide

Running E2E on CI

Making it easy for users is not easy

Revolution in CI

Paid Features 💵

  • recording test artifacts
  • test parallelization
  • GitHub integration
  • (WIP) historical insights

Paid Features 💵

Paid Features 💵: artifacts

test output, video, screenshots

I have 100s of tests ...

$ npx cypress run --record --parallel

Cypress v3.1.0

Spin N CI machines and

Text

cypress-dashboard parallelization

cypress-dashboard parallelization

# machines run duration time savings
1 22:50 ~
2 11:47 48%
3 7:51 65%
4 5:56 74%
6 3:50 83%
8 3:00 87%
10 2:19 90%

cypress-dashboard parallelization

10 machines = 10x speed up

Historical insights

Bridging the Gap

Unit

E2E

Component

For React users

import { HelloState } from '../src/hello-x.jsx'
import React from 'react'
it('changes state', () => {
  cy.mount(<HelloState />)
  cy.contains('Hello Spider-man!')
  const stateToSet = { name: 'React' }
  cy.get(HelloState).invoke('setState', stateToSet)
  cy.get(HelloState)
    .its('state')
    .should('deep.equal', stateToSet)
  cy.contains('Hello React!')
})

Coming in PR #5923

Anything you want to test in React app: https://github.com/bahmutov/cypress-react-unit-test/pull/108

Let's Talk about Code Coverage

  • first test
  • two more tests
  • hundreds of tests!!!

What features do your tests cover?

it('adds todos', () => {
  cy.visit('/')
  cy.get('.new-todo')
    .type('write code{enter}')
    .type('write tests{enter}')
    .type('deploy{enter}')
  cy.get('.todo').should('have.length', 3)
})

E2E tests are extremely effective at covering a lot of app code

Coverage as a guide to writing E2E tests

it('adds todos', () => {
  cy.visit('/')
  cy.get('.new-todo')
    .type('write code{enter}')
    .type('write tests{enter}')
    .type('deploy{enter}')
  cy.get('.todo').should('have.length', 3)
})

Coverage as a guide to writing E2E tests

it('adds todos', () => {
  cy.visit('/')
  cy.get('.new-todo')
    .type('write code{enter}')
    .type('write tests{enter}')
    .type('deploy{enter}')
  cy.get('.todo').should('have.length', 3)
})

We have tested "add todo"

Need tests

Ughh, missed it

Can we always write an E2E test?

What is this code doing?

Can I write an E2E test to hit this code?

Sometimes you cannot

This code should be unreachable from the user interface

Write a Unit Test!

import {getVisibleTodos} from '../../src/selectors'

describe('getVisibleTodos', () => {
  it('throws an error for unknown visibility filter', () => {
    expect(() => {
      getVisibleTodos({
        todos: [],
        visibilityFilter: 'unknown-filter'
      })
    }).to.throw()
  })
})

@cypress/code-coverage plugin

We got this line from the unit test

@cypress/code-coverage plugin

Nice job, @cypress/code-coverage plugin

combines e2e and unit test coverage automatically

Visual testing

it('draws pizza correctly', function () {
  cy.percySnapshot('Empty Pizza')

  cy.enterDeliveryInformation()
  const toppings = ['Pepperoni', 'Chili', 'Onion']
  cy.pickToppings(...toppings)

  // make sure the web app has updated
  cy.contains('.pizza-summary__total-price', 'Total: $12.06')
  cy.percySnapshot(toppings.join(' - '))
})

Visual review workflow

<style type="text/css">
  .st0{fill:#FFD8A1;}
  .st1{fill:#E8C08A;}
- .st2{fill:#FFDC71;}
+ .st2{fill:#71FF71;}
  .st3{fill:#DFBA86;}
</style>

changes crust color SVG

1. Make pull request with visual changes

Visual review workflow

catch visual changes in the pull request

2. Review & approve visual changes

Visual review workflow

2. Review & approve visual changes

App Actions

// app code
var model = new app.TodoModel('react-todos')
if (window.Cypress) {
  window.model = model
}
// test code
export const addTodos = (...todos) => {
  cy.window()
    .its('model')
    .should('be.an', 'object')
    .invoke('addTodo', ...todos)
}

Why developers love Cypress

  • Fast, flake-free

  • GUI, time travel

  • Test recording

  • Documentation

Coming soon

  • Retries / flake factor

  • Full network stubbing

  • so many more ideas ...

Thank you 👏

@cypress_io

@bahmutov

Automated Testing with Cypress.io

By Gleb Bahmutov

Automated Testing with Cypress.io

This talk shows how quick and simple it can be to write end-to-end tests for web applications – if your testing tools are not fighting you all the time. I will go over writing E2E tests using Cypress.io (https://www.cypress.io/), controlling the network during tests, using visual testing and setting up continuous integration to perform E2E tests on each commit.

  • 293
Loading comments...

More from Gleb Bahmutov