Gleb Bahmutov, PhD

Distinguished Engineer

@bahmutov

Introduction to

Contents

  • Testing approach
  • Cypress demo
  • Cypress documentation
  • Cypress Dashboard features
    • test artifacts
    • parallelization
    • GitHub integration
    • test insights
    • test flake detection
  • Component testing
  • Visual testing
  • Q & A

Speaker: Gleb Bahmutov PhD

C / C++ / C# / Java / CoffeeScript / JavaScript / Node / Angular / Vue / Cycle.js / functional

50 people. Atlanta, Philly, Boston, NYC, the world

Fast, easy and reliable testing for anything that runs in a browser

Quality software behaves the way users expect it to behave

We going to need some tests

E2E

integration

unit

Testing Pyramid △

E2E

integration

unit

Smallest pieces

Testing Pyramid △

Unit tests pass...

E2E

integration

unit

Component

E2E

integration

unit

Website / API

E2E

integration

unit

Really important to users

Really important to developers

E2E

integration

unit

Web application

  • Open real browser
  • Load actual app
  • Interact with app like a real user
  • See if it works
$ npm install -D cypress

Cypress demo

  • Typical test
  • Failing test
  • Recording tests
  • CI setup
  • Cypress dashboard
// ui-spec.js
it('loads the app', () => {
  cy.visit('http://localhost:3000')
  cy.get('.todoapp').should('be.visible')
})

Mocha BDD syntax

Chai assertions

it('adds 2 todos', () => {
  cy.visit('http://localhost:3000')
  cy.get('.new-todo')
    .type('learn testing{enter}')
    .type('be cool{enter}')
  cy.get('.todo-list li')
    .should('have.length', 2)
})
$ npx cypress open

DOM

storage

location

cookies

Cypress tests run in the same browser

DOM

storage

location

cookies

Cypress acts as a proxy for your app

Cypress tests run in the browser and Cypress can interact with the OS via its plugin file that runs in the Node process

Cypress Philosophy

describe('intercept', () => {
  it('returns different fruits every 30 seconds', () => {
    cy.clock()

    // return difference responses on each call
    // notice the order of the intercepts
    cy.intercept('/favorite-fruits', ['kiwi 🥝']) // 3rd, 4th, etc
    cy.intercept('/favorite-fruits', { times: 1 }, ['grapes 🍇']) // 2nd
    cy.intercept('/favorite-fruits', { times: 1 }, ['apples 🍎']) // 1st

    cy.visit('/fruits.html')
    cy.contains('apples 🍎')
    cy.tick(30000)
    cy.contains('grapes 🍇')
    // after using the first two intercepts
    // forever reply with "kiwi" stub
    cy.tick(30000)
    cy.contains('kiwi 🥝')
    cy.tick(30000)
    cy.contains('kiwi 🥝')
    cy.tick(30000)
    cy.contains('kiwi 🥝')
  })
})

Cypress Power

controls app clock and mocks network

Cypress Power

describe('intercept', () => {
  it('returns different fruits every 30 seconds', () => {
    cy.clock()

    // return difference responses on each call
    // notice the order of the intercepts
    cy.intercept('/favorite-fruits', ['kiwi 🥝']) // 3rd, 4th, etc
    cy.intercept('/favorite-fruits', { times: 1 }, ['grapes 🍇']) // 2nd
    cy.intercept('/favorite-fruits', { times: 1 }, ['apples 🍎']) // 1st

    cy.visit('/fruits.html')
    cy.contains('apples 🍎')
    cy.tick(30000)
    cy.contains('grapes 🍇')
    // after using the first two intercepts
    // forever reply with "kiwi" stub
    cy.tick(30000)
    cy.contains('kiwi 🥝')
    cy.tick(30000)
    cy.contains('kiwi 🥝')
    cy.tick(30000)
    cy.contains('kiwi 🥝')
  })
})

controls app clock and mocks network

Tutorials

Api

Examples

Cypress Tips & Tricks

https://www.youtube.com/glebbahmutov

Good Cypress Examples

Full App Example

Workshop

Workshop

  • "cypress open" - GUI interactive mode

  • "cypress run" - headless mode

Cypress CLI has 2 main commands

full video of the run, screenshots of every  failure

  • Every CI should be good

  • Or use a Docker image we provide

Running E2E on CI

Making it easy for users is not easy

Cypress + CI = ❤️

Netlify

[[plugins]]
  # local Cypress plugin will test our site after it is built
  package = "netlify-plugin-cypress"

CircleCI

version: 2.1
orbs:
  cypress: cypress-io/cypress@1
workflows:
  build:
    jobs:
      - cypress/run

Cypress + CI = ❤️

GitHub Actions

name: End-to-end tests
on: [push]
jobs:
  cypress-run:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - uses: cypress-io/github-action@v2

Cypress + CI = ❤️

Workshop

Workshop

Paid Features 💵

  • recording test artifacts
  • test parallelization
  • GitHub integration
  • test analytics

Paid Features 💵

Paid Features 💵: artifacts

test output, video, screenshots

I have 100s of tests ...

$ npx cypress run --record --parallel

Cypress v3.1.0

Spin N CI machines and

Text

cypress-dashboard parallelization

cypress-dashboard parallelization

# machines run duration time savings
1 22:50 ~
2 11:47 48%
3 7:51 65%
4 5:56 74%
6 3:50 83%
8 3:00 87%
10 2:19 90%

cypress-dashboard parallelization

10 machines = 10x speed up

Test Analytics and History

{
  "retries": {
    // Configure retry attempts for `cypress run`
    // Default is 0
    "runMode": 2,
    // Configure retry attempts for `cypress open`
    // Default is 0
    "openMode": 0
  }
}

For React/Vue/Angular/X users

import { HelloState } from '../src/hello-x.jsx'
import { mount } from '@cypress/react' // or @cypress/vue
import React from 'react'

it('changes state', () => {
  mount(<HelloState />)
  cy.contains('Hello Spider-man!')
  const stateToSet = { name: 'React' }
  cy.get(HelloState).invoke('setState', stateToSet)
  cy.get(HelloState)
    .its('state')
    .should('deep.equal', stateToSet)
  cy.contains('Hello React!')
})

Code Coverage

For React users

$ npm i -D @cypress/instrument-cra
+ @cypress/instrument-cra@1.0.0

"start": "react-scripts -r @cypress/instrument-cra start"

Visual testing

it('draws pizza correctly', function () {
  cy.percySnapshot('Empty Pizza')

  cy.enterDeliveryInformation()
  const toppings = ['Pepperoni', 'Chili', 'Onion']
  cy.pickToppings(...toppings)

  // make sure the web app has updated
  cy.contains('.pizza-summary__total-price', 'Total: $12.06')
  cy.percySnapshot(toppings.join(' - '))
})

Visual review workflow

<style type="text/css">
  .st0{fill:#FFD8A1;}
  .st1{fill:#E8C08A;}
- .st2{fill:#FFDC71;}
+ .st2{fill:#71FF71;}
  .st3{fill:#DFBA86;}
</style>

changes crust color SVG

1. Make pull request with visual changes

Visual review workflow

catch visual changes in the pull request

2. Review & approve visual changes

Visual review workflow

2. Review & approve visual changes

Cypress Visual Testing

App Actions

// app code
var model = new app.TodoModel('react-todos')
if (window.Cypress) {
  window.model = model
}
// test code
export const addTodos = (...todos) => {
  cy.window()
    .its('model')
    .should('be.an', 'object')
    .invoke('addTodo', ...todos)
}

Why developers love Cypress

  • Fast, flake-free

  • GUI, time travel

  • Test recording

  • Documentation

E2E

+ visual testing

component

unit

api

My view of testing efficiency

Coming soon

  • Cypress Studio - record tests from UI
  • Session API - traverse multiple domains
  • Better IFrame support
  • WebKit (Safari) support

Thank you 👏

@cypress_io

@bahmutov

Upcoming workshops

CommitEd: Introduction to Cypress

By Gleb Bahmutov

CommitEd: Introduction to Cypress

Introduction to Cypress. This 1-hour presentation gives an introduction to the Cypress Test Runner, its architecture, and the problems it is solving. We will cover end-to-end testing, accessing the application’s internal state, component testing, API testing, visual testing, and collecting code coverage from tests. We will also look at network stubbing and the trade-offs between the full stubbing and spying on the network traffic. The presentation will be followed by questions and answers. Video at https://commited.tech/events/session-introduction-to-cypress/

  • 3,282