Gleb Bahmutov, PhD
Distinguished Engineer
@bahmutov
Introduction to
Contents
- Testing approach
- Cypress demo
- Cypress documentation
- Cypress Dashboard features
- test artifacts
- parallelization
- GitHub integration
- test insights
- test flake detection
- Component testing
- Visual testing
- Q & A
Speaker: Gleb Bahmutov PhD
C / C++ / C# / Java / CoffeeScript / JavaScript / Node / Angular / Vue / Cycle.js / functional
50 people. Atlanta, Philly, Boston, NYC, the world
Fast, easy and reliable testing for anything that runs in a browser
Quality software behaves the way users expect it to behave
We going to need some tests
E2E
integration
unit
Testing Pyramid △
E2E
integration
unit
Smallest pieces
Testing Pyramid △
Unit tests pass...
E2E
integration
unit
Component
E2E
integration
unit
Website / API
E2E
integration
unit
Really important to users
Really important to developers
E2E
integration
unit
Web application
- Open real browser
- Load actual app
- Interact with app like a real user
- See if it works
$ npm install -D cypress
- Win / Mac / Linux
- MIT License
- OSS https://github.com/cypress-io/cypress
- https://on.cypress.io/introduction-to-cypress
Cypress demo
- Typical test
- Failing test
- Recording tests
- CI setup
- Cypress dashboard
// ui-spec.js
it('loads the app', () => {
cy.visit('http://localhost:3000')
cy.get('.todoapp').should('be.visible')
})
Mocha BDD syntax
Chai assertions
it('adds 2 todos', () => {
cy.visit('http://localhost:3000')
cy.get('.new-todo')
.type('learn testing{enter}')
.type('be cool{enter}')
cy.get('.todo-list li')
.should('have.length', 2)
})
$ npx cypress openDOM
storage
location
cookies
Cypress tests run in the same browser
DOM
storage
location
cookies
Cypress acts as a proxy for your app
Cypress tests run in the browser and Cypress can interact with the OS via its plugin file that runs in the Node process
Cypress Philosophy
-
Test your app
-
Stub 3rd party services
-
https://on.cypress.io/network-requests
describe('intercept', () => {
it('returns different fruits every 30 seconds', () => {
cy.clock()
// return difference responses on each call
// notice the order of the intercepts
cy.intercept('/favorite-fruits', ['kiwi 🥝']) // 3rd, 4th, etc
cy.intercept('/favorite-fruits', { times: 1 }, ['grapes 🍇']) // 2nd
cy.intercept('/favorite-fruits', { times: 1 }, ['apples 🍎']) // 1st
cy.visit('/fruits.html')
cy.contains('apples 🍎')
cy.tick(30000)
cy.contains('grapes 🍇')
// after using the first two intercepts
// forever reply with "kiwi" stub
cy.tick(30000)
cy.contains('kiwi 🥝')
cy.tick(30000)
cy.contains('kiwi 🥝')
cy.tick(30000)
cy.contains('kiwi 🥝')
})
})Cypress Power
controls app clock and mocks network
Cypress Power
describe('intercept', () => {
it('returns different fruits every 30 seconds', () => {
cy.clock()
// return difference responses on each call
// notice the order of the intercepts
cy.intercept('/favorite-fruits', ['kiwi 🥝']) // 3rd, 4th, etc
cy.intercept('/favorite-fruits', { times: 1 }, ['grapes 🍇']) // 2nd
cy.intercept('/favorite-fruits', { times: 1 }, ['apples 🍎']) // 1st
cy.visit('/fruits.html')
cy.contains('apples 🍎')
cy.tick(30000)
cy.contains('grapes 🍇')
// after using the first two intercepts
// forever reply with "kiwi" stub
cy.tick(30000)
cy.contains('kiwi 🥝')
cy.tick(30000)
cy.contains('kiwi 🥝')
cy.tick(30000)
cy.contains('kiwi 🥝')
})
})controls app clock and mocks network
Tutorials
Api
Examples
Cypress Tips & Tricks
Good Cypress Examples
- https://on.cypress.io/examples
- search GitHub for topic:cypress-example
- add GH username topic:cypress-example user:bahmutov
- free and paid https://on.cypress.io/courses
Full App Example
Workshop
Workshop
-
"cypress open" - GUI interactive mode
-
"cypress run" - headless mode
Cypress CLI has 2 main commands
full video of the run, screenshots of every failure
-
Every CI should be good
-
Or use a Docker image we provide
Running E2E on CI
Making it easy for users is not easy
Cypress + CI = ❤️
Netlify
[[plugins]]
# local Cypress plugin will test our site after it is built
package = "netlify-plugin-cypress"CircleCI
version: 2.1
orbs:
cypress: cypress-io/cypress@1
workflows:
build:
jobs:
- cypress/runCypress + CI = ❤️
GitHub Actions
name: End-to-end tests
on: [push]
jobs:
cypress-run:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- uses: cypress-io/github-action@v2Cypress + CI = ❤️
Workshop
Workshop
Paid Features 💵
- recording test artifacts
- test parallelization
- GitHub integration
- test analytics
Paid Features 💵
Paid Features 💵: artifacts
test output, video, screenshots
I have 100s of tests ...
$ npx cypress run --record --parallel
Cypress v3.1.0
Spin N CI machines and
Text
cypress-dashboard parallelization
cypress-dashboard parallelization
| # machines | run duration | time savings |
|---|---|---|
| 1 | 22:50 | ~ |
| 2 | 11:47 | 48% |
| 3 | 7:51 | 65% |
| 4 | 5:56 | 74% |
| 6 | 3:50 | 83% |
| 8 | 3:00 | 87% |
| 10 | 2:19 | 90% |
cypress-dashboard parallelization
10 machines = 10x speed up
Test Analytics and History
{
"retries": {
// Configure retry attempts for `cypress run`
// Default is 0
"runMode": 2,
// Configure retry attempts for `cypress open`
// Default is 0
"openMode": 0
}
}
For React/Vue/Angular/X users
import { HelloState } from '../src/hello-x.jsx'
import { mount } from '@cypress/react' // or @cypress/vue
import React from 'react'
it('changes state', () => {
mount(<HelloState />)
cy.contains('Hello Spider-man!')
const stateToSet = { name: 'React' }
cy.get(HelloState).invoke('setState', stateToSet)
cy.get(HelloState)
.its('state')
.should('deep.equal', stateToSet)
cy.contains('Hello React!')
})Code Coverage
For React users
$ npm i -D @cypress/instrument-cra
+ @cypress/instrument-cra@1.0.0
"start": "react-scripts -r @cypress/instrument-cra start"
Visual testing
it('draws pizza correctly', function () {
cy.percySnapshot('Empty Pizza')
cy.enterDeliveryInformation()
const toppings = ['Pepperoni', 'Chili', 'Onion']
cy.pickToppings(...toppings)
// make sure the web app has updated
cy.contains('.pizza-summary__total-price', 'Total: $12.06')
cy.percySnapshot(toppings.join(' - '))
})Visual review workflow
<style type="text/css">
.st0{fill:#FFD8A1;}
.st1{fill:#E8C08A;}
- .st2{fill:#FFDC71;}
+ .st2{fill:#71FF71;}
.st3{fill:#DFBA86;}
</style>changes crust color SVG
1. Make pull request with visual changes
Visual review workflow
catch visual changes in the pull request
2. Review & approve visual changes
Visual review workflow
2. Review & approve visual changes
Cypress Visual Testing
App Actions
// app code
var model = new app.TodoModel('react-todos')
if (window.Cypress) {
window.model = model
}
// test code
export const addTodos = (...todos) => {
cy.window()
.its('model')
.should('be.an', 'object')
.invoke('addTodo', ...todos)
}Why developers love Cypress
-
Fast, flake-free
-
GUI, time travel
-
Test recording
-
Documentation
E2E
+ visual testing
component
unit
api
My view of testing efficiency
Coming soon
- Cypress Studio - record tests from UI
- Session API - traverse multiple domains
- Better IFrame support
- WebKit (Safari) support
Thank you 👏
@cypress_io
@bahmutov
Upcoming workshops