Gleb Bahmutov, PhD
VP of Engineering
@bahmutov
End-to-end Testing with
Technical Testing Guild
Contents
- Testing approach
- Cypress demo
- Cypress documentation
- Cypress Dashboard features
- test artifacts
- parallelization
- GitHub integration
- test insights
- test flake detection
- Component testing
- Visual testing
- Q & A
Dr Gleb Bahmutov PhD
C / C++ / C# / Java / CoffeeScript / JavaScript / Node / Angular / Vue / Cycle.js / functional
(these slides)
40 people. Atlanta, Philly, Boston, NYC, the world
Fast, easy and reliable testing for anything that runs in a browser
Quality software behaves the way users expect it to behave
We going to need some tests
E2E
integration
unit
Testing Pyramid △
E2E
integration
unit
Smallest pieces
Testing Pyramid △
Unit tests pass...
E2E
integration
unit
Component
E2E
integration
unit
Website / API
E2E
integration
unit
Really important to users
Really important to developers
E2E
integration
unit
Web application
- Open real browser
- Load actual app
- Interact with app like a real user
- See if it works
$ npm install -D cypress
- Win / Mac / Linux
- MIT License
- OSS https://github.com/cypress-io/cypress
- http://on.cypress.io/introduction-to-cypress
Cypress demo
- Typical test
- Failing test
- Recording tests
- CI setup
- Cypress dashboard
// ui-spec.js
it('loads the app', () => {
cy.visit('http://localhost:3000')
cy.get('.todoapp').should('be.visible')
})
Mocha BDD syntax
Chai assertions
it('adds 2 todos', () => {
cy.visit('http://localhost:3000')
cy.get('.new-todo')
.type('learn testing{enter}')
.type('be cool{enter}')
cy.get('.todo-list li')
.should('have.length', 2)
})
$ npx cypress openDOM
storage
location
cookies
Cypress tests run in the same browser
DOM
storage
location
cookies
Cypress acts as a proxy for your app
Cypress Philosophy
-
Test your app
-
Stub 3rd party services
-
https://on.cypress.io/network-requests
Full App Example
Tutorials
Api
Examples
-
"cypress open" - GUI interactive mode
-
"cypress run" - headless mode
Cypress CLI has 2 main commands
full video of the run, screenshots of every failure
-
Every CI should be good
-
Or use a Docker image we provide
Running E2E on CI
Making it easy for users is not easy
Paid Features 💵
- recording test artifacts
- test parallelization
- GitHub integration
- test analytics
Paid Features 💵
Paid Features 💵: artifacts
test output, video, screenshots
I have 100s of tests ...
$ npx cypress run --record --parallel
Cypress v3.1.0
Spin N CI machines and
Text
cypress-dashboard parallelization
cypress-dashboard parallelization
| # machines | run duration | time savings |
|---|---|---|
| 1 | 22:50 | ~ |
| 2 | 11:47 | 48% |
| 3 | 7:51 | 65% |
| 4 | 5:56 | 74% |
| 6 | 3:50 | 83% |
| 8 | 3:00 | 87% |
| 10 | 2:19 | 90% |
cypress-dashboard parallelization
10 machines = 10x speed up
Test Analytics and History
{
"retries": {
// Configure retry attempts for `cypress run`
// Default is 0
"runMode": 2,
// Configure retry attempts for `cypress open`
// Default is 0
"openMode": 0
}
}
For React/Vue/X users
import { HelloState } from '../src/hello-x.jsx'
import {mount} from 'cypress-react-unit-test'
import React from 'react'
it('changes state', () => {
mount(<HelloState />)
cy.contains('Hello Spider-man!')
const stateToSet = { name: 'React' }
cy.get(HelloState).invoke('setState', stateToSet)
cy.get(HelloState)
.its('state')
.should('deep.equal', stateToSet)
cy.contains('Hello React!')
})Code Coverage
For React users
$ npm i -D @cypress/instrument-cra
+ @cypress/instrument-cra@1.0.0
"start": "react-scripts -r @cypress/instrument-cra start"
Visual testing
it('draws pizza correctly', function () {
cy.percySnapshot('Empty Pizza')
cy.enterDeliveryInformation()
const toppings = ['Pepperoni', 'Chili', 'Onion']
cy.pickToppings(...toppings)
// make sure the web app has updated
cy.contains('.pizza-summary__total-price', 'Total: $12.06')
cy.percySnapshot(toppings.join(' - '))
})Visual review workflow
<style type="text/css">
.st0{fill:#FFD8A1;}
.st1{fill:#E8C08A;}
- .st2{fill:#FFDC71;}
+ .st2{fill:#71FF71;}
.st3{fill:#DFBA86;}
</style>changes crust color SVG
1. Make pull request with visual changes
Visual review workflow
catch visual changes in the pull request
2. Review & approve visual changes
Visual review workflow
2. Review & approve visual changes
App Actions
// app code
var model = new app.TodoModel('react-todos')
if (window.Cypress) {
window.model = model
}
// test code
export const addTodos = (...todos) => {
cy.window()
.its('model')
.should('be.an', 'object')
.invoke('addTodo', ...todos)
}Why developers love Cypress
-
Fast, flake-free
-
GUI, time travel
-
Test recording
-
Documentation
E2E
+ visual testing
component
unit
api
My view of testing efficiency
Coming soon
-
Component testing
-
Full network stubbing finished
-
so many more ideas ...
Thank you 👏
@cypress_io
@bahmutov