Gleb Bahmutov PRO
JavaScript ninja, image processing expert, software quality fanatic
Gleb Bahmutov, PhD
VP of Engineering
@bahmutov
End-to-end Testing with
Technical Testing Guild
Contents
- Testing approach
- Cypress demo
- Cypress documentation
- Cypress Dashboard features
- test artifacts
- parallelization
- GitHub integration
- test insights
- test flake detection
- Component testing
- Visual testing
- Q & A
Dr Gleb Bahmutov PhD
C / C++ / C# / Java / CoffeeScript / JavaScript / Node / Angular / Vue / Cycle.js / functional
(these slides)
40 people. Atlanta, Philly, Boston, NYC, the world
Fast, easy and reliable testing for anything that runs in a browser
Quality software behaves the way users expect it to behave
We going to need some tests
E2E
integration
unit
Testing Pyramid △
E2E
integration
unit
Smallest pieces
Testing Pyramid △
Unit tests pass...
E2E
integration
unit
Component
E2E
integration
unit
Website / API
E2E
integration
unit
Really important to users
Really important to developers
E2E
integration
unit
Web application
- Open real browser
- Load actual app
- Interact with app like a real user
- See if it works
$ npm install -D cypress
- Win / Mac / Linux
- MIT License
- OSS https://github.com/cypress-io/cypress
- http://on.cypress.io/introduction-to-cypress
Cypress demo
- Typical test
- Failing test
- Recording tests
- CI setup
- Cypress dashboard
// ui-spec.js
it('loads the app', () => {
cy.visit('http://localhost:3000')
cy.get('.todoapp').should('be.visible')
})
Mocha BDD syntax
Chai assertions
it('adds 2 todos', () => {
cy.visit('http://localhost:3000')
cy.get('.new-todo')
.type('learn testing{enter}')
.type('be cool{enter}')
cy.get('.todo-list li')
.should('have.length', 2)
})
$ npx cypress openDOM
storage
location
cookies
Cypress tests run in the same browser
DOM
storage
location
cookies
Cypress acts as a proxy for your app
Cypress Philosophy
-
Test your app
-
Stub 3rd party services
-
https://on.cypress.io/network-requests
Full App Example
Tutorials
Api
Examples
-
"cypress open" - GUI interactive mode
-
"cypress run" - headless mode
Cypress CLI has 2 main commands
full video of the run, screenshots of every failure
-
Every CI should be good
-
Or use a Docker image we provide
Running E2E on CI
Making it easy for users is not easy
Paid Features 💵
- recording test artifacts
- test parallelization
- GitHub integration
- test analytics
Paid Features 💵
Paid Features 💵: artifacts
test output, video, screenshots
I have 100s of tests ...
$ npx cypress run --record --parallel
Cypress v3.1.0
Spin N CI machines and
Text
cypress-dashboard parallelization
cypress-dashboard parallelization
| # machines | run duration | time savings |
|---|---|---|
| 1 | 22:50 | ~ |
| 2 | 11:47 | 48% |
| 3 | 7:51 | 65% |
| 4 | 5:56 | 74% |
| 6 | 3:50 | 83% |
| 8 | 3:00 | 87% |
| 10 | 2:19 | 90% |
cypress-dashboard parallelization
10 machines = 10x speed up
Test Analytics and History
{
"retries": {
// Configure retry attempts for `cypress run`
// Default is 0
"runMode": 2,
// Configure retry attempts for `cypress open`
// Default is 0
"openMode": 0
}
}
For React/Vue/X users
import { HelloState } from '../src/hello-x.jsx'
import {mount} from 'cypress-react-unit-test'
import React from 'react'
it('changes state', () => {
mount(<HelloState />)
cy.contains('Hello Spider-man!')
const stateToSet = { name: 'React' }
cy.get(HelloState).invoke('setState', stateToSet)
cy.get(HelloState)
.its('state')
.should('deep.equal', stateToSet)
cy.contains('Hello React!')
})Code Coverage
For React users
$ npm i -D @cypress/instrument-cra
+ @cypress/instrument-cra@1.0.0
"start": "react-scripts -r @cypress/instrument-cra start"
Visual testing
it('draws pizza correctly', function () {
cy.percySnapshot('Empty Pizza')
cy.enterDeliveryInformation()
const toppings = ['Pepperoni', 'Chili', 'Onion']
cy.pickToppings(...toppings)
// make sure the web app has updated
cy.contains('.pizza-summary__total-price', 'Total: $12.06')
cy.percySnapshot(toppings.join(' - '))
})Visual review workflow
<style type="text/css">
.st0{fill:#FFD8A1;}
.st1{fill:#E8C08A;}
- .st2{fill:#FFDC71;}
+ .st2{fill:#71FF71;}
.st3{fill:#DFBA86;}
</style>changes crust color SVG
1. Make pull request with visual changes
Visual review workflow
catch visual changes in the pull request
2. Review & approve visual changes
Visual review workflow
2. Review & approve visual changes
App Actions
// app code
var model = new app.TodoModel('react-todos')
if (window.Cypress) {
window.model = model
}
// test code
export const addTodos = (...todos) => {
cy.window()
.its('model')
.should('be.an', 'object')
.invoke('addTodo', ...todos)
}Why developers love Cypress
-
Fast, flake-free
-
GUI, time travel
-
Test recording
-
Documentation
E2E
+ visual testing
component
unit
api
My view of testing efficiency
Coming soon
-
Component testing
-
Full network stubbing finished
-
so many more ideas ...
Thank you 👏
@cypress_io
@bahmutov
End-to-end Testing With Cypress
By Gleb Bahmutov
End-to-end Testing With Cypress
Unit testing is hard and time-consuming; and worse - the users and the customers do not care! They only want the features working in the production system, everything else is development overhead. If this is the case, how do we improve web application quality? How do we catch the bugs early? How can we test the deployed system effectively? This presentation tries to answer many of these questions in a web framework agnostic way using open source test runner Cypress.io.
