Gleb Bahmutov, PhD
VP of Engineering
Automated Testing with
Software Crafters Montréal
our planet is in imminent danger
https://lizkeogh.com/2019/07/02/off-the-charts/
+3 degrees Celsius will be the end.
we have to act today
ME
you
you
you
you
vote & lobby & rebel
if I can help I will
@bahmutov
gleb.bahmutov@gmail.com
Dr Gleb Bahmutov PhD
C / C++ / C# / Java / CoffeeScript / JavaScript / Node / Angular / Vue / Cycle.js / functional
(these slides)
30 people. Atlanta, Philly, Boston, NYC, the world
Fast, easy and reliable testing for anything that runs in a browser
Quality software behaves the way users expect it to behave
We going to need some tests
E2E
integration
unit
Testing Pyramid △
E2E
integration
unit
Smallest pieces
Testing Pyramid △
Unit tests pass...
E2E
integration
unit
Component
E2E
integration
unit
Website / API
E2E
integration
unit
Really important to users
Really important to developers
E2E
integration
unit
Web application
- Open real browser
- Load actual app
- Interact with app like a real user
- See if it works
$ npm install -D cypress
- Win / Mac / Linux
- MIT License
- OSS https://github.com/cypress-io/cypress
- http://on.cypress.io/introduction-to-cypress
Cypress demo
- Typical test
- Failing test
- Recording tests
- CI setup
- Cypress dashboard
// ui-spec.js
it('loads the app', () => {
cy.visit('http://localhost:3000')
cy.get('.todoapp').should('be.visible')
})
Mocha BDD syntax
Chai assertions
it('adds 2 todos', () => {
cy.visit('http://localhost:3000')
cy.get('.new-todo')
.type('learn testing{enter}')
.type('be cool{enter}')
cy.get('.todo-list li')
.should('have.length', 2)
})
$ npx cypress openDOM
storage
location
cookies
Cypress tests run in the same browser
DOM
storage
location
cookies
Cypress acts as a proxy for your app
Cypress Philosophy
-
Test your app
-
Stub 3rd party services
-
Refactor, while tests are passing
-
Strangle old code 😃
Tutorials
Api
Examples
-
"cypress open" - GUI interactive mode
-
"cypress run" - headless mode
Cypress CLI has 2 main commands
full video of the run, screenshots of every failure
-
Every CI should be good
-
Or use a Docker image we provide
Running E2E on CI
Making it easy for users is not easy
Revolution in CI
Paid Features 💵
- recording test artifacts
- test parallelization
- GitHub integration
- (WIP) historical insights
Paid Features 💵
Paid Features 💵: artifacts
test output, video, screenshots
I have 100s of tests ...
$ npx cypress run --record --parallel
Cypress v3.1.0
Spin N CI machines and
Text
cypress-dashboard parallelization
cypress-dashboard parallelization
| # machines | run duration | time savings |
|---|---|---|
| 1 | 22:50 | ~ |
| 2 | 11:47 | 48% |
| 3 | 7:51 | 65% |
| 4 | 5:56 | 74% |
| 6 | 3:50 | 83% |
| 8 | 3:00 | 87% |
| 10 | 2:19 | 90% |
cypress-dashboard parallelization
10 machines = 10x speed up
Historical insights
Bridging the Gap
Unit
E2E
Component
For React users
import { HelloState } from '../src/hello-x.jsx'
import React from 'react'
it('changes state', () => {
cy.mount(<HelloState />)
cy.contains('Hello Spider-man!')
const stateToSet = { name: 'React' }
cy.get(HelloState).invoke('setState', stateToSet)
cy.get(HelloState)
.its('state')
.should('deep.equal', stateToSet)
cy.contains('Hello React!')
})
Coming in PR #5923
Anything you want to test in React app: https://github.com/bahmutov/cypress-react-unit-test/pull/108
Let's Talk about Code Coverage
- first test
- two more tests
- hundreds of tests!!!
What features do your tests cover?
it('adds todos', () => {
cy.visit('/')
cy.get('.new-todo')
.type('write code{enter}')
.type('write tests{enter}')
.type('deploy{enter}')
cy.get('.todo').should('have.length', 3)
})
E2E tests are extremely effective at covering a lot of app code
Coverage as a guide to writing E2E tests
it('adds todos', () => {
cy.visit('/')
cy.get('.new-todo')
.type('write code{enter}')
.type('write tests{enter}')
.type('deploy{enter}')
cy.get('.todo').should('have.length', 3)
})Coverage as a guide to writing E2E tests
it('adds todos', () => {
cy.visit('/')
cy.get('.new-todo')
.type('write code{enter}')
.type('write tests{enter}')
.type('deploy{enter}')
cy.get('.todo').should('have.length', 3)
})
We have tested "add todo"
Need tests
Ughh, missed it
Can we always write an E2E test?
What is this code doing?
Can I write an E2E test to hit this code?
Sometimes you cannot
This code should be unreachable from the user interface
Write a Unit Test!
import {getVisibleTodos} from '../../src/selectors'
describe('getVisibleTodos', () => {
it('throws an error for unknown visibility filter', () => {
expect(() => {
getVisibleTodos({
todos: [],
visibilityFilter: 'unknown-filter'
})
}).to.throw()
})
})@cypress/code-coverage plugin
We got this line from the unit test
@cypress/code-coverage plugin
Nice job, @cypress/code-coverage plugin
combines e2e and unit test coverage automatically
Visual testing
it('draws pizza correctly', function () {
cy.percySnapshot('Empty Pizza')
cy.enterDeliveryInformation()
const toppings = ['Pepperoni', 'Chili', 'Onion']
cy.pickToppings(...toppings)
// make sure the web app has updated
cy.contains('.pizza-summary__total-price', 'Total: $12.06')
cy.percySnapshot(toppings.join(' - '))
})Visual review workflow
<style type="text/css">
.st0{fill:#FFD8A1;}
.st1{fill:#E8C08A;}
- .st2{fill:#FFDC71;}
+ .st2{fill:#71FF71;}
.st3{fill:#DFBA86;}
</style>changes crust color SVG
1. Make pull request with visual changes
Visual review workflow
catch visual changes in the pull request
2. Review & approve visual changes
Visual review workflow
2. Review & approve visual changes
App Actions
// app code
var model = new app.TodoModel('react-todos')
if (window.Cypress) {
window.model = model
}
// test code
export const addTodos = (...todos) => {
cy.window()
.its('model')
.should('be.an', 'object')
.invoke('addTodo', ...todos)
}Why developers love Cypress
-
Fast, flake-free
-
GUI, time travel
-
Test recording
-
Documentation
Coming soon
-
Retries / flake factor
-
Full network stubbing
-
so many more ideas ...
Thank you 👏
@cypress_io
@bahmutov