Gleb Bahmutov, PhD
VP of Engineering, Cypress.io
🛣
End-to-end Testing
with
Why is all software broken?
Will Klein
Quality software behaves the way users expect it to behave
We going to need some tests
E2E
integration
unit
Smallest pieces
Testing Pyramid △
Unit tests pass...
E2E
integration
unit
Component
E2E
integration
unit
Website / API
E2E
integration
unit
Really important to users
Really important to developers
$ npm install -D cypress
- Win / Mac / Linux
- MIT License
- OSS https://github.com/cypress-io/cypress
- http://on.cypress.io/introduction-to-cypress
// ui-spec.js
it('loads the app', () => {
cy.visit('http://localhost:3030')
cy.get('.todoapp').should('be.visible')
})
Mocha BDD syntax
Chai assertions
it('adds 2 todos', () => {
cy.visit('http://localhost:3000')
cy.get('.new-todo')
.type('learn testing{enter}')
.type('be cool{enter}')
cy.get('.todo-list li')
.should('have.length', 2)
})
it('adds 2 todos', () => {
const user = cy
user.visit('http://localhost:3000')
user.get('.new-todo')
.type('learn testing{enter}')
.type('be cool{enter}')
user.get('.todo-list li')
.should('have.length', 2)
})
Kent C Dodds https://testingjavascript.com/
Cypress demo
- Typical test
- Failing test
- Recording tests
- CI setup
- Cypress dashboard
Example: testing how app handles failed XHR call
// app loading todos on start
loadTodos ({ commit }) {
commit('SET_LOADING', true)
axios
.get('/todos')
.then(r => r.data)
.then(todos => {
console.log('got %d todos', todos.length)
commit('SET_TODOS', todos)
commit('SET_LOADING', false)
})
.catch(e => {
console.error('could not load todos')
console.error(e.message)
console.error(e.response.data)
})
},
Application
it('handles 404 when loading todos', () => {
cy.server()
cy.route({
url: '/todos',
response: 'test does not allow it',
status: 404
})
cy.visit('/', {
onBeforeLoad: win => {
cy.spy(win.console, 'error').as('console-error')
}
})
// observe external effect from the app
cy.get('@console-error')
.should('have.been.calledWithExactly',
'test does not allow it')
})
control network
Test
it('handles 404 when loading todos', () => {
cy.server()
cy.route({
url: '/todos',
response: 'test does not allow it',
status: 404
})
cy.visit('/', {
onBeforeLoad: win => {
cy.spy(win.console, 'error').as('console-error')
}
})
// observe external effect from the app
cy.get('@console-error')
.should('have.been.calledWithExactly',
'test does not allow it')
})
spy on app's console
Test
it('handles 404 when loading todos', () => {
cy.server()
cy.route({
url: '/todos',
response: 'test does not allow it',
status: 404
})
cy.visit('/', {
onBeforeLoad: win => {
cy.spy(win.console, 'error').as('console-error')
}
})
// observe external effect from the app
cy.get('@console-error')
.should('have.been.calledWithExactly',
'test does not allow it')
})
assert it happens
Test
But how does it FEEL?
When I test with Cypress
DOM
Network
storage
If you can write E2E tests in a framework-agnostic way
You can replace framework X with Y
(without breaking things)
But how is this different from tool X?
Hint: browsers are complicated
Node to Browser actions are hard
Node
Browser
Node
Cy backend
cy.task(name, ...args)
on('task', { name: (...args) => ... })
Browser to Node is easy
cy.task cy.exec cy.request
(and you only send data, not code)
Browser to Node is easy
cy.task
Browser to Node is easy
it('finds record in the database', () => {
// random text to avoid confusion
const id = Cypress._.random(1, 1e6)
const title = `todo ${id}`
cy.get('.new-todo').type(`${title}{enter}`)
})
runs in the browser
drive via DOM
Observe Database Effect
it('finds record in the database', () => {
// random text to avoid confusion
const id = Cypress._.random(1, 1e6)
const title = `todo ${id}`
cy.get('.new-todo').type(`${title}{enter}`)
cy.task('hasSavedRecord', title).should('equal', true)
})
runs in the browser
Observe Database Effect
const hasRecordAsync = (title, ms) => {
// use promise-retry or convergence
...
}
module.exports = (on, config) => {
on('task', {
hasSavedRecord (title, ms = 3000) {
return hasRecordAsync(title, ms)
}
})
}
runs in Node in cypress/plugins/index.js
Observe Database Effect
task completes as soon as the server gets POST from the app and saves record to DB
task checks for wrong title and eventually times out
Tutorials
Api
Examples
Video course
Full workshop
-
"cypress open" - GUI interactive mode
-
"cypress run" - headless mode
Cypress CLI has 2 main commands
full video of the run, screenshots of every failure
-
Every CI should be good
-
Or use a Docker image we provide
Running E2E on CI
Other Demos
many presentations and videos about Cypress
Paid Features 💵
Paid Features 💵: artifacts
test output, video, screenshots
Paid Features 💵 - load balancing
cypress run --record
cypress run --record --group single
cypress run --record --group parallel --parallel
Most CIs should just work 🙏
Paid Features 💵 - load balancing
cypress-dashboard parallelization
# machines | run duration | time savings |
---|---|---|
1 | 22:50 | ~ |
2 | 11:47 | 48% |
3 | 7:51 | 65% |
4 | 5:56 | 74% |
6 | 3:50 | 83% |
8 | 3:00 | 87% |
10 | 2:19 | 90% |
cypress-dashboard parallelization
10 machines = 10x speed up
Why developers love Cypress
-
Fast, flake-free
-
GUI, time travel
-
Test recording
-
Documentation
Why developers love Cypress
-
Fast, flake-free
-
GUI, time travel
-
Test recording
-
Documentation
100% FREE & OSS
Cool Things You Can do With Cypress
Test components from frameworks
Test components from these frameworks with ease
Cool Things You Can do With Cypress
Control your application's behavior
Cool Things You Can do With Cypress
Change Cypress color theme
Coming soon
-
Retries / flake factor
-
Cross-browser
-
Full network stubbing
-
so many more ideas ...
IE11
Gleb Bahmutov, PhD
VP of Engineering
🛣
Thank you
@cypress_io
@bahmutov
slides.com/bahmutov/cypress-reactive-roadshow
github.com/cypress-io/cypress ⭐️