Gleb Bahmutov PRO
JavaScript ninja, image processing expert, software quality fanatic
Flexible Data Setup And Caching For Cypress.io E2E Tests
Gleb Bahmutov, @bahmutov
our planet is in imminent danger
https://lizkeogh.com/2019/07/02/off-the-charts/
+3 degrees Celsius will be the end and we are close
survival is possible* but we need to act now
- change your life
- change your bank
- join an organization
- vote and protest
Speaker: Gleb Bahmutov PhD
C / C++ / C# / Java / CoffeeScript / JavaScript / Node / Angular / Vue / Cycle.js / functional programming / testing
Gleb Bahmutov
Sr Director of Engineering
Chat application
Test:
Send and receive messages
Chat room
User
Create manually
Pass to the test runner
{
"env": {
"username": "gleb1",
"password": "gleb1"
}
}# Cypress config and the spec files
const username = Cypress.env('username')
const password = Cypress.env('password')
cy.visit('/')
cy.get('.login-form').within(() => {
cy.get('[placeholder=username]').type(username)
cy.get('[placeholder=password]').type(password)
cy.contains('button', 'login').click()
})
cy.location('pathname').should('equal', '/rooms')cypress.json config file
cypress spec file
$ npx cypress open
# or pass the username and password via CLI arguments
$ npx cypress open --env username=gleb1,password=gleb1🚨 Slow
🚨 Not repeatable
🚨 Hard to run tests in parallel
Create manually
Pass to the test runner
🚨 Slow
🚨 Not repeatable
🚨 Hard to run tests in parallel
Create manually
Pass to the test runner
How did I create this user? Ughh, does it need permission X set? Let me ask around...
🚨 Slow
🚨 Not repeatable
🚨 Hard to run tests in parallel
Create manually
Pass to the test runner
At MercariUS we only use env variables to pass basic auth that allows one to visit the deployed dev environment URL
const username = Cypress.env('username')
const password = Cypress.env('password')
cy.visit('/', {
auth: {username, password}
})Alternative: Create user in the test
it('registers user in the test', () => {
const username = 'Test'
const password = 'MySecreT'
registerUser(username, password)
loginUser(username, password)
// if the user has been created and could log in
// we should be redirected to the home page with the rooms
cy.location('pathname').should('equal', '/rooms')
})- ✅ First test run
- 🚨 Second test run
it('creates a random user', () => {
const username = 'Gleb-' + Cypress._.random(1e3)
const password = '¡SoSecret!'
registerUser(username, password)
loginUser(username, password)
cy.location('pathname').should('equal', '/rooms')
cy.contains('.user-info', username)
})Generate using random data
The test should create all the data it needs
Tip: do not clean up the created data from the test itself.
If test data grows to be a problem, use a dedicated cron job to delete it
The test should create all the data it needs
- ✅ Tests are independent of each other
- ✅ Repeatable
- 🚨 Might be slow
Dealing with caching edge-cases
5
6
Real-world testing success at MercariUS and Extend
The End
4
Dependencies between data sessions; sharing across specs
1
Introduction and the data creation problem
2
Creating and caching the user, the session, the room using custom code
3
Creating data using cypress-data-session plugin
The test should create all the data it needs
once
let username
let password
beforeEach(() => {
if (!username) {
username = 'Gleb-' + Cypress._.random(1e3)
password = '¡SoSecret!'
registerUser(username, password)
}
})
it('creates a random user once', () => {
loginUser(username, password)
cy.location('pathname').should('equal', '/rooms')
cy.contains('.user-info', username)
})The test should create all the data it needs
once
let username
let password
beforeEach(() => {
if (!username) {
username = 'Gleb-' + Cypress._.random(1e3)
password = '¡SoSecret!'
registerUser(username, password)
}
})
it('creates a random user once', () => {
loginUser(username, password)
cy.location('pathname').should('equal', '/rooms')
cy.contains('.user-info', username)
})Not going to work
variables are reset during each test run
Use more permanent memory to store the created data
Cypress.env https://on.cypress.io/env
let username
let password
beforeEach(() => {
username = Cypress.env('username')
password = Cypress.env('password')
if (!username) {
username = 'Gleb-' + Cypress._.random(1e3)
password = '¡SoSecret!'
Cypress.env('username', username)
Cypress.env('password', password)
registerUser(username, password)
}
})Use more permanent memory to store the created data
Cypress.env https://on.cypress.io/env
let username
let password
beforeEach(() => {
username = Cypress.env('username')
password = Cypress.env('password')
if (!username) {
username = 'Gleb-' + Cypress._.random(1e3)
password = '¡SoSecret!'
Cypress.env('username', username)
Cypress.env('password', password)
registerUser(username, password)
}
})
Look up the created user from DevTools
- 🐢 create + login 4.67s
- 🐎 login 2.3s
Reusing cached data makes the tests faster
How to log in instantly
When the server logs the user, it sends back the session cookie named "connect.sid"
Cypress can read and set cookies:
- cy.getCookie
- cy.setCookie
- cy.clearCookie
let username
let password
beforeEach(() => {
username = Cypress.env('username')
password = Cypress.env('password')
if (!username) {
username = 'Gleb-' + Cypress._.random(1e3)
password = '¡SoSecret!'
Cypress.env('username', username)
Cypress.env('password', password)
registerUser(username, password)
}
})
beforeEach(() => {
if (Cypress.env('cookie')) {
cy.setCookie('connect.sid', Cypress.env('cookie'))
cy.visit('/')
} else {
loginUser(username, password)
cy.getCookie('connect.sid').then((cookie) => {
Cypress.env('cookie', cookie.value)
})
}
})
it('logs in using a cookie', () => {
cy.location('pathname').should('equal', '/rooms')
cy.contains('.user-info', username)
})- Create and cache the user as before
- Get and cache the cookie
- 🐢 create + login 4.67s
- 🐎 login 2.3s
- 🚀 cookie 0.3s
Create a room if necessary
beforeEach(() => {
if (!Cypress.env('room')) {
const roomName = 'Room-' + Cypress._.random(1e3)
cy.get('input[aria-label="New room name"]')
.type(roomName)
.next()
.click()
cy.contains('.room-item', roomName).should('be.visible')
Cypress.env('room', roomName)
}
})Pass the room name via alias
beforeEach(() => {
if (!Cypress.env('room')) {
const roomName = 'Room-' + Cypress._.random(1e3)
cy.get('input[aria-label="New room name"]')
.type(roomName)
.next()
.click()
cy.contains('.room-item', roomName).should('be.visible')
Cypress.env('room', roomName)
cy.wrap(roomName).as('roomName')
} else {
// need to set the alias before each test
cy.wrap(Cypress.env('room')).as('roomName')
}
})
Use the alias in the test
beforeEach(() => {
cy.wrap(roomName).as('roomName')
})
it('logs in using a cookie', function () {
cy.location('pathname').should('equal', '/rooms')
cy.contains('.user-info', username)
cy.contains('.room-item', this.roomName).click()
cy.location('pathname').should('include', '/chat/')
cy.contains('.chat-room', this.roomName).should('be.visible')
})Use the alias in the test
beforeEach(() => {
cy.wrap(roomName).as('roomName')
})
it('logs in using a cookie', function () {
cy.location('pathname').should('equal', '/rooms')
cy.contains('.user-info', username)
cy.contains('.room-item', this.roomName).click()
cy.location('pathname').should('include', '/chat/')
cy.contains('.chat-room', this.roomName).should('be.visible')
})Common patterns
let username
let password
beforeEach(() => {
username = Cypress.env('username')
password = Cypress.env('password')
if (!username) {
username = 'Gleb-' + Cypress._.random(1e3)
password = '¡SoSecret!'
Cypress.env('username', username)
Cypress.env('password', password)
registerUser(username, password)
}
})⠔ checking memory
beforeEach(() => {
if (Cypress.env('cookie')) {
cy.setCookie('connect.sid', Cypress.env('cookie'))
cy.visit('')
} else {
loginUser(username, password)
cy.getCookie('connect.sid').then((cookie) => {
Cypress.env('cookie', cookie.value)
})
}
})beforeEach(() => {
if (!Cypress.env('room')) {
const roomName = 'Room-' + Cypress._.random(1e3)
cy.get('input[aria-label="New room name"]')
...
Cypress.env('room', roomName)
cy.wrap(roomName).as('roomName')
} else {
// need to set the alias before each test
cy.wrap(Cypress.env('room')).as('roomName')
}
})Common patterns
let username
let password
beforeEach(() => {
username = Cypress.env('username')
password = Cypress.env('password')
if (!username) {
username = 'Gleb-' + Cypress._.random(1e3)
password = '¡SoSecret!'
Cypress.env('username', username)
Cypress.env('password', password)
registerUser(username, password)
}
})⠜ "setup" vs "recreate"
beforeEach(() => {
if (!Cypress.env('cookie')) {
loginUser(username, password)
cy.getCookie('connect.sid').then((cookie) => {
Cypress.env('cookie', cookie.value)
})
} else {
cy.setCookie('connect.sid', Cypress.env('cookie'))
cy.visit('/')
}
})beforeEach(() => {
if (!Cypress.env('room')) {
const roomName = 'Room-' + Cypress._.random(1e3)
cy.get('input[aria-label="New room name"]')
...
Cypress.env('room', roomName)
cy.wrap(roomName).as('roomName')
} else {
// need to set the alias before each test
cy.wrap(Cypress.env('room')).as('roomName')
}
})Common patterns
let username
let password
beforeEach(() => {
username = Cypress.env('username')
password = Cypress.env('password')
if (!username) {
username = 'Gleb-' + Cypress._.random(1e3)
password = '¡SoSecret!'
Cypress.env('username', username)
Cypress.env('password', password)
registerUser(username, password)
}
})⠜ saving values
beforeEach(() => {
if (!Cypress.env('cookie')) {
loginUser(username, password)
cy.getCookie('connect.sid').then((cookie) => {
Cypress.env('cookie', cookie.value)
})
} else {
cy.setCookie('connect.sid', Cypress.env('cookie'))
cy.visit('/')
}
})beforeEach(() => {
if (!Cypress.env('room')) {
const roomName = 'Room-' + Cypress._.random(1e3)
cy.get('input[aria-label="New room name"]')
...
Cypress.env('room', roomName)
cy.wrap(roomName).as('roomName')
} else {
// need to set the alias before each test
cy.wrap(Cypress.env('room')).as('roomName')
}
})cypress-data-session
let username
let password
beforeEach(() => {
username = Cypress.env('username')
password = Cypress.env('password')
if (!username) {
username = 'Gleb-' + random(1e3)
password = '¡SoSecret!'
Cypress.env('username', username)
Cypress.env('password', password)
registerUser(username, password)
}
})beforeEach(() => {
cy.dataSession({
name: 'user',
setup() {
const username = 'Gleb-' + random(1e3)
const password = '¡SoSecret!'
registerUser(username, password)
cy.wrap({ username, password })
},
})
})Cypress command for flexible test data setup
cypress-data-session@v2
beforeEach(() => {
cy.dataSession({
name: 'user',
setup() {
const username = 'Gleb-' + random(1e3)
const password = '¡SoSecret!'
registerUser(username, password)
cy.wrap({ username, password })
},
})
})beforeEach(() => {
cy.dataSession({
name: 'user',
setup() {
const username = 'Gleb-' + random(1e3)
const password = '¡SoSecret!'
registerUser(username, password)
cy.wrap({ username, password })
},
})
})
it('has a user', function () {
// each value is stored under the session name
expect(this.user.password).to.equal('¡SoSecret!')
})The value yielded by the last setup command is cached in memory
cypress-data-session
beforeEach(() => {
if (Cypress.env('cookie')) {
cy.setCookie('connect.sid', Cypress.env('cookie'))
cy.visit('')
} else {
loginUser(username, password)
cy.getCookie('connect.sid').then((cookie) => {
Cypress.env('cookie', cookie.value)
})
}
})beforeEach(function () {
cy.dataSession({
name: 'login',
setup() {
loginUser(this.user.username, this.user.password)
cy.getCookie('connect.sid').its('value')
},
recreate(value) {
cy.setCookie('connect.sid', value)
cy.visit('/')
},
})
})Store the cookie using data session
First test run
creates the user
creates the session
Second test run
takes the saved user and session
cypress-data-session
beforeEach(() => {
if (!Cypress.env('room')) {
const roomName = 'Room-' + Cypress._.random(1e3)
cy.get('input[aria-label="New room name"]')
...
Cypress.env('room', roomName)
cy.wrap(roomName).as('roomName')
} else {
// need to set the alias before each test
cy.wrap(Cypress.env('room')).as('roomName')
}
})beforeEach(() => {
cy.dataSession({
name: 'roomName',
setup() {
const roomName = 'Room-' + random(1e3)
cy.get('input[aria-label="New room name"]')
.type(roomName)
.next()
.click()
cy.wrap(roomName)
},
})
})Store the created room using data session
it('uses data session', function () {
cy.location('pathname').should('equal', '/rooms')
cy.contains('.user-info', this.user.username)
cy.contains('.room-item', this.roomName).click()
cy.location('pathname').should('include', '/chat/')
cy.contains('.chat-room', this.roomName).should('be.visible')
})value from the last command in the "setup" method
See the data session value from DevTools
Hard browser reload ⌘-R clears Cypress.env
beforeEach(() => {
cy.dataSession({
name: 'user',
setup() {
const username = 'Gleb-' + random(1e3)
const password = '¡SoSecret!'
registerUser(username, password)
cy.wrap({ username, password })
},
+ shareAcrossSpecs: true
})
}) cy.dataSession({
name: 'login',
setup() {
loginUser(this.user.username, this.user.password)
cy.getCookie('connect.sid').its('value')
},
recreate(value) {
cy.setCookie('connect.sid', value)
cy.visit('/')
},
+ shareAcrossSpecs: true,
}) cy.dataSession({
name: 'roomName',
setup() {
const roomName = 'Room-' + random(1e3)
cy.get('input[aria-label="New room name"]')
.type(roomName)
.next()
.click()
cy.contains('.room-item', roomName).should('be.visible')
cy.wrap(roomName)
},
+ shareAcrossSpecs: true,
})shareAcrossSpecs: true 😄
data sessions
Cypress process
Browser process
Data can be cached in the browser or in the Cypress plugins process (shareAcrossSpecs: true)
Dealing with caching edge-cases
5
6
Real-world testing success at MercariUS and Extend
The End
4
Dependencies between data sessions; sharing across specs
1
Introduction and the data creation problem
2
Creating and caching the user, the session, the room using custom code
3
Creating data using cypress-data-session plugin
Test:
Send and receive messages
Chat room
User
What if the user is no longer there?
Delete the user session
Hmm, the "login" data session still uses the old cookie for the first user
cy.dataSession({
name: 'login',
setup() {
loginUser(this.user.username, this.user.password)
cy.getCookie('connect.sid').its('value')
},
recreate(value) {
cy.setCookie('connect.sid', value)
cy.visit('/')
},
shareAcrossSpecs: true,
+ dependsOn: ['user']
})
Use directed acyclical graph to recompute data sessions
Cached data vs real data
beforeEach(() => {
cy.dataSession({
name: 'user',
setup() {
const username = 'Gleb-' + random(1e3)
const password = '¡SoSecret!'
registerUser(username, password)
cy.wrap({ username, password })
},
})
})- The user is created
- Another test deletes all users
- The test tries to use the user that no longer exists in the database
- ✅ memory
- ❌ database
Need to verify the data first
beforeEach(() => {
cy.dataSession({
name: 'user',
setup() {
const username = 'Gleb-' + random(1e3)
const password = '¡SoSecret!'
registerUser(username, password)
cy.wrap({ username, password })
},
validate(user) {
// check if the user is still valid
// yield true | false
}
})
})Need to verify the data first
beforeEach(() => {
cy.dataSession({
name: 'user',
setup() {
const username = 'Gleb-' + random(1e3)
const password = '¡SoSecret!'
registerUser(username, password)
cy.wrap({ username, password })
},
validate(user) {
return cy.task('findUser', user.username)
}
})
})// cypress/plugins/index.js
const database = require('../../app/database')
async function findUser(username) {
console.log('find user', username)
if (typeof username !== 'string') {
throw new Error('username must be a string')
}
return database.models.user.findOne({ username })
}
on('task', {
findUser
})browser test file
Cypress plugins file
Need to verify the data first
beforeEach(() => {
cy.dataSession({
name: 'user',
setup() {
const username = 'Gleb-' + random(1e3)
const password = '¡SoSecret!'
registerUser(username, password)
cy.wrap({ username, password })
},
validate(user) {
return cy.task('findUser', user.username)
}
})
})browser test file
Initialize from real data
beforeEach(() => {
cy.dataSession({
name: 'user',
setup() {
// no ramdom data, just the username
const username = 'Gleb'
const password = '¡SoSecret!'
registerUser(username, password)
cy.wrap({ username, password })
},
validate(user) {
return cy.task('findUser', user.username)
}
})
})- The user "Gleb" is created
- The next time Cypress is loaded, it tries to create the user "Gleb"
- The test fails - there is a user with the username "Gleb" in the database
- ❌ memory
- ✅ database
Initialize from real data
beforeEach(() => {
// no ramdom data, just the username
const username = 'Gleb'
const password = '¡SoSecret!'
cy.dataSession({
name: 'user',
init() {
cy.task('findUser', username)
},
setup() {
registerUser(username, password)
cy.wrap({ username, password })
},
validate(user) {
return cy.task('findUser', user.username)
}
})
})
cypress-data-session control flow
Take shortcuts for ⏱
-
Go through the page
-
Go through the API calls
-
Go through the server code / database
tasks
Cypress process
Browser process
App api
Database
Take shortcuts
Testing the user registration
cy.visit('/')
cy.get('.register-form')
.within(() => {
cy.get('[placeholder=username]')
.type(username)
cy.get('[placeholder=password]')
.type(password)
cy.contains('button', 'register')
.click()
})User data session
cy.dataSession({
name: 'user',
setup () {
cy.visit('/')
cy.get('.register-form')
.within(() => {
cy.get('[placeholder=username]')
.type(username)
cy.get('[placeholder=password]')
.type(password)
cy.contains('button', 'register')
.click()
})
cy.wrap({ username, password })
}
})
Take shortcuts
Testing the user registration
cy.visit('/')
cy.get('.register-form')
.within(() => {
cy.get('[placeholder=username]')
.type(username)
cy.get('[placeholder=password]')
.type(password)
cy.contains('button', 'register')
.click()
})User data session
cy.dataSession({
name: 'user',
setup () {
cy.task('makeUser', { username, password })
cy.wrap({ username, password })
}
})
Take shortcuts
Testing the user login
cy.visit('/')
cy.get('.login-form')
.within(() => {
cy.get('[placeholder=username]')
.type(username)
cy.get('[placeholder=password]')
.type(password)
cy.contains('button', 'login')
.click()
})
cy.location('pathname')
.should('equal', '/rooms')
cy.contains('.user-info', username)Login data session
cy.dataSession({
name: 'login',
setup() {
cy.request({
method: 'POST',
url: '/login',
form: true,
body: {
username: this.user.username,
password: this.user.password,
},
})
cy.getCookie('connect.sid').its('value')
},
recreate(value) {
cy.setCookie('connect.sid', value)
},
shareAcrossSpecs: true,
dependsOn: ['user'],
})Take shortcuts
Testing the user login
Login data session
{Summary}
Creating complex data to use during tests is its own problem.
- Speed
- Use
- Consistency
{Summary}
My solution is the cypress-data-session plugin
- Speed
- Use
- Consistency
{Summary}
My solution is the cypress-data-session plugin
- Speed
Only create the test data once, use API / database
- Use
- Consistency
{Summary}
My solution is the cypress-data-session plugin
Create aliases automatically
- Speed
- Use
- Consistency
{Summary}
My solution is the cypress-data-session plugin
validate(), init(), recreate() + invalidation API
- Speed
- Use
- Consistency
cypress-data-session is used at MercariUS
abstractions that use cy.dataSession
cypress-data-session is used at MercariUS
during the test
cypress-data-session is used at Extend https://www.extend.com/
Thank You 👏
Gleb Bahmutov, @bahmutov
Examples, blog posts, videos
Flexible Data Setup & Caching for Cypress End-to-End Tests
By Gleb Bahmutov
Flexible Data Setup & Caching for Cypress End-to-End Tests
Gleb will introduce you to a very powerful way of creating and re-using data in your Cypress.io end-to-end tests. By re-using objects like users, projects, etc. that are typically very resource-intensive to create, you will make your tests much more efficient, faster, easier to read, and simpler to maintain. The key takeaways from this talk are: - How to measure and understand where your end-to-end tests spend the majority of their time. - How to make parts of the test faster by skipping the user interface and driving the application directly through its HTTP API - How to control the state of the application by resetting it before each spec file and before each individual test - How to make the application tests faster by caching and reusing test data, while keeping the tests independent from each other - The trade-offs between shared backend database vs individual databases for each test agent running in its own container on CI
