Gleb Bahmutov PRO
JavaScript ninja, image processing expert, software quality fanatic
Network Control for End-to-End Web Testing
Gleb Bahmutov
LA Fires
brought to you by the fossil fuel companies
image source: https://www.dailysabah.com/world/americas/more-damage-anticipated-as-california-fire-season-sets-records
Control your life
Join an organization
Vote
Greenpeace 350 Sunrise Citizen Climate Lobby
Speaker: Gleb Bahmutov PhD
C / C++ / C# / Java / CoffeeScript / JavaScript / Node / Angular / Vue / Cycle.js / functional programming / testing
Gleb Bahmutov
Sr Director of Engineering
🌎 🔥 350.org 🌎 🔥 citizensclimatelobby.org 🌎 🔥
Mercari Does A Lot Of Testing
A typical Mercari US Cypress E2E test
Example app
Web vs App server
- Web server: http, css, static js
- nginx
- apache
- github pages
- App server: server-side code execution
- Node.js
- Go
- PHP
Web vs App server
- Web server: http, css, static js
- Simple!
- Caching
- Edge caching
- App server: server-side code execution
- Slow
- Shared data
- Security
Typical App Server Test
- reset the data (if possible)
- interact with the application
// branch a1
// cypress/e2e/add-todo.cy.js
beforeEach(() => {
cy.request('POST', '/reset', { todos: [] })
})
it('adds a todo', () => {
cy.visit('/')
cy.log('**confirm the items are loaded**')
cy.get('.loaded')
cy.get('.new-todo').type('item 1{enter}')
cy.get('li.todo').should('have.length', 1)
cy.get('.new-todo').type('item 2{enter}')
cy.get('li.todo').should('have.length', 2)
cy.log('**confirm the items are saved**')
cy.reload()
cy.get('li.todo').should('have.length', 2)
})Reminder: Switch to branch "a1"
The test is slow!
Goal:
- Avoid app server during testing
- Testing is SO much simpler
- Parallel testing
- Make the tests faster
Step 1: Record API Calls 🎥
// branch a2
// cypress.config.js
// https://github.com/bahmutov/cypress-magic-backend
magicBackend: {
// this app makes "XHR" calls to load and update "/todos"
// match calls like
// GET /todos
// POST /todos
// DELETE /todos/1234
apiCallsToIntercept: [
// TODO: insert the intercept definitions
],
},
Press the "🪄 🎥" Record button
Reminder: Switch to branch "a2" and restart the app
Saved JSON file with API calls
{
"pluginName": "cypress-magic-backend",
"pluginVersion": "1.11.0",
"specName": "cypress/e2e/add-todo.cy.js",
"testName": "adds a todo",
"apiCallsInThisTest": [
{
"method": "GET",
"url": "/todos",
"request": "",
"response": [],
"duration": 1013
},
{
"method": "POST",
"url": "/todos",
"request": {
"title": "item 1",
"completed": false,
"id": "6895127373"
},
"response": {
"title": "item 1",
"completed": false,
"id": "6895127373"
},
"duration": 1009
},
{
"method": "POST",
"url": "/todos",
"request": {
"title": "item 2",
"completed": false,
"id": "6787924392"
},
"response": {
"title": "item 2",
"completed": false,
"id": "6787924392"
},
"duration": 1008
},
{
"method": "GET",
"url": "/todos",
"request": "",
"response": [
{
"title": "item 1",
"completed": false,
"id": "6895127373"
},
{
"title": "item 2",
"completed": false,
"id": "6787924392"
}
],
"duration": 1010
}
]
}cypress/magic-backend/cypress/e2e/add-todo.cy.js_adds_a_todo_api_calls.json
Magic Replay Mode
We do not need the "POST /reset" API call in the replay mode
// branch a2
// cypress/e2e/add-todo.cy.js
beforeEach(() => {
const mode = Cypress.env('magic_backend_mode')
if (mode !== 'playback') {
mode && cy.log(`during the test the mode is "${mode}"`)
cy.request('POST', '/reset', { todos: [] })
}
})cypress-magic-backend: from 6s to 0.6s
cypress-watch-and-reload
// branch a3
// cypress.config.js
// list the files and file patterns to watch
// https://github.com/bahmutov/cypress-watch-and-reload
'cypress-watch-and-reload': {
watch: ['index.html', 'app.js'],
},Use the "locked" Replay Mode "🪄 🎞️ ☑️"
Reminder: Switch to branch "a3" and restart the app
What does this "🪄 🧐" button do?
The "Inspect" button is my ❤️
Reminder: Switch to branch "a4" and restart the app
Hmm, something has changed in our API,
the "POST /todos" call is now twice slower!
You can see more details about the observed timing differences by clicking on the warning in the Command Log
Reminder: Switch to branch "a5" and restart the app
Hmm, something isn't right
Even if the test is green...
But where could the problem be ...
Run in the "Inspect Mode" and see exactly what the problem is and where it started
A change in the front-end code started the entire chain of errors!
cypress-magic-backend
Inspect Mode 🪄 🧐
- Compared API calls against recorded JSON files
- Reports timing changes
- Reports schema changes
- But not the values!
Magic-backed as a Service
- No need to manage JSON files
- More information for the inspect mode
Reminder: Switch to branch "a6"
run "npm install" and restart the app
using the "as-a ." command to use the API key
// branch a6
// cypress.config.js
magicBackend: {
// where to store recorded API calls?
// local: store all API calls locally in JSON files
// remote: send API calls to a remote server at cypress.tips
store: 'remote',
}Works just like before but without local JSON files
Numbers example
Run the "number.cy.js" spec
Change the input number
// branch a6
// cypress/e2e/number.cy.js
// change to produce some negative numbers
const random = Cypress._.random(3, 30)Try getting both passing and failing tests
then open DevTools console and see a failed test
Can you guess which input leads to a failure?
Learn More 🎓
- https://github.com/bahmutov/cypress-magic-backend
- 📝 read the blog post "Magic Backed For E2E Testing" https://glebbahmutov.com/blog/magic-backend/
- 📝 read the blog post "Magic Backed Inspection Mode" https://glebbahmutov.com/blog/magic-backend-inspect-mode/
- 📝 read the blog post "Watch Mock And Reload" https://glebbahmutov.com/blog/watch-mock-reload/
- 📝 read the blog post "Mock But Verify" https://glebbahmutov.com/blog/mock-but-verify/
- 📺 videos on the plugin at https://youtube.com/@gleb
OSS and/or Money
400 OSS
projects
plus example repos, blog posts, videos, GH issues triage
Time could it be...
Reality
Can this continue?
Is it fair?
- OSS
- Talks
- Blog posts
- Videos
- Workshops
- Online courses
Is There A Better Way?
Dual license:
< 100 employees, free to use non-transferable
>= 100 employees, small license, non-transferable
- Free use for small companies and individuals to capture market share
- Small / large fee for large companies to pay for suppoer
I don't know what the future looks like
I would like to give OSS the effort it deserves. OSS is worth it.
Network Control for End-to-End Web Testing
Gleb Bahmutov
👏 Thank you 👏
Network Control for End-to-End Web Testing
By Gleb Bahmutov
Network Control for End-to-End Web Testing
A modern web application probably makes tens of network calls during a typical user flow. A good end-to-end web test must observe and in some cases mock the network calls to make the test reliable, complete, and easy to debug. I will show some of my tips and tricks for using and controlling network API calls during tests. We will see how Cypress, Playwright, and Vitest implement network spies and stubs, and what we can achieve by writing just a little bit of custom code on top of existing features.
