Stefano Magni
I'm a passionate Front-end Engineer, a speaker, and an instructor. I love creating high-quality products, testing and automating everything, learning and sharing my knowledge, helping people. I work remotely for Hasura.
Stefano Magni
I'm a passionate Front-end engineer, and an instructor.
Â
I'm in the Platform team of Hasura, a SAAS that gives you instant GraphQL & REST APIs on new & existing data sources.
You can find all my articles, courses, OSS contributions here
Including UI Testing Best Practices (1.4K âïļ)
The E2E tests are slow
â ïļ cy.wait(10_000) â ïļ
Slow ðĒ
Nonexplanatory ðĪ
Possibly flaky ðĪĶââïļ
Let's fix it!... ?
Multiple requests ðģ
Unpredictable order ðĨķ
Different servers ð
The E2E tests are slow
Approaches
Collect all the outgoing requests and
wait for their completion ðĪĶââïļ
Intercept the same request to
different servers ð
Wait for something that reflects the response ðĪ·ââïļ
(commenting the whys)
The E2E tests are slow
The E2E tests are flaky
Undermine confidence and trust ðĪ
Add friction âïļ
The E2E tests are flaky
CI misconfiguration
Slow CI ðĒ
Tests went outdated ðī
We were paying for them ðļ
The E2E tests are flaky
The E2E tests are cryptic
Tests' goals
To test features ðĪŠ
Ease the readers' job ðĪ
Ease debugging themselves ðĄ
The E2E tests are flaky
Abstraction
Requires creating a mental model ðĩââïļ
Gets in the way of debugging âïļ
The E2E tests are cryptic
Abstraction
The E2E tests are cryptic
// spec.ts file
it('Create Query Action', createQueryAction);
// test.ts file (simplified version)
export const createMutationAction = () => {
// ...
clearActionDef();
typeIntoActionDef(statements.createMutationActionText);
clearActionTypes();
// ...
};
// test.ts file contains the clearActionDef, typeIntoActionDef, etc.
const clearActionDef = () => {
cy.get('textarea').first().type('{selectall}', { force: true });
cy.get('textarea').first().trigger('keydown', {
keyCode: 46,
which: 46,
force: true,
});
};
// test.ts file contains also the statements
const statements = {
createMutationActionText: `type Mutation {
login (username: String!, password: String!): LoginResponse
}`,
createMutationCustomType: `type LoginResponse {
accessToken: String!
}
`,
createMutationHandler: 'https://hasura-actions-demo.glitch.me/login',
// ...
}
Abstraction
The E2E tests are cryptic
it('Create Query Action', () => {
cy.get('textarea').eq(0).as('actionDefinitionTextarea');
cy.get('textarea').eq(1).as('typeConfigurationTextarea');
cy.get('@actionDefinitionTextarea')
.clearConsoleTextarea()
.type(
`type Mutation {
login (username: String!, password: String!): LoginResponse
}`,
{ force: true, delay: 0 }
);
cy.get('@typeConfigurationTextarea')
.clearConsoleTextarea()
.type(
`type LoginResponse {
accessToken: String!
}
`,
{ force: true, delay: 0 }
);
// ...
})
When abstraction is good?
When hides workarounds ðŦĢ
When they are soft ð
The E2E tests are cryptic
Abstraction
The E2E tests are cryptic
/**
* Clear a Console's textarea.
* Work around cy.clear sometimes not
* working in the Console's textareas.
*/
Cypress.Commands.add('clearConsoleTextarea',
{ prevSubject: 'element' },
el => {
cy.wrap(el).type('{selectall}',
{ force: true })
.trigger('keydown', {
keyCode: 46,
which: 46,
force: true,
});
});
Abstraction
The E2E tests are cryptic
// â bad abstraction
export const expectNotification = (
{
type,
title,
message,
}: {
type: 'success' | 'error';
title: string;
message?: string;
},
timeout = 10000
) => {
const el = cy.get(
type === 'success' ? '.notification-success' : '.notification-error',
{ timeout }
);
el.should('be.visible');
el.should('contain', title);
if (message) el.should('contain', message);
};
// â
good abstraction
function expectSuccessNotification(title: string) {
cy.get('.notification-success')
.should('be.visible')
.should('contain', title)
}
Matching the test's code and test runner's commands
The E2E tests are cryptic
The E2E tests are cryptic
Use clear selectors
The E2E tests are cryptic
cy.get('textarea') // ðĪ
.eq(0) // ðĪ
.type(`{enter}{uparrow}${statements.createMutationGQLQuery}`, {
force: true,
});
cy.get('textarea').eq(0).as('actionDefinitionTextarea');
cy.get('textarea').eq(1).as('typeConfigurationTextarea');
// âĪïļ
cy.get('@actionDefinitionTextarea').clearConsoleTextarea().type(/* ... */);
The E2E tests are cryptic
There's more in the article
Reduce data-testid attributes
Group related actions
Debugging the E2E tests is challenging
What do you prefer?
Small and dependent E2E tests
Small and independent E2E tests
One long E2E test
Debugging the E2E tests is challenging
Small and dependent E2E tests
It's an antipattern ð
Impossible to run tests in isolation ð
A fake sense of the test's size ð
Debugging the E2E tests is challenging
Small and dependent E2E tests
START (the application state is empty)
Test 1: create the entity
Test 2: modify the entity
Test 3: delete the entity
END (the application state is empty)
Debugging the E2E tests is challenging
Small and independent E2E tests
They can run in isolation ð
It's a best practice ð
Making them fast requires boilerplate ð
Debugging the E2E tests is challenging
Making them fast is boilerplaty?
Every test should use the previous app state or create it ðą
Debugging the E2E tests is challenging
Small and independent E2E tests
1. START (the application state is empty)
2. Test 1: create the entity
1. BEFORE: Load the page (the application state is empty)
2. create the entity
3. AFTER: Delete the entity (the application state is empty)
3. Test 2: modify the entity
1. BEFORE: create the entity (through APIs)
2. BEFORE: Load the page (the application state is empty)
3. modify the entity
4. AFTER: Delete the entity (through APIs, the application state is empty)
4. Test 3: delete the entity
1. BEFORE: create the entity (through APIs)
2. BEFORE: Load the page (the application state is empty)
3. delete the entity
4. AFTER: Delete the action (the application state is empty)
5. END (the application state is empty)
Debugging the E2E tests is challenging
Small and independent E2E tests
1. START (the application state is empty)
2. Test 1: create the entity
1. BEFORE: Does the entity exist?
1. NO: it's ok!
2. YES: delete the entity (through APIs)
2. BEFORE: Load the page (the application state is empty)
3. create the entity
3. Test 2: modify the entity
1. BEFORE: Does the entity exist?
1. YES: it's ok!
2. NO: create the entity (through APIs)
2. BEFORE: Does the entity already includes the change the test is going to make?
1. YES: it's ok!
2. NO: modify the entity (through APIs)
3. BEFORE: Are we already on the correct page?
1. YES: it's ok!
2. NO: load the page
4. modify the entity
And so on...
Debugging the E2E tests is challenging
One long E2E test
It's long ð
It's not modular ð
Doesn't lie ðĪŠ
Debugging the E2E tests is challenging
One long E2E test
1. START (the application state is empty)
2. Test: CRUD
1. BEFORE: Delete the entity if it exists (the application state is empty)
2. BEFORE: Load the page
3. create the entity
4. modify the entity
5. delete the entity
6. AFTER: Delete the entity if it exists (the application state is empty)
3. END (the application state is empty)
Debugging the E2E tests is challenging
The E2E nature is the problem...
Server-free tests are faster ð
Application state problems do not exist ð
Must be kept in sync with the server changes ð (TypeScript for the win!)
The server teams use the E2E tests also to test the server
For every server change...
Does the Console's E2E work? Nice! ð
... Who said CI misconfiguration? ð
Server and Console tests must be split!
Splitting server and Console tests
UI-free tests are faster ð
Server-free tests are faster ð
The server teams use the E2E tests also to test the server
A lot of work...
Refactored some E2E tests...
Tracked and skipped all the flaky tests...
Asked the Feature teams to fix the tests...
The future
More Storybook tests ðïļ
TypeScript types generated by the server ðą
Playwright instead of Cypress ðïļ
Stefano Magni
By Stefano Magni
I joined Hasura in May 2022, and one of my first tasks was to fix the E2E tests of the Hasura Console, main Hasura's front-end application. The main problems were: they were slow, and they were flaky. Then, by digging into the topic, there was more to say, more to decide, more to fix, and more to do. This presentation is about what we found out, the problems, the best practices, etc.
I'm a passionate Front-end Engineer, a speaker, and an instructor. I love creating high-quality products, testing and automating everything, learning and sharing my knowledge, helping people. I work remotely for Hasura.