Tiffany Le-Nguyen
Staff Frontend Engineer at Netlify ✈️
USER-CENTRIC TESTING FOR 2020:
How Expedia is Modernizing its Testing for the Web
🐧
Hallo I'm Tiffany.
Engineer at ExpediaGroup and newbie speaker 🐣.
Mentor for freeCodeCamp Montreal
Work on terminal shell Starship sometimes.
twitter/sirmerr
github.com/sirmerr
linkedin.com/in/tiffanyln
A GOOD TESTING PIPELINE IS...
a pipeline that captures unintended changes affecting your users
a pipeline that gives you confidence that production won't break with new changes
a pipeline that provides quick and effective feedback
🤔✅❌
Are the following tests generally GOOD or BAD, and WHY?
This visual test caught that the image changed from Banff National Park to a screaming frog 🐸
Type your answers in the group chat 🕐
✅ Good catch. Actually impacts the user experience
This snapshot test caught that the DOM structure for the heading has changed.
The dev removed a div.
<div>
<h1>
Find hotels near Banff National Park, Alberta
</h1>
</div>
<h1>
Find hotels near Banff National Park, Alberta
</h1>
--<div>
--</div>
Type your answers in the group chat 🕐
❌ Bad catch. Fragile test
that annoys developers and
discourages refactoring
<div>
<h1>
Find hotels near Banff National Park, Alberta
</h1>
</div>
<h1>
Find hotels near Banff National Park, Alberta
</h1>
--<div>
--</div>
please don't use snapshot tests they're 99% chaotic bad and will make your devs cry more than usual
please don't use snapshot tests they're 99% chaotic bad and will make your devs cry more than usual
So for Expedia...
Our Users
Our Users
All customers
💁♿
Want to enter dates and search for properties
Our Users
Want to enter dates and search for properties
Data Analysts
📊
Want to track events of the customer through the funnel
All customers
💁♿
Our Users
🤖
Bots (SEO)
Want to enter dates and search for properties
Wants good metadata, performance and accessibility
Data Analysts
📊
Want to track events of the customer through the funnel
All customers
💁♿
Visual
modified testing pyramid by Kent C. Dodds & Martin Fowler
🐇
🐢
cross-device
💵
💰💰💰
Things we should avoid doing to test like the user
- Using class-based selectors to find elements
- Using DOM structure to find elements
- Manually triggering the re-rendering of components
- Using fixed-duration waits when expecting state changes
- Simulating browser events
// Instead of using
const usernameInput = container.querySelectorAll('input')[0];
const passwordInput = wrapper.find('input').last();
// ...or using
const usernameInput = wrapper.find('.sign-in-modal .username-field');
const passwordInput = container.querySelector('.sign-in-modal .password-field');
Using class-based selectors/DOM structure
Using class-based selectors/DOM structure
// Instead of using
// const usernameInput = container.querySelectorAll('input')[0];
// const passwordInput = wrapper.find('input').last();
// ...or using
// const usernameInput = wrapper.find('.sign-in-modal .username-field');
// const passwordInput = container.querySelector('.sign-in-modal .password-field');
// We should consider
const usernameInput = getByLabelText('Username');
const passwordInput = getByLabelText('Password');But what if you want
-
to select a specific element in a list?
But what if you want
-
to check that an element is not in the document?
// A shortcut for document.querySelector(`[data-testid="${yourId}"]`)
// <input data-testid="username-input" />
const usernameInputElement = getByTestId(container, 'username-input')getByTestId()
Should only be used in situations where all other selectors don't work for your use case.
// In a situation where you'd want to select a specific item in a list
const thirdLiInUl = container.querySelector('ul > li:nth-child(3)')
// You can include an index or ID in your attribute
const items = [
/* your items */
]
const thirdItem = getByTestId(`item-${items[2].id}`)Integration tests *
- Jest: testing platform library
- testing-library/react: light wrapper around ReactDOM that promotes good testing practices and one guiding principle
- jest-dom/extend-expect, custom jest matchers to test the state of the DOM
All customers
💁♿
Data Analysts
📊
All customers
💁♿
Data Analysts
📊
Checking the inputs and images are in the DOM
expect(queryByLabelText('Check-in')).tobeIntheDocument();
expect(queryByAltText('Frog screaming')).toBeInTheDocument();Integration tests examples
Integration tests examples
All customers
💁♿
Data Analysts
📊
Checking the inputs and images are in the DOM
Checking the track event sent on click
expect(queryByLabelText('Check-in')).tobeIntheDocument();
expect(queryByAltText('Frog screaming')).toBeInTheDocument();const searchButton = getByText('Search');
search.click();
expect(trackEvent).toHaveBeenCalledWith(...);
All customers
💁♿
Data Analysts
📊
Checking the inputs and images are in the DOM
expect(queryByLabelText('Check-in')).tobeIntheDocument();
expect(queryByAltText('Frog screaming')).toBeInTheDocument();const searchButton = getByText('Search');
search.click();
expect(trackEvent).toHaveBeenCalledWith(...);
or Checking error states from bad customer input and bad API responses
Checking the track event sent on click
Integration tests examples
End to End tests
- Cypress for end to end testing
- cypress-testing-library: additional cypress commands to match our jest ones
All customers
💁♿
Actually building and running the app locally or in CI and interacting with the page the way a user would
End to End tests Examples
All customers
💁♿
// Enter check in and out dates
cy.findByLabelText('Check-in').type('June 23, 2020');
cy.findByLabelText('Check-out').type('June 24, 2020');
// Submit dates
cy.findByText('Send').click();
// Check it sends to the correct url with query params
cy.location().should(loc => {
expect(loc.pathname).to.eq('/Hotel-Search');
expect(loc.search).to.eq('?startDate=2020-06-23&endDate=2020-06-23')
})
Visual Regression
All customers
💁♿
Most smaller applications don't need this
Useful when you have external UI libraries. Beware of CI font differences.
Visual Regression
All customers
💁♿
- Cypress for taking screenshots
- cypress-screenshot-diff for matching screenshots and running visual diffs
cy.matchScreenshot();Other checks
- Lighthouse for web audits
- size-limit for bundle size
🤖
Bots (SEO)
All customers
💁♿
Warning: Cross Browser/Device
All customers
💁♿
- Most our code isn't written by us anymore (React is tested cross-browser and handles events for you)
- Polyfills make sure code is compatible cross-browser
- Static tooling to ensure we're not using APIs that are missing polyfills (eslint-plugin-compat)
- We use browserstack/selenium as smoke tests, but it should not be the bulk as that is too painful to TDD.
That being said, our applications are large enough that we need it for a few cases
All customers
💁♿
User Centric Testing Pipeline
By Tiffany Le-Nguyen
User Centric Testing Pipeline
The present-day web: interactions are complex, accessibility's a priority, performance is essential and UI updates keep us on our toes. How can we have confidence that changes won't cause unwanted side effects, while minimally burdening devs? Let's walk through a modern end-to-end testing suite, which puts users first, for a maintainable and scalable codebase. ExpediaGroup has hundreds of devs and millions of users. Changes have to be airtight, easily refactorable, and consistent to users. Over the year, Tiffany built a user-centric testing suite that is now the standard at Expedia. She'll be sharing how modern web apps can define a self-aligning testing methodology, from unit to visual regression that will adapt to change. Attendees will be given an overview of how to identify their users and test for their individual needs while balancing cost and confidence in their systems.
