Mastering UI Testing

Abstract: have you ever wrote a successful UI test? How many times did you end up discarding a UI test because it's brittle and unmaintainable? And even if you don't write tests, tell me: do you click/type/click/click/setcookie/click/click while developing your UI?

Β 

I'm going to explain you the most important best practices that allow you to master the UI tests hell, sleep at night (because you are sure your project works as expected), test the back-end responses too and completely revolutionize the way you develop your UI!

Β 

I'm going to talk about - UI testing best practices - live demos with Cypress - improve your workflow using a UI test framework as your everyday working tool (so it's not only about testing)

Mastering UI Testing

I'm Stefano Magni (@NoriSte)

I'm a passionate front-end developer, a testing lover, and an instructor.

Β 

I work for Conio, a bitcoin startup based in Milan.

Β 

My recent OS contributions: github.com/NoriSte/all-my-contributions

(or just run $ npx noriste)

Slides info

Don't take notes or make photos... These slides are publicly available πŸŽ‰
Β 

slides.com/noriste/voxxed-days-ticino-2019-mastering-ui-testing

or

bit.ly/2mcwNai

Slides info

There are a lot of linksΒ into the slides πŸ”—

Almost all the logos are links too πŸ”—

Slides are organized both horizontally and vertically (see the bottom-right arrows).

Best practices are highlighted

My two UI testing principles:

β€’ tests must not fail if the app is working (false negatives)

Β 

β€’ if they fail, they must drive me directly to problem

Why is E2E testing hard?

Let's start:

Why is E2E testing hard?

Because we need to test four kind of contracts at the same time!

β€’ HTML - πŸ“„ semantic markup
β€’ CSSΒ  - πŸ‘ visual regressions
β€’ JSΒ  Β - πŸ•Ή interaction flows
β€’ APIΒ  - πŸ” client/server communication

Β 

Why is E2E testing hard?

Because it needs a real browser and a real context!

β€’ βŒ›οΈ everything is asynchronous
β€’ πŸ“¦ E2E tests need a lot of reliable data
β€’ 🐞 debugging E2E tests is hard
β€’ 😭 the browser is slow, the UI is slow, the network is slow

Β 

Why is E2E testing hard?

Because generic browser automation tools s**k!

β€’ ⌨️ no UI
β€’ πŸ”’ no step-by-step test utilities
β€’ πŸš€ no integrated utilities
β€’ πŸ€” no clear reasons for failures

Β 

How could we make E2E testing easier?

First of all:

😱😱😱

Full E2E tests are not

so important!

UI integration tests (with a stubbed server) are way much more reliable!

Suffers from Full E2E test UI integration test
the network Yes πŸ˜“ No πŸŽ‰
the server Yes πŸ˜“ No πŸŽ‰
data resetting Yes πŸ˜“ No πŸŽ‰
complex case data Yes πŸ˜“ No πŸŽ‰

Full E2E tests are not so important!

Full E2E test UI integration test
confidence 100% πŸ’ͺ 90% πŸ€·β€β™‚οΈ
performance* ~20"/test πŸ˜“ ~2.5"/test πŸš€

* benchmarking the Conio's back office test suite (env: staging)

UI integration tests (with a stubbed server) are way much faster!

Full E2E tests are not so important!

⬆️ E2E tests vs UI Integration tests ⬇️

Write a few E2E tests and a lot of UI integration tests

πŸ€™

// Cypress example
cy.server();
cy.route({
  url: "/auth/token",
  method: "POST",
  response: "fixture:authentication-success.json"
})

cy.visit("/login")
cy.get(".username").type("stefano@conio.com")
cy.get(".password").type("mysupersecretpassword")
cy.get(".btn-primary").click()

// the front-end app is going to receive the content
// of the authentication-success.json fixture

Stubbing the back-end is easy with Cypress.

Here an authentication form example.

cy.route({
  url: "/auth/token",
  method: "POST",
  response: "fixture:unexisting-user.json"
})

cy.route({
  url: "/auth/token",
  method: "POST",
  response: "fixture:already-registered-user.json"
})

Stubbing the back-end allows us to check every user flow with ease.

cy.route({
  url: "/auth/token",
  method: "POST",
  response: {},
  status: 401
})

cy.route({
  url: "/auth/token",
  method: "POST",
  response: {},
  status: 500
})

And write an E2E test just for the happy paths.

Β 

At the moment, the Conio's back office test suite is composed by 110 UI integration tests and 15 E2E tests

πŸ€”

What about the

client/server contract?

Test every request and response payload

πŸ”

How many times the front-end app doesn't work because of bad/misaligned payloads?

cy.server();
cy.route({
  url: "/auth/token",
  method: "POST",
  response: "fixture:authentication-success.json"
}).as("login-request")

cy.visit("/login")
cy.get(".username").type("stefano@conio.com")
cy.get(".password").type("mysupersecretpassword")
cy.get(".btn-primary").click()

cy.wait("@login-request").then(xhr => {
  expect(xhr.request.body).to.have.property("username", "stefano@conio.com")
  expect(xhr.request.body).to.have.property("password", "mysupersecretpassword")
})

Check the request payloads with the UI integration tests.

cy.server();
cy.route({
  url: "/auth/token",
  method: "POST"
}).as("login-request")

cy.visit("/login")
cy.get(".username").type("stefano@conio.com")
cy.get(".password").type("mysupersecretpassword")
cy.get(".btn-primary").click()

cy.wait("@login-request").then(xhr => {
  expect(xhr.status).to.equal(200)
  expect(xhr.response.body).to.have.property("access_token")
  expect(xhr.response.body).to.have.property("refresh_token")
})

Check the response payloads with the E2E tests.

Export/check every back-end schema/elasticsearch mapping/typescript definitions.

Β 

You can use Postman, Pact, GIT, my NPM package url-content-changes-checker, Jest snapshot testing.

My url-content-changes-checkerΒ in action

It highlights the changes in a text document

Do not slow down your tests with unnecessary sleeping

πŸ˜΄βŒ›οΈ

Always wait for some determinist events, never make your tests sleeping.

How can I define the deterministic events?

Some examples are:

β€’ every XHR requests

β€’ UI element appearings

β€’ everything valuable for the business

And how can I avoid test sleepings?

Cypress automatically waits!

// Cypress waits up to 60 seconds for a page load
cy.visit("/login")

// up to 4 seconds for an element appearing
cy.get(".username").type("stefano@conio.com")

// up to 5 seconds for an XHR request and
// up to 30 seconds for the XHR response
cy.wait("@login-request")

// and if cypress cannot wait for something,
// you can use my own plugin
cy.waitUntil(() => cy.getCookie("token")
  .then(cookie => Boolean(cookie && cookie.value)))

Cypress automatically waits!

Expose an authentication shortcut from the front-end

πŸŽπŸ’¨

All the tests need authentication, do not authenticate through the UI all the times.

// front-end
if(window.Cypress) {
  window.conioApp.store = reduxStore
}

// Cypress
window.conioApp.store.dispatch({
  type: "LOGIN_REQUEST",
  "stafeno@conio.com",
  "mysupersecretpassword"
})

Consuming some shortcuts (app actions) instead of the UI:

β€’ is blazing fast
β€’ avoids you to duplicate the authentication management

⬆️ through UI vs through shortcuts ⬇️

πŸ’ͺ

Expose some constants from the front-end

A lot of times some tests fail for little to insignificant

front-end changes.

// instead of...
cy.visit("/login")

// you can
import {LOGIN_PATH} from "@/navigation/paths";
cy.visit(LOGIN_PATH)

// and if "/login" becomes "/sign-in" you will not
// face an annoying test failure

We all hate false negative test failures. Work to reduce them as much as possible.

πŸ™‹β€β™‚οΈπŸ™‹β€β™€οΈ

Assert frequently

The more you assert, the more the failures drive you to the exact problem.

// Cypress example
cy.server();
cy.route("POST", "/auth/token").as("login-request");

cy.visit(LOGIN_PATH)
cy.get(".username").type("stefano@conio.com")
cy.get(".password").type("mysupersecretpassword")
cy.get(".btn-primary").click()

cy.wait("@route_login").then(xhr => {
  expect(xhr.request.body).to.have.property("username", "stefano@conio.com")
  expect(xhr.request.body).to.have.property("password", "mysupersecretpassword")
  expect(xhr.status).to.equal(200)
  expect(xhr.response.body).to.have.property("access_token")
  expect(xhr.response.body).to.have.property("refresh_token")
});

cy.get("#success-feedback").should("exist")
cy.url().should("not.include", LOGIN_PATH)

🏎 πŸš™ 🚜

Split the different kind of tests

Different kind of tests have different purposes, usually you need to launch just some of them.

{
  "cy:test:integration": "cypress run --spec \"cypress/**/*.integration.*\"",
  "cy:test:e2e": "cypress run --spec \"cypress/**/*.e2e.*\""
}

My three types of Cypress tests

Type Filename When
UI integration tests <name>.integration.test.js β€’ development
β€’ CI pipeline
e2e tests <name>.e2e.test.js β€’ post CI pipeline
β€’ cron
monitoring tests <name>.monitoring.test.js β€’ cron

πŸ‘

Test the front-end the same way the user consumes it

The user does not care about selectors, he cares about contents.

// instead of testing the front-end through selectors...
cy.get(".username").type("stefano@conio.com")
cy.get(".password").type("mysupersecretpassword")
cy.get(".btn-primary").click()
cy.get("#success-feedback").should("exist")

// ... we can test it though contents, like the user does
import {
  USERNAME,
  PASSWORD,
  LOGIN,
  SUCCESS_FEEDBACK
} from "@/components/LoginForm/strings"

cy.findByText(USERNAME).type("stefano@conio.com")
cy.findByText(PASSWORD).type("mysupersecretpassword")
cy.findByText(LOGIN).click()
cy.findByText(SUCCESS_FEEDBACK).should("be.visible")

It's made easy by some plugins like cypress-testing-library or cypress-xpath.

More: if your tests consume contents (and not CSS selectors, IDs etc.)...

... You just need to take a look at a screenshot to understand why a test failed!

If you can not do that, use ARIA or data-testid attributes.

❀️

Are you a TDD lover?

Cypress is perfect to be used as your main development tool and browser.

You can write your acceptance test and apply an outside-in approach to your front-end app.

Cypress can be launched:

# headlessly
$ cypress run

# with its UI
$ cypress open

Cypress can be launched with your Chrome version of choice:

Cypress creates a dedicated, persistent, Chrome user. You can install your DevTools of choice:

The Test Runner is amazing:

The Test Runner allows time-travelling:

Error reporting is amazing:

skip-and-only-uiΒ and cypress-watch-and-reloadΒ plugins are useful too:

Cypress Dashboard is perfect for CI pipelines:

β€’ Tests are re-launched on every CTRL+S

β€’ if a test fails, Cypress automatically save screenshots and videos

β€’ clock (setTimeout) control
β€’ code coverage from E2E tests unified with the standard one

... And more:

πŸŽ‰

So you do not have to test your front-end manually anymore!

Some common questions:

😊

Does Cypress supports only Chrome?

They are working on cross-browser support but yes, only Chrome is supported until now!

Uhm πŸ€”

Cross-browser tests are more useful for visual regression tests, integrate AppliTools with Cypress and you get them πŸ˜‰

Is Cypress free?

Yes, you pay only if you want to have it upload a lot of test videos on its servers. πŸ˜‰

Is it all about front-end testing?

πŸ€”

Cypress solves the biggest testing problems but: no, there are other kind of tests.

β€’ JS unit testing
β€’ Component testing
Β  Β β€’ snapshot testing (HTML)
Β  Β β€’ visual regression testing (CSS)
Β  Β β€’ API testing
β€’ App visual testing (CSS)

For component development and tests: take a look at Storybook and its storyshots addon.

Just after the ReactJSDay course, we are preparing a two-day workshop.

I and Jaga SantagostinoΒ are preparing a two-day workshop about everything related to JavaScript testing.

Β 

The first one will be about React testing.

Β 

Drop us a line if you want to know more about it πŸ—£

I'm working on a "UI Testing Best Practices" book on GitHub.

I've just begun writing it, all contributions are welcome πŸ™Œ

Recommended courses

Recommended sources

Thank you!

Made with Slides.com