Who Am I?

Keywords: Front-end Engineer & Tester, Father, Husband, Disc Golfer, Stargazer, Gardener, Gamer, Geek, Soccer Player, Scale Modeller, Piano Player, Freelancer, Teacher, Writer, Designer, Developer, ​et cetera.​

What do I know??

Professional

• 7 years experience as a tester
• Multiple companies, various technologies
  • WebdriverIO, Playwright, Cypress, and more

Personal

• Wrote a book about it
• Created a course
  • https://learn.webdriver.io
• Made 100+ YouTube videos
  • youtube.com/@frontendtesting

C________

E________

R________

C________

E________

R________

M______

"You need the monitoring system to be substantially more reliable than the failure prone system or you don't gain reliability, just complexity."

- some reddit user

Rockets!

Tests must be more reliable than the code being tested

(that's surprisingly hard to do)

What tests promise to do

• Prevent costly bugs
• Accelerate software development

What tests Usually do

• Distract/impede developers
• Cause lots of false failures
• Get ignored
• Make for great hackathon presentations

How do you test that a car starts?

What does it take to run your test?

A fully working car!

• Gas
• Starter
• Spark plugs
• Belts
• Electronics/sensors
• Charged battery
• The right key!

How do you test a website works?

What does it take to run your test?

A fully working website!

• Browser/Device
• UI (HTML/CSS/JS)
• Server
• API
• Database
• Proper Data (search results for that word)
• Visual/Styles
• Time of day
• Third-party services/utilities
• Geographic region

C________

E________

R________

M______

C________

E________

R________

M______

Confidence

E________

R________

M______

What's the problem?

"Come, make UI testing work for us!"

• A sprint or two building out basic testing suite
• If lucky, integrated into CICD process
• Pieced together & built fast

AKA, the start of the End-to-End Death Spiral

A dark secret

"Why are our tests always failing?"

• Useful changes (improvements/tweaks)
• HTML structure/selector updates
• API changes
• Data changes
• Test flakiness issues (pauses and waits failing)
• Time changes
• Maybe bugs being caught?

Many layers = difficulty pinpointing change

Constant catch-up

"Getting behind the aircraft"

• Problems take a long time to figure out
• As you fix one problem, two new ones emerge
• As problems add up, you get further from the root cause, slowing down debugging even more
• Test usefulness plummets
• MAYDAY! MAYDAY!

A Testing failure

"Nice to have, but not for us"

Because testing was not properly invested in...

• All these failing tests create massive friction
• Friction creates massive frustration
• Frustration leads to ignoring results
• Ignoring results leads to out-of-date tests
• Out-of-date tests cause more failures
• Test suite crashes and burns

Confidence

E________

R________

M______

Confidence

E________

Reliability

M______

High Confidence, High Reliability

• Ad hoc servers isolated from unrelated changes
• Test fixture setup/tear down
  • API integration
• Dedicated resources building and maintaining test suite
• Fully integrated into CICD process
• "Shift-left testing"

Confidence

Easier

Reliability

M______

Reduce scope of testing

• Test fewer layers (mocking)
• Isolate what's being tested (e.g., Storybook)

Write UI Tests, not End-to-End tests

• Test the server on the server, not the browser

• Treat back-end as black box

• UI tests only check that user actions "work"

Optimize for readability & debugging

• Don't hide assertions in utility functions

• Use non-random/consistent data in tests

• Avoid generated tests

Write tests fast,
FIX Tests FAST

• Hard-code values in assertions, versus dynamic content
• Abstract as little as possible
• Don't DRY
• Use "snapshot" tools to capture expected state

Only test important things

• You absolutely do not have to write tests for everything
• Focus on value gained from the test

Only test easy things

• Some things are *really* hard to test (e.g., animations)
• Just don't test them!
• Is it worth the time?

Confidence

Easier

Reliability

Mocks/Components

An "Easy" Solution

GUI Based test tools

Confidence

Easier

Reliability

Mocks/Components

Confidence

Easier

Reliability

Mocks/Components

Confidence

Easier

Reliability

Mocks/Components

Mix and Match!

Some questions

What environment(s) should we test on?

Prod/Staging/Local

How do we handle test user accounts?

What are the best ways to handle authentication (e.g., Auth0)

How should we think about "resetting" the database/environment after a test is complete?

Is it better to write small single-task tests, or combine many actions in a big test?

What's your philosophy around testing only user-visible things, vs testing that user actions result in a change in the database?

How do we avoid captchas or bot detection - especially when our apps use third-party services

Should we use helper classes?

(e.g., Page objects)

When do you recommend that we mock services instead of testing the real thing?