Keywords: Front-end Engineer & Tester, Father, Husband, Disc Golfer, Stargazer, Gardener, Gamer, Geek, Soccer Player, Scale Modeller, Piano Player, Freelancer, Teacher, Writer, Designer, Developer, et cetera.
Professional
Personal
C________
E________
R________
C________
E________
R________
S______
E___
M______
"You need the monitoring system to be substantially more reliable than the failure prone system or you don't gain reliability, just complexity."
- some reddit user
Tests must be more reliable than the code being tested
(that's surprisingly hard to do)
How do you test that a car starts?
What does it take to run your test?
A fully working car!
How do you test a website works?
What does it take to run your test?
A fully working website!
C________
E________
R________
S______
E________
M______
C________
E________
R________
S______
End-to-End
M______
Confidence
E________
R________
S______
End-to-End
M______
"Come, make UI testing work for us!"
AKA, the start of the End-to-End Death Spiral
"Why are our tests always failing?"
Many layers = difficulty pinpointing change
"Getting behind the aircraft"
"Nice to have, but not for us"
Because testing was not properly invested in...
Confidence
E________
R________
S______
End-to-End
M______
Confidence
E________
Reliability
S______
End-to-End
M______
High Confidence, High Reliability
Confidence
Easier
Reliability
S______
End-to-End
M______
Test the server on the server, not the browser
Treat back-end as black box
UI tests only check that user actions "work"
Don't hide assertions in utility functions
Use non-random/consistent data in tests
Avoid generated tests
Confidence
Easier
Reliability
Snapshots
End-to-End
Mocks/Components
GUI Based test tools
Confidence
Easier
Reliability
Snapshots
End-to-End
Mocks/Components
Confidence
Easier
Reliability
Snapshots
End-to-End
Mocks/Components
Confidence
Easier
Reliability
Snapshots
End-to-End
Mocks/Components
Mix and Match!
What environment(s) should we test on?
Prod/Staging/Local
How do we handle test user accounts?
What are the best ways to handle authentication (e.g., Auth0)
How should we think about "resetting" the database/environment after a test is complete?
Is it better to write small single-task tests, or combine many actions in a big test?
What's your philosophy around testing only user-visible things, vs testing that user actions result in a change in the database?
How do we avoid captchas or bot detection - especially when our apps use third-party services
Should we use helper classes?
(e.g., Page objects)
When do you recommend that we mock services instead of testing the real thing?
You won't have perfect tests, but you will have valuable tests