Selenium at the Speed of Light: Turbocharging Your Test Execution
About Me
Developer Experience - Test Frameworks
Crowdstrike
What is Open MCT?
Red Bull Formula One
In this talk
- Why Network Matters
- What You Can Do About It
- Future Exploration
- Q&A
Let's set the stage
Tests
UI App
CI / Jenkins
Test Runner
Frontend
HTTP Rest API
Chrome
Box of Test Concerns
Box of App Concerns
vertically until we exhaust CPU
Test Runner
Test1
Test2
Test3
UI App
Let's break it down...
Let's Parallelize the tests!
(Horizontally)
Test Runner
Test 2
Test Runner
Test Runner
Test 1
Test Runner
Test 3
UI App
Test Runner
UI App
Test Runner
Test Runner
Test Runner
Pool
Test Runner
Test Runner
Test Runner
BrowserPool
Latency
- turbocharge-selenium
- Webdriver.io tests configured in bidi and webdriver
- Chaos Engineering Tool toxiproxy to inject latency between containers
- Open MCT in Docker
git clone
docker compose up -d
npm run toxi:setup
npm run test:*github.com/unlikelyzero/turbocharge-selenium
Latency 🫠
Results w/ 500ms
| Chaos | Test Runtime | Runtime % |
|---|---|---|
A
B
C
chromedriver
A (Selenium)
B (webdriver)
C (App Gap)
B (bidi)
4s
7s
21s
15s
8s
100%
175%
425%
275%
200%
Summary
- To scale, we need to add network
- Network is really bad for UI testing performance
- (Not shown) Timeouts start to cause flakiness
- Red Hat Example. 27 -> 9 hours. Flakiness !
What can we do about it?
- Architecturally
- Write tests differently
- Question everything?
Colocate!
Test Runner
UI App
Test Runner
Test Runner
Test Runner
Test 1
Test 2
Test 3
Write tests differently
- 1. API Interaction
- 2. Direct navigation
import axios from 'axios';
describe('Create via API and Navigate', () => {
it('should create item and navigate to its page', async () => {
const res = await axios.post('https://api.example.com/items', {
name: 'Test Item'
});
const uuid = res.data.id;
await browser.url(`https://example.com/items/${uuid}`);
await expect(browser).toHaveUrlContaining(`/items/${uuid}`);
//
//Continue on with the rest of your test
//
});
});Are you testing the UI or through it?
REST API calls are 1000 to 1
Runtime under 5 minutes and Culture
Future!
- Bidi
- CPU Throttling
- Backend (Current Dragons)
Summary
- Why Network Matters
- What You Can Do About It
- Future Exploration
Thanks!
Contact and Links
openmct
turbo-selenium
CUTTING ROOM FLOOR
Testing Pyramid
- Slow
- Flaky
- Expensive
Title Text
Turbo-charging Selenium
By John Hill
