Selenium at the Speed of Light: Turbocharging Your Test Execution

About Me

Developer Experience - Test Frameworks
Crowdstrike

What is Open MCT?

github.com/nasa/openmct

Red Bull Formula One

In this talk

  • Why Network Matters
  • What You Can Do About It
  • Future Exploration
  • Q&A

Let's set the stage

Tests

UI App

CI / Jenkins

Test Runner

Frontend

HTTP Rest API

Chrome

Box of Test Concerns

Box of App Concerns

vertically until we exhaust CPU

Test Runner

Test1
Test2
Test3

UI App

Let's break it down...

Let's Parallelize the tests!
(Horizontally)

Test Runner

Test 2

Test Runner

Test Runner

Test 1

Test Runner

Test 3

UI App

🏊 Pool Party! 🏊

Test Runner

UI App

Test Runner

Test Runner

Test Runner
Pool

Test Runner

Test Runner

Test Runner

BrowserPool

Latency 

  • turbocharge-selenium
  • Webdriver.io tests configured in bidi and webdriver
  • Chaos Engineering Tool toxiproxy to inject latency between containers
  • Open MCT in Docker
git clone
docker compose up -d
npm run toxi:setup
npm run test:*

github.com/unlikelyzero/turbocharge-selenium

Latency 🫠

🙏 Live 🙏 Demo 🙏

Results w/ 500ms

Chaos Test Runtime Runtime %

A

B

C

chromedriver

A (Selenium)

B (webdriver)

C (App Gap)

B (bidi)

4s

7s

21s

15s

8s

100%

175%

425%

275%

200%

⌛

⌛

⌛

Summary 

  • To scale, we need to add network 
  • Network is really bad for UI testing performance 
  • (Not shown) Timeouts start to cause flakiness
  • Red Hat Example. 27 -> 9 hours. Flakiness !

What can we do about it?

  • Architecturally
  • Write tests differently 
  • Question everything?

Colocate!

Test Runner

UI App

Test Runner

Test Runner

Test Runner

Test 1

Test 2

Test 3

Write tests differently 

  • 1. API Interaction
  • 2. Direct navigation 
import axios from 'axios';

describe('Create via API and Navigate', () => {
  it('should create item and navigate to its page', async () => {
    const res = await axios.post('https://api.example.com/items', {
      name: 'Test Item'
    });
    const uuid = res.data.id;
    await browser.url(`https://example.com/items/${uuid}`);
    await expect(browser).toHaveUrlContaining(`/items/${uuid}`);
    //
    //Continue on with the rest of your test
    //
  });
});

Are you testing the UI or through it?

REST API calls are 1000 to 1

Runtime under 5 minutes and Culture

Future!

  • Bidi
  • CPU Throttling
  • Backend (Current Dragons)

Summary

  • Why Network Matters
  • What You Can Do About It
  • Future Exploration

Thanks!
 

Contact and Links

openmct

turbo-selenium

CUTTING ROOM FLOOR

Testing Pyramid

  • Slow
  • Flaky
  • Expensive

Title Text

Made with Slides.com