It's an initiative to spark collaboration on Selenium Testing at Red Hat

QE Shark Tank

Primary purpose was to save Red Hat QE $10k

Secondary purpose was "more about getting us all to think about working smarter and more efficiently."

Proposal

1. Architect a Shared Selenium Cluster to reduce our SaaS $$$

2. Provide Se base images for selenium testing that feed into cluster

3. Act as a CoE to combine our siloed efforts

SaaS Providers

• Lots of competitors
• Dashboard/Video Recording
• Huge browser/platform matrix

Walls of text (and bad math) incoming

Cost Savings Calculation

• SaaS providers charge $60-$200 per parallel session.
• Not always billed by usage (idle for more than 12 hours a day).
• $7k month for 50 sessions is $140/month/session.
• If a team is running nightly builds to generate 5 hours of tests to run 40 tests in parallel, they’re still paying $140 * 50 = $7000 / month
• In GKE, Zalenium is $36.23 + $0.000792 per minute / session.
• That same CI Bill would become (5 hours) * ($0.000792 * 60 min/hour) * ( 40 Sessions ) = $45.734 / month
• $6954.266 will be saved per month for a single team. 

CI Time = $$$

• The latency between Brno and DC is ~111ms.
• Requests Per Second (RPS) on average are 2.
• That means that a testsuite run from Brno against a selenium node in DC will experience (2.5 * 110ms) = 0.275s of additional latency per second.
• If a nightly testrun were to take 5 hours to run with a geolocated selenium cluster, that same testrun would take approximately (18000 * 1.275 = 22950 seconds) 6.38 hours.
• Now, because we're billing by the minute, CI Time = $$$
• We're effectively paying an additional 27.5% to run tests when using geo-distributed selenium nodes.

Satellite

• Regression Runtime went from 27 hours to 10

What changed

• Architect Distributed and Centralized Shared Selenium Cluster to replace SaaS $$$
• Grid Templates, not base container images
• Performance, performance, performance

Original Implementation

Comcast Test

• Local Zalenium Grid run in docker
• Ansible Tower QE Smoke tests run against local zalenium grid
• Comcast configured with specific network delay on bridge interface

Since the Project Started

• 13 Chromedriver changes
• jsonWire protocol deprecated in Chrome
• 6 Versions of Chrome
• Chromium-based Edge announced
• Selenium 4.0 alpha released

Current Implementation

Templates and Configs

• Gitlab/sharedselenium

• Docker-compose for Zalenium
• PSI Openshift Template

Demo

• Openstack / Openshift Zalenium
• SaaS Route out speed
• Video Recording
• Moon / Selenoid

In Order

1. Are you paying for a SaaS solution?
2. Running zalenium?
3. Running a grid?
4. Need to lower regression times