I don't work for:

• Zalando
• Aerokube
• Browserstack
• Saucelabs
• Testingbot


 

Browser

Wrap the browser in a Containerized Selenium image

Selenium Docker Images

seleniumhq/Docker-selenium  
2973 Github Stars, 10M+ DockerHub Pulls
 

elgalu/docker-selenium
1100 Github Stars, 10M+ DockerHub Pulls
 

aerokube/selenoid
​913 Github Stars, 3M Docker Pulls

What's inside?

Dependency Heck

• House of Cards
• Hot issues
  • Se can't run on Java9
• Issues uncovered building demo
  • selenium/issues/5674
  • elgalu/docker-selenium/issues/201

"It works on my machine"

Parallelism

Demo

• Serial
• Parallel
• Remote Parallel
• Remote Parallel++

CPU & RAM

• ~ 1 CPU/Thread per session
• ~ 1000m in k8s/Openshift
• ~1 GB of RAM per session
• (More if recording video)

Network

• Selenium traffic is http traffic
• Latency between each component directly affects the speed of a test
• Each Test generates between 1-5 Requests Per Second (RPS)
• Up to 20 RPS

Anatomy of a Selenium Test in CI

Test Sharding

Scaling Path A

Cons

• Constrained by resource size of test executor
• Contesting other CI resources

Pros

• No network latency
• Local = CI

External Selenium Cluster

Scaling Path B

Cons

• Network latency

Pros

• Local = CI. Literally.
• Run locally "at scale"
• Separated concern
• Independent resources
• Many teams and frameworks can share resources

What does it take to create a cluster of selenium nodes?

Who has maintained a Selenium Grid before?

Problems with Se Grid

• Single Point of Failure
• Not Highly Available
• No Load Balancing Strategy
• Hub must be configured, installed and hosted
• No "Autodiscovery" of Se Nodes
• Parasitic Network Traffic
• Dependency Heck++
• Lack of Observability
• Not Fault Tolerant

Zalenium "Creates" Docker-Selenium Nodes

Pros

• All-in-one solution
• "Elastic" Nodes
• Works as a generic Hub with any Se Node
• Proxies to SaaS
• Native k8s/Openshift support
• Pay-as-you-go
• Observability
• Scales to 150 Nodes

Cons

• Not Highly Available
• Not Fault Tolerant
• Java-based
• Scales to 150 Nodes

Zalenium "Proxies" to Se-as-a-Service

GGR

Pros

• Intelligent Load balancing
• Works with any Selenium Hubs
• Works with SaaS
• Enables Highly Available clusters

Cons

• Not Fault Tolerant
• Not using standard selenium code
• Multiple GGR Nodes and a LB needed for HA
• Issues with 80% utilization

GoGridRouter

Pros

• Helm Chart config
• Perfect Load distribution
• Elastic!
• Pay-as-you-go
• 1000 Sessions with single cluster

Cons

• Closed Source
• Not FOSS
• Not Fault Tolerant

Saas

• Generic SaaS Pros apply
• Browserstack
• Saucelabs
• Testingbot

Cons

• Network Latency
• Issues at scale
• Costs

Costs

• Saucelabs
  • $75-$200 / session / month
  • ~$140,000 / month for 1000 Sessions
• Moon
  • $5 / session / month
  • k8s + $5,000 /month for 1000 Sessions
• Zalenium
  • $0.00017 to 0.00118 / minute / session
  • $36 + $34.67 / session / month

When?

• Christmas 2018
• Chinese New Year 2019
• SeleniumHQ/selenium/projects/2
• (Opinion) Mired by backwards compatibility
• All new Grid Features may not land at launch

Grid 2.0

To complicate matters, the world has moved on since we wrote the original grid implementations. When we started writing grid, machines were underpowered, had limited memory, and SSDs didn’t really exist. Virtual machines were provided by VMWare and getting any real density of these required absolutely huge servers that few could afford. Consequently, we used multiple machines, self-hosted in racks rooms and local data centres. This lead to a really simple design for grid, which has allowed it to continue functioning relatively well to this day." - Simon Stewart

Grid 4.0

• Rewrite
• 7 Components
  • Edge Router + Distributor = GGR
  • Session Storage = Fault Tolerance via Redis
  • New Session Queuing System = AWS Lambda

Should we wait?