Kevin C.
I love sharing my learning journey in the technology land and hopefully you can also takeaway some interesting tidbits from mine for your own learning and inspire many others to pass on these knowledges. Enjoy and happy learning & sharing!
UI Automation Testing with Headless Chrome
Kevin C Chen @ 2018 SDET Summit
Streamlines DevOps
Testing Workflow
Makes Testing an Enjoyable Exp
https://devopscube.com/what-is-devops-what-does-it-really-mean/
Test features work from the user’s perspective
via Web UI
Challenges
Complex Deployment & Setup
Flaky Vs. Robustness
Lengthy test runs
Cost of upkeeping / tuning with sleep(), wait()
image source: https://cdn.edureka.co/blog/wp-content/uploads/2017/05/Selenium-blog-1.gif
Goals
- Painless to setup, zero configuration, no server
-
No need to learn another proprietary tool that talks to browser
- Instrument browser directly with its native API
- Can run both in headless mode in CI/CD pipeline or interactive mode with browser
- Access to browser DevTools features
Headless Browser
=
Chrome without Chrome
What is Headless Chrome?
- A way to run the Chrome browser in a headless environment
- It brings all modern web platform features provided by Chromium and the Blink rendering engine to the command line
- A great tool for automated testing and server environments where a visible UI shell is not needed
TL;DR
Why Headless Chrome?
-
Stays up to date with the web
-
Runs fast without actual browser UI
- Test all modern features (ES6 / CSS / HTML)
streams, css grid, push notification, service worker...
- Programmatic access to DevTools
network throttling, device emulation, code coverage...
What can Headless Chrome do?
- Pre-rendering JS sites / server sider rendering static markup
- Validate lazy loading with Code Coverage
- A/B Testing e.g. Stylesheets Vs Inline critical styles
- Catch issues for the Google crawler to determine if your app might not render correctly in Google search
- Create custom PDF
- Crawl a Single Page Application
- Verify service worker offline caching
- Make your browser talk (headful)
- Test a Chrome extension (headful)
- Control keyboard event
Demo Test Run
What is Puppeteer?
- A Node Library open sourced by Google (Chrome DevTools team)
- Provides a high-level API to control headless (or full) Chrome or Chromium over the DevTools Protocol
-
Bundles with latest Chromium: Zero configuration
-
Canonical reference for the Chrome DevTools protocol
- Almost zero performance overhead over an automated page
https://www.youtube.com/embed/lhZOFUY1weo?enablejsapi=1
What can Puppeteer do?
- Most things that you can do manually in the browser!
- Built for Automation
- JS async/await for asynchronous things to avoid arbitrary wait times
- Works for both traditional JSP UI & modern React UI (API based)
https://pptr.dev/#?product=Puppeteer&version=v1.9.0&show=api-overview
How do we perform UI tests?
- Jest - JavaScript Test Framework by Facebook
- testing structure
- assertions functions
- mocks, spies
- display test result, code coverage reports
- Parallelizes test runs across test suites to maximize performance
https://www.youtube.com/embed/lhZOFUY1weo?enablejsapi=1
Test Walkthrough
CI/CD with Docker Container
- Self-contained image that can run on Jenkins CI pipeline
- Installation of Chromium can be part of npm dependencies or with a separate download within the container
- An on demand regression test run
- Run before PR merge with UI changes
Alternatives
- We had Selenium already, why bother?
- slow, brittle, and costly
- lot of configuration and unexpected behaviour
-
But Selenium sometimes makes more sense
- If our UI has a lot of device or browser specific code, we may still want Selenium
- If our UI has a lot of device or browser specific code, we may still want Selenium
- WebDriver and ChromeDriver
-
Cypress
- Trade-offs
Bigger Questions
- How do we test our UI today?
- What is our automation coverage for UI test?
- A step back: How many UI to test?
- What is our test strategy to ensure we deliver a good user experience?
Testing from the user’s perspective
- Time to interactive (TTI) - how long it takes for app to load and become capable of quickly responding to user interactions
User Perceived Performance?
https://developers.google.com/web/fundamentals/performance/rail
User having to wait for the browser to respond to an event. Amount of pain is directly proportional to the time user waits
-
First Input Delay (FID) - measures the delay that users experience when they interact with the page while it's not yet interactive
User Perceived Pain?
https://developers.google.com/web/updates/2018/05/first-input-delay
RAIL - User-centric Performance Model
https://developers.google.com/web/fundamentals/performance/rail
produce a frame < 10ms
process events
< 50ms
maximize
Time To Interactive
< 5s
How to Measure: Lighthouse
- Headless Browser
- How to automate with Puppeteer
- How to build UI automation test suite with Jest
- CI/CD integration with Docker container
- Alternative approaches
- UI testing strategy and considerations
What We've Learnt
References
- Puppeteer
- Jest
- Diagrams based on this repo
UI Automation Testing with Headless Chrome Kevin C Chen @ 2018 SDET Summit
UI Test Automation with Headless Chrome (Puppeteer + Jest + Docker)
By Kevin C.
UI Test Automation with Headless Chrome (Puppeteer + Jest + Docker)
This presentation demonstrates how we could automate many end-to-end UI tests with Headless Chrome via Puppeteer (Node API). Furthermore, to integrate with the CI pipeline, we can make a docker container that executes the tests.
- 1,615
