JavaScript End to End Testing Framework 

Comparing Nightwatch, Selenium, Puppeteer and WebDriverIO + giving an overview of Cypress 🌸

WebDriver Architecture

  • It's unable to observe the application in real time.
  • So you can't perform based on network operations. You can't perform based on existing state.
  • You're assuming that everything is clean slate every time, that's why it's flaky. You always have to add fixed waits.

Cypress Architecture

Node Server

  • Cypress can respond to events in real time
  • Cypress observes web traffic in real time
  • You can listen to and debug uncaught errors in the app code
  • Because Cypress is installed locally on your machine, it can additionally tap into the operating system itself for automation tasks.

Selenium, Nightwatch & WebdriverIO Architecture

Nightwatch uses Selenium under the hood, so it's the same architecture

WebdriverIO is a Selenium WebDriver for Node, so it's the same architecture

Selenium uses the WebDriver interface, so it's the same architecture

Puppeteer

  • Automation Framework only (no ui or test runner)

  • Needs Test Runner on top (jest)
  • Have to await, close each instance, etc.
  • Only chromium desktop

Recap

  • WebDriver: a spec for interacting with the browser
  • Puppeteer: Chrome API for driving the browser (Uses DevTools Protocol)
  • WebDriverIO: WebDriver API for driving the browser (Uses Selenium Webdriver)
  • Nightwatch: Full framework (Uses Selenium Webdriver)
  • Selenium: Abstraction over various WebDriver clients (ChromeDriver, GeckoDriver, etc.)

Bootstrapping a new project

Setup with Selenium

Install selenium server or framework

Install browser driver

Install test runner

Install assertion library

Install mocking/stubbing/spying library

Setup with Cypress

Testing with Cypress

Debugging with Cypress

Continuous Integration

Continuous Integration with Cypress

Pros

Cons

All-inclusive testing system

Powerful time-travel debugging

Automatic waits

Built-in function and network mocking

Testing directly on the browser without the need for WebDriver

Allows for parallelization of test suites

No support for cross-browser testing yet (WIP, FireFox in beta)

Browser support updates almost immediately

Cross-browser testing

  • Most our code isn't written by us anymore (React is tested  cross-browser)
  • Polyfills make sure code is compatible cross-browser
  • Static tooling to ensure we're not using APIs that are missing polyfills
  • eslint-plugin-compat (far from perfect but makes sure it's compatible)
  • We can have browserstack/selenium as smoke tests, but it should not be the bulk as that is too painful and so is TDD. 
Made with Slides.com