JavaScript End to End Testing Framework 

Comparing Nightwatch, Selenium, Puppeteer and WebDriverIO + giving an overview of Cypress 🌸

WebDriver Architecture

• It's unable to observe the application in real time.
• So you can't perform based on network operations. You can't perform based on existing state.
• You're assuming that everything is clean slate every time, that's why it's flaky. You always have to add fixed waits.

Cypress Architecture

Node Server

• Cypress can respond to events in real time
• Cypress observes web traffic in real time
• You can listen to and debug uncaught errors in the app code
• Because Cypress is installed locally on your machine, it can additionally tap into the operating system itself for automation tasks.

Selenium, Nightwatch & WebdriverIO Architecture

Nightwatch uses Selenium under the hood, so it's the same architecture

WebdriverIO is a Selenium WebDriver for Node, so it's the same architecture

Selenium uses the WebDriver interface, so it's the same architecture

Puppeteer

• Automation Framework only (no ui or test runner)

• Needs Test Runner on top (jest)
• Have to await, close each instance, etc.
• Only chromium desktop

Recap

• WebDriver: a spec for interacting with the browser
• Puppeteer: Chrome API for driving the browser (Uses DevTools Protocol)
• WebDriverIO: WebDriver API for driving the browser (Uses Selenium Webdriver)
• Nightwatch: Full framework (Uses Selenium Webdriver)
• Selenium: Abstraction over various WebDriver clients (ChromeDriver, GeckoDriver, etc.)

Bootstrapping a new project

Setup with Selenium

Install selenium server or framework

Install browser driver

Install test runner

Install assertion library

Install mocking/stubbing/spying library

Setup with Cypress

Testing with Cypress

Debugging with Cypress

Continuous Integration

Continuous Integration with Cypress

Pros

Cons

All-inclusive testing system

Powerful time-travel debugging

Automatic waits

Built-in function and network mocking

Testing directly on the browser without the need for WebDriver

Allows for parallelization of test suites

No support for cross-browser testing yet (WIP, FireFox in beta)

Browser support updates almost immediately

Cross-browser testing

• Most our code isn't written by us anymore (React is tested  cross-browser)
• Polyfills make sure code is compatible cross-browser
• Static tooling to ensure we're not using APIs that are missing polyfills
• eslint-plugin-compat (far from perfect but makes sure it's compatible)
• We can have browserstack/selenium as smoke tests, but it should not be the bulk as that is too painful and so is TDD.