Effective E2E Testing with Cypress

Dr Gleb Bahmutov PhD

VP of Eng Cypress.io

🌲🌲🌲🌲🌲🌲🌲🌲🌲🌲🌲🌲

🌲🌲🌲🌲🌲🌲🌲🌲🌲🌲🌲🌲

🌲🌲🌲🌲🌲🌲🌲🌲🌲🌲🌲🌲

🌲🌲🌲🌲🌲🌲🌲🌲🌲🌲🌲🌲

🌲🌲🌲🌲🌲🌲🌲🌲🌲🌲🌲🌲

🌲🌲🌲🌲🌲🌲🌲🌲🌲🌲🌲🌲

C / C++ / C# / Java / CoffeeScript / JavaScript / Node / Angular / Vue / Cycle.js / functional

EveryScape

virtual tours

MathWorks

MatLab on the web

Kensho

finance dashboards

66 blog posts on testing

Testing sucks

It is too hard

It takes too long

It is not that useful

Testing is

a drag

Testing Pyramid

We write more unit tests because our tools make it EASY

$ npm install -D jest
const add = require('./add')
describe('addition', () => {
  it('adds numbers', () => {
    expect(add(1, 2)).toBe(3)
  })
})
$ npm test

Great at testing

windshield wipers

Not so good at checking if a car can drive to Maine.

In winter

How test end to end

  1. During a full moon sacrifice two goats

  2. Install Selenium

  3. Quickly write E2E tests and throw them into Selenium before it gets angry

E2E should be easy

  • Easy to install

  • Easy to write tests

  • Simple to run

Easy to debug when a test fails

Web testing nirvana with Cypress

May 2, 2016

VP of Eng Cypress.io

May 8, 2017

Testing, the way it should be

$ npm install -D cypress

Tip: cache "node_modules" folder on CI

$ $(npm bin)/cypress init
# or
$ $(npm bin)/cypress open

Let's test "TodoMVC"!

$ rm cypress/integration/example_spec.js
$ rm cypress/integration/example_spec.js
$ touch cypress/integration/todo-spec.js
$ ls cypress
   integration/
     todo-spec.js
   fixtures/
     example.json
   support/
     commands.js
     index.js

test files (specs)

mock data / fixtures

extra code you want to load

it('opens todo app', () => {
  const url = 'http://todomvc.com/examples/angular2/'
  cy.visit(url)
})

Is the app focused on "New Todo" input?

this looks close

it('is focused on new item', () => {
  cy.visit(url)
  cy.focused()
})

that's result of "cy.focused()"

it('is focused on new item', () => {
  cy.visit(url)
  cy.focused().should('have.class', 'new-todo')
})

I know exactly what happens

it('is focused on new item', () => {
  cy.visit(url)
  cy.focused().should('have.class', 'new-todo')
})

Every command

outputs detailed info

in DevTools

Cypress.io

Selenium

Install, getting started

describe('New Todo', () => {
  const url = '...'
  beforeEach(() => cy.visit(url))

  it('loads a page', () => {
    cy.contains('h1', 'todos')
  })

  it('adds 1 todo', () => {
    // test body
  })
})

Test organization

Spec files group tests

  • add-todos-spec.js

  • delete-todos-spec.js

  • filtering-spec.js

$ cypress run cypress/integration/add-todos-spec.js

Making Assertions

Chai, Chai-jQuery, Sinon-Chai

Default Assertions

  • cy.visit() expects the page to send text/html content with a 200 status code
  • cy.get() expects the element to eventually exist in the DOM
  • .type() expects the element to eventually be in a typeable state

and many more ...

Cypress.io

Selenium

Assertions

Failing Test

The first time one of your

tests fails in Cypress

will be one of your best days

as a developer

const firstTodo = 'Learn Cypress'
it.only('adds 1 todo', () => {
  cy.get('#new-todo')
    .type(firstTodo)
    .type('{enter}')
  cy.get('.todo-list li')
    .first()
    .find('label').should('contain', firstTodo)
})

zero elements found

cy.get('.todo-list')
  .find('li')
  .first()
  .find('label').should('contain', firstTodo)

Get rid of compound selector

by splitting it into ".get(...).find(...)"

The ".todo-list" is missing

It should be "#todo-list"

Everything is fixed

cy.get('#todo-list')
  .find('li')
  .first()
  .type('foo')

Trying to type into a list item element

Cypress team

carefully hand crafts

each and every

error message

Giant API

80 commands + events + utilities (lodash, moment, jQuery) + assertions

Every command is described in excruciating detail

  • Syntax
  • Correct usage
  • Incorrect usage
  • Options
  • What it logs
  • Examples and best practices
cy.viewport('iphone-6')

Framework-specific Page Objects

import { browser, by, element } from 'protractor';

export class AppPage {
  navigateTo() {
    return browser.get('/');
  }

  getParagraphText() {
    return element(by.css('app-root h1')).getText();
  }
}

Cypress

const getApp = () =>
  cy.get('app-root')

const getHeader = () =>
  getApp().find('h1')

it('has welcome text', () => {
  getHeader()
    .contains('Welcome to app')
})

Cypress

// utils.js
export const getApp = () =>
  cy.get('app-root')

export const getHeader = () =>
  getApp().find('h1')
// spec.js
import {getHeader} from './utils'
it('has welcome text', () => {
  getHeader()
    .contains('Welcome to app')
})

Cypress also supports ES2015 out of the box. You can use either ES2015 modules or CommonJS modules. This means you can import or require both npm packages and local relative modules.

Test whatever you want

const url = 'http://todomvc.com/examples/vue/'
cy.visit(url)
{
  "baseUrl: "http://todomvc.com/examples/aurelia/"
}
cy.visit('/')
cypress.json
$ export CYPRESS_baseUrl=http://todomvc.com/examples/typescript-react/
$ $(npm bin)/cypress run

Cypress.io

Selenium

Dev Experience

I can't really give negative score with bar chart

*

Need for Speed

Cypress is VERY FAST

Selenium

var webdriverio = require('webdriverio');
var options = { desiredCapabilities: { browserName: 'chrome' } };
var client = webdriverio.remote(options);
client
    .init()
    .url('https://duckduckgo.com/')
    .setValue('#search_form_input_homepage', 'WebdriverIO')
    .click('#search_button_homepage')
    .getTitle().then(function(title) {
        console.log('Title is: ' + title);
        // outputs:
        // "Title is: WebdriverIO (Software) at DuckDuckGo"
    })
    .end();
Wait<WebDriver> wait = new FluentWait<WebDriver>(driver)
       .withTimeout(30, SECONDS)
       .pollingEvery(5, SECONDS)
       .ignoring(NoSuchElementException.class);

Timeouts, timeouts everywhere!

Cypress

Test continues as soon as DOM updates

const count$ = action$
  // delay events by 2 seconds
  .map(i => xs.fromPromise(by2seconds(i)))
  .flatten()
  .fold((x, y) => x + y, 0);

Usual E2E runners need delays or run slow in this case ... (Karma, Protractor, Selenium)

beforeEach(() => {
  cy.visit('index.html')
})
it('starts with 0', () => {
  cy.contains('Counter: 0')
})
it('increments counter 3 times', () => {
  cy.contains('Increment').click()
  cy.contains('Counter: 1')
  cy.contains('Increment').click()
  cy.contains('Counter: 2')
  cy.contains('Increment').click()
  cy.contains('Counter: 3')
})

Not Β single "delay" yet it runs in 6 seconds!

Cypress.io

Selenium

Speed

But wait, there is more ...

  • Add your commands

  • Server stubbing

  • Screenshots and videos

  • Chromium-based browsers

  • Easy CI setup

Is there is a catch?

  • Multi-browser support is coming

  • Windows support is coming

  • Plugin system is coming

Cypress is free

Cypress will be open source

Cypress has been in a private beta since 2015, and we are nearing our public release! What’s that mean, anyway?

  • Open Source All The Things!
  • The Desktop Application will no longer require logging in.
  • Anyone (everyone!) will be able to download and use the Desktop Application without contacting us first or signing up for anything.

Giving Cypress Away

Cypress Dashboard

See, share, and fix failing tests.

Dev CI
Staging
Prod

Single test run is a point

See the whole picture

Cypress Dashboard

  • Is a paid Saas complement to Cypress app
  • Perfect place to store test results, screenshots and videos
  • Helps find why and when a test started failing

Cypress Dashboard

  • New sweet features:
    • Test load balancing
    • Visual and performance regressions
    • UI coverage and test analytics

Thank you

$ npm install -D cypress

New!

Effective E2E Testing with Cypress

By Gleb Bahmutov

Effective E2E Testing with Cypress

The testing pyramid is a bad bad practice; it puts focus on unit tests, while your users really care about an end to end quality. For web developers, we are open sourcing a very powerful E2E tool called Cypress that is capable of giving your confidence that the app is going to work. Presented at AngularDevSummit 2017

  • 2,067
Loading comments...

More from Gleb Bahmutov