Testing workshop

Edd Yerburgh

@eddyerburgh

What about you?

Topics

  • Linting

  • Unit tests (Jest)

  • Snapshot / Visual regression tests (Jest)

  • Integration tests with Cypress

  • E2E tests with WebDriver

Outcome

  npm it
npm install && npm run test

Format

  1. Learn

  2. Question

  3. Apply

Resources

Slides

slides.com/eddyerburgh/testing-vue-workshop

 

Repo

github.com/eddyerburgh/testing-vue-workshop

Participation tips

Raise your hand for questions at any time!

Please no recording

Disclaimer

Front-end testing is an unsolved problem

Questions

Testing overview

Manual testing

Automated testing

  • Static analysis
  • Executed tests

 

Why?

  • Verify new features/bug fixes behave correctly

  • Catch regressions

  • Documentation
  • Enable workflows

 

Static analysis

Type checking

Linting

Formatting

Linting

Type checking

 

Executed tests

  • E2E tests

  • Integration tests

  • Snapshot / Visual regression tests

  • Unit tests

 

E2E tests

  • Automate a browser

  • Test entire system

 

Integration tests

  • Difficult to define!

  • Mock parts of the system

 

Snapshot tests/

(Visual regression tests)

 

Unit tests

  • Test smallest units of code
  • Run quickly

Questions?

 

Problems

  • Tests take time to write
  • UI testing is hard
  • Can lead to a false sense of security (bugs still slip through!)

 

Questions?

Linting

Topics

  • eslint

  • Prettier

eslint

  • Configurable

  • Good for catching unfixable errors

  • Don't use for formatting

prettier

  • Formats code

  • Stops trivial arguments in PRs

  • Add to a pre-commit hook

Linting

(No formatting rules)

"lint": "eslint src/**/*.{js,vue}"
yarn add --dev \
eslint \
eslint-plugin-vue \
eslint-config-prettier \
eslint-config-standard \
eslint-plugin-standard \
eslint-plugin-import \
eslint-plugin-node \
eslint-plugin-promise
{
  "root": true,
  "extends": [
    "plugin:vue/recommended",
    "standard",
    "prettier",
    "prettier/standard"
  ],
  "rules": {
    "no-new": 0
  }
}

Formatting

yarn add --dev prettier lint-staged husky@next
// package.json
{
  //..
  "husky": {
    "hooks": {
      "pre-commit": "lint-staged"
    }
  },
  "lint-staged": {
    "*.{js,json,css,md,vue}": [
      "npm run format",
      "git add"
    ]
  },
}
"format": "prettier --no-semi --single-quote --write '{,!(node_modules)/**/}*.{js,json,jsx,md,vue}'"

Questions?

Unit tests

Topics

  • Using Jest to write unit tests

  • Testing SFCs with Vue Test Utils

  • Using mocks

  • Testing asynchronous code

  • Mocking file dependencies

Unit test

const sum = require('sum')

if(sum(2, 2) !== 4) {
  throw new Error('sum did not return 2')
}

Assertion

Unit tests

  • Fast
  • Easy to debug
  • Not much confidence

Jest

import sum from './sum'

test('returns sum of input', () => {
  expect(sum(2,2)).toBe(4)
})
import sum from './sum'

test('returns sum of input', () => {
  if(sum(2, 2) !== 4) {
    throw new Error('sum did not return 2')
  }
})

Assertion error

Test report

Assertions matter!

expect(arr).toHaveLength(4)

expect(str).toContain('some str')

expect(fn).toThrow()

https://jestjs.io/docs/en/expect

Adding Jest

yarn add --dev jest
// package.json
{
  //..
  "scripts": {
    // ..
    "test:unit": "jest"
  }
}

Filters

<template>
  <div>{{ val | capitalize }}</div>
</template>

Questions?

Exercise 1

src/filters.spec.js

Unit tests must be deterministic

 

Non-deterministic functions cause problems in unit tests:

Math.random()

Date.now()

Mocking

Date.now = () => new Date('2018')

Replace non-deterministic functions with deterministic functions

Questions?

Exercise 2

src/filters.spec.js

Unit testing components

1. compile component

2. mount component

3. provide input

4. assert output

1. compile component

Modal.vue

Modal.spec.js

Compiler

Test runner

Vue runtime doesn't understand template strings

`<div><nav-bar /></div>`

Vue runtime

<template>
  <div></div>
</template>

<script>
  export default {
    props: ['visible']
  }
</script>
module.exports = {
  render: function () {
    var _vm=this;
    var _h=_vm.$createElement;
    var _c=_vm._self._c||_h;
    return _c('div')
  }
  staticRenderFns: [],
  props: ['visible']
}

compiled SFC

SFC

Vue runtime

Compiling with Jest

  • Compile Vue SFCs with vue-jest
  • Compile JavaScript with babel-jest

Note: for a full guide to setting up with vue-jest, check out the vue-jest README: https://github.com/vuejs/vue-jest

// package.json
{
  "jest": {
    "transform": {
      "^.+\\.js$": "babel-jest",
      "^.+\\.vue$": "vue-jest"
    }
  }
}

Avoid configuration with the Vue CLI

vue add @vue/cli-plugin-unit-jest

https://github.com/vuejs/vue-cli

https://github.com/vuejs/vue-cli/tree/dev/packages/%40vue/cli-plugin-unit-jest

2. mount component

 

const Constructor = Vue.extend(Component)
const vm = new Constructor().$mount()
vm.$el.textContent
import { mount } from '@vue/test-utils'

const wrapper = mount(Component)
wrapper.vm.$el.textContent
import { mount } from '@vue/test-utils'

const wrapper = mount(Component)
wrapper.text()

JSDOM

  • JavaScript implementation of DOM
  • Run tests in DOM environment in node
  • Does not support layout
  • Does not support navigation

shallowMount

List

ListItem

ListItem

Link

Button

mount(List)
shallowMount(List)

3. provide input

4. assert output

Input

Output

Component

  • User action
  • Props
  • Store
  • Rendered output
  • Vue events
  • Function calls

Component contract

  • What do users need to know
  • What is exposed to the outside world (the component's public API)
  • What would cause other components to break if it changed

Mounting options

mount(Component, {
  propsData: {
    prop: 'some value'
  },
  mocks: {
    $store: { state: { count: 1 } }
  }
})

https://vue-test-utils.vuejs.org/api/options.html

.find

wrapper.find('a').text()
  • Traverses rendered nodes in DOM tree order
  • Returns wrapper containing first matching node

.findAll

wrapper.findAll('a').at(0).text()
  • Traverses rendered nodes in DOM tree order
  • Returns wrapper array containing all matching nodes
  • access individual node with .at method

Selectors

wrapper.find('[data-test="button"]')

👌

wrapper.find('[aria-label="close"]')
wrapper.find('button')
wrapper.find('.close-modal')

🤷‍♂️

Arrange, act, assert

test('renders when clicked', () => {
  // Arrange
  const wrapper = mount(TestComponent)

  // Act
  wrapper.trigger('click')

  // Assert
  expect(wrapper.text()).toBe()
})

Questions?

Exercise 3

src/components/ProgressBar.spec.js

src/components/Question.spec.js

Testing components is difficult

  • Unanswered questions
  • What to test?!

Downside of unit testing components

  • High maintenance
  • Difficult to refactor (refactor fatique)

Benefits of unit testing components

  • Defined component API
  • Easy to debug tests

Dispatching events

const wrapper = shallowMount(Component)

wrapper.find('[aria-label="close modal"]')
  .trigger('click')
const wrapper = shallowMount(Component)

const event = new Event('click')
wrapper.find('[aria-label="close modal"]')
  .element.dispatchEvent(event)

https://vue-test-utils.vuejs.org/api/wrapper-array/trigger.html

Questions?

Exercise 4

src/components/SignInModal.spec.js

Mocks, mocking, and mock functions

  • Mocks are objects/ functions created to replace other objects
  • Mocking is the process of replacing existing functionality with mocks
  • Mock functions are jest functions that can be controlled, and used to test if functions were called

Using mocks

function logError(err) {
  const message =  err.message || 'There was an error'
  console.error(message)
}
const mockFn = function(...args) {
  mock.calls.push(args) 
}

mockFn.calls = []

console.error = mockFn

console.error('message')

mockFn.calls // [['message']]
const mockFn = jest.fn()

console.error = mockFn

console.error('message')

mockFn.mock.calls // [['message']]

https://jestjs.io/docs/en/mock-function-api#methods

const mockFn = jest.fn()

expect(mockFn).toHaveBeenCalled()
expect(mockFn).toHaveBeenCalledWith('value')
wrapper.find('input').element.value = 'hello'
wrapper.find('input').setValue('hello')

Complex mocks

const $store = {
  dispatch: jest.fn(),
  commit: jest.fn()
}

shallowMount(List, {
  mocks: { $store }
})
const player = {
  settings: {},
  addPlugin(plugin) {
    if (!this.settings.plugins) {
      this.settings.plugins = []
    }
    this.settings.plugins.push(plugin)
  }
}

Problems

  • Mocks make tests more brittle
  • Mocks can be buggy
  • Mock are extra overhead

 

When to use?

  • When mocks are simple
  • When unit contract is to call function

Mocking modules

// db.js

export function deleteQuestion() {
  fetch('/questions', { method: 'DELETE' })
  // ..
}
// utils.js

import { deleteQuestion } from './db'

export function promptDelete() {
  const answer = prompt('delete question?')
  if(answer === 'yes') {
    deleteQuestion()
  }
}

Questions?

Exercise 5

src/components/NavBar.spec.js

jest.mock

// utils.spec.js

import { promptDelete }

test('calls deleteQuestion if user answers yes', () => {
  window.prompt = () => 'yes'
  
  promptDelete()

  // ?
})
// utils.spec.js

import { promptDelete } from './utils'
import { deleteQuestion } from './db'

jest.mock('./db')

test('calls deleteQuestion if user answers yes', () => {
  window.prompt = () => 'yes'
  
  promptDelete()

  expect(deleteQuestion).toHaveBeenCalled()
})

How it works

import { deleteQuestion } from './db'

jest.mock('./db')

deleteQuestion // is mock function
'use strict';

jest.mock('./db');

var _db = require('./db');

_db.deleteQuestion; // is mock function

When to use

  • Stop SLOW side effects (HTTP calls,  connecting to a database)
  • To control return value
  • To assert method was called

Questions?

Exercise 6

src/components/Question.spec.js

<template>
  <a v-if="signedIn" @click="signOut">
    Sign out
  </a>
  <router-link to="/sign-in" v-else >
    Sign in
  </router-link>
</template>
import Modal from './Modal.vue'

wrapper.find(Modal).props()
import { RouterLinkStub, shallowMount } from '@vue/test-utils'

const wrapper = shallowMount(SignInOutLink, {
  stubs: {
    'router-link': RouterLinkStub
  }
})

wrapper.find(RouterLinkStub).props()

Stubbing components

import Vue from 'vue'
import VueRouter from 'vue-router'

Vue.use(VueRouter)

// router-view and router-link will be 
// registered components
mount(Component)

WARNING ⚠️

Never install Vue Router on base constructor

// under the hood

Object.defineProperty(Vue.prototype, '$router', {
  get () { return this._routerRoot._router }
})

Object.defineProperty(Vue.prototype, '$route', {
  get () { return this._routerRoot._route }
})

https://vue-test-utils.vuejs.org/guides/using-with-vue-router.html

Questions?

Exercise 7

src/components/Question.spec.js

Timer functions

  • Functions like setInterval, setTimeout, setImmediate

  • Run in real-time 
  • Need to be mocked in unit tests
function hideModal(cb) {
  setTimeout(() => {
    document.querySelector('.modal')
      .classList.remove('show')
  }, 3000)
}

Mocking timer functions

jest.useFakeTimers()

jest.advanceTimersByTime(2000) // cb fired
window.setTimeout = mockSetTimeout

Questions?

Exercise 8

src/components/ProgressBar.spec.js

Testing async code

test('returns data', () => {
  // ..
  const data = fetchItems()
  expect(data).toBe(expectedData)
})
test('returns data', async () => {
  expect.assertions(1)
  const data = await fetchItems()
  expect(data).toBe(expectedData)
})
<script>
  import { fetchItems } from './api'

  export default {
    beforeCreate() {
      fetchItems()
        .then(items => {
          this.items = items 
        })
    }
  }
</script>
const flushPromises = () => (
  new Promise(resolve => setTimeout(resolve))
)
test('renders items using items from fetchItems', async () => {
  expect.assertions(1)
  const wrapper = shallowMount(ItemList)
  expect(wrapper.findAll(Item)).toHaveLength(3)
})
test('renders items using items from fetchItems', async () => {
  expect.assertions(1)
  const wrapper = shallowMount(ItemList)
  await flushPromises()
  expect(wrapper.findAll(Item)).toHaveLength(3)
})

Microtasks

Spec:

https://html.spec.whatwg.org/multipage/webappapis.html#event-loop-processing-model

Blog post:

https://jakearchibald.com/2015/tasks-microtasks-queues-and-schedules/

While the event loop's microtask queue is not empty:

                      [Execute micro tasks]

Questions?

Exercise 9

src/views/ProfileView.spec.js

 

Code coverage

jest --coverage

 

Diminishing returns

time writing tests

code

coverage

Snapshot tests

Snapshot testing

// Jest Snapshot v1, https://goo.gl/fbAQLP

exports[`renders correctly 1`] = `
<div class="modal is-active">
    <div class="modal-background"></div>
    <div class="modal-contents">
        <div class="box">
            <button aria-label="close" class="delete"></button>
            <p>some content</p>
        </div>
    </div>
</div>
`;

Does previous

snapshot exist?

create snapshot

compare to previous

snapshot

test passes

No

Yes

does output

match

snapshshot?

create snapshot file

test fails

test passes

Compare to previous snapshot

Yes

No

Benefits of snapshot tests

  • Ensure presentational HTML doesn't change
  • Lots of coverage with little code

Questions?

Exercise 10

src/components/ErrorModal.spec.js

Integration tests

Topics

  • Cypress

Integration tests

  • Mock part of the system
  • Tricky to define
  • More coverage with less code
  • Can be difficult to debug

Cypress

Adding to project

yarn add --dev cypress
// package.json
{
  // ..
  scripts: {
    // ..
    "test": "npm run lint && npm run test:unit && npm run test:integration",
    "test:integration": "cypress run"
  }
  // ..
}
# Initialize cypress 

$ npx cypress open

Trade offs

  • Browser support
  • No support for tabs
  • You cannot visit different domains in the same test

Questions?

Exercise 11

cypress/integration/sign-in.spec.js

E2E tests

Topics

  • WebDriver

E2E Tests

  • Automate browser to interact with app
  • No mocking

WebDriver

  • Automates a browser

  • Supported by almost all browsers

  • W3C Recommendation

selenium server

POST

/wd/hub/session

200 POST

/wd/hub/session

Automating a browser using selenium server and the WebDriver protocol

WebDriver problems

  • Slow

  • Flakey

  • Difficult to debug

Run WebDriver tests in a Docker container to solve reproducibility issues!

browser
  .url(devServerURL)
  .waitForElementVisible('#app', 5000)
  .click('.modal button')
  .assert.elementNotPresent('.modal')
  .end()

Questions?

E2E

tests

Integration tests

Unit tests/ Snapshot tests

https://goo.gl/Auof8K

Feedback

Time to add to a real-world project

Testing Vue apps workshop

By Edd Yerburgh

Testing Vue apps workshop

Slides for my workshop on testing Vue apps

  • 834
Loading comments...

More from Edd Yerburgh