Snapshot Testing the Hard Way

Gleb Bahmutov aka "the sub"

KENSHO

Technology that brings transparency to complex systems

Harvard Sq, WTC NYC

Tests!!!

source: http://www.citiconstruction.net/apps/blog/show/43096160-citi-scaffolding-pte-ltd-

Tests are scaffolding

source: http://www.citiconstruction.net/apps/blog/show/43096160-citi-scaffolding-pte-ltd-

Customer does not pay for tests 😡

source: http://www.citiconstruction.net/apps/blog/show/43096160-citi-scaffolding-pte-ltd-

Once building is finished and sold...

source: http://www.citiconstruction.net/apps/blog/show/43096160-citi-scaffolding-pte-ltd-

You keep maintaining the tests! 😡😡😡

Test types

unit tests, unit tests, unit tests, unit tests, unit tests, unit tests

integration tests, integration tests

end to end tests

Test types: practice

linter, unit tests, unit tests, unit tests, crash reporting

integration tests, integration tests

end to end tests

Test types: tools

linter, unit tests, unit tests, unit tests, crash reporting

integration tests, integration tests

end to end tests

standard

mocha, ava, jest

sentry

API / browser tests

A good unit test

describe('something', () => {
  it('does this', () => {




  })
})

https://glebbahmutov.com/blog/unit-test-node-code-in-10-seconds/

    const actual = ...
    assert.equals(actual, expected)

A good API test

describe('API feature', () => {
  it('returns items', () => {
    return get(url)
      .then(list => {
        assert.equals(list, expected)
      })
  })
})

https://glebbahmutov.com/blog/rest-api-testing-made-easy/

where does the expected value come from?

Save expected value

describe('API feature', () => {
  it('returns items', () => {
    return get(url)
      .then(list => {








        assert.equals(list, expected)
      })
  })
})

boilerplate!

        let expected
        if (firstTime) {
          save('returns-items.json', list)
          expected = list
        } else {
          expected = load('returns-items.json')
        }

Jest snapshots

describe('API feature', () => {
  test('returns items', () => {
    return get(url)
      .then(list => {
        expect(list).toMatchSnapshot()
      })
  })
})

https://facebook.github.io/jest/blog/2016/07/27/jest-14.html

Jest snapshots

// __snapshots__/test.snapshot.js
exports[`returns items 1`] = `
['foo', 'bar', 'baz']
`

https://facebook.github.io/jest/blog/2016/07/27/jest-14.html

Ava has added snapshot testing

https://github.com/avajs/ava/releases/tag/v0.18.0

Separate jest-snapshot

https://github.com/facebook/jest/issues/2497

My favorite is Mocha 😞

https://www.javascripting.com/view/mocha

snap-shot

https://github.com/bahmutov/snap-shot

const snapshot = require('snap-shot')
it('is 42', () => {
  snapshot(42)
})

Any testing framework, async, transpiled code, multiple snapshots per test, anonymous tests

Object snapshots

it('compares objects', () => {
  const o = {
    inner: {
      a: 10,
      b: 20
    },
    foo: 'foo (changed)',
    newProperty: 'here'
  }
  snapshot(o)
})

changed value

new value

Text snapshots

it('compares multi line strings', () => {
  snapshot(`
    this is a line
    and a second line (changed)
    with number 42
  `)
})

changed line

Implementation details

const snapshot = require('snap-shot')
it('is 42', () => {
  snapshot(42)
})
// snap-shot/index.js
function snapshot(what) {
  // hmm, which test called me?
}

Jest / Ava / ...

const snapshot = require('snap-shot')
it('is 42', () => {
  snapshot(42)
})
// other frameworks
function snapshot(what) {
  const testName = this.currentText.name
}

snap-shot

const snapshot = require('snap-shot')
it('is 42', () => {
  snapshot(42)
})
// snap-shot/index.js
function snapshot(what) {
  const stack = (new Error('')).stack
  // snapshot (node_modules/snap-shot/index.js:20:10)
  // anonymous (my-spec.js:3:2)
  // ...
}

use any name!

Abstract Syntax Tree

location around

call to "snapshot"

const falafel = require('falafel')
const callLocation = {line: 3}
falafel(source, options, node => {
  if (node.type === 'CallExpression') {
    // if node is around callLocation
    // found it!
  }
})

Different uses of Abstract Syntax Tree in "real world" https://glebbahmutov.com/blog/tags/ast/

Parsing and traversing AST is simple with https://github.com/substack/node-falafel

Once we know the test name

Saving / loading /diffing values is simple

const snapShot = require('snap-shot-core')
const what // my object / value
const out = snapShot({
  what,
  file: __filename,
  specName: 'my test',
  store,
  compare: compareFn,
  ext: '.test'
})

Dynamic data?

const snapshot = require('snap-shot')
it('returns most popular item', () => {
  const top = api.getMostPopularItem()
  snapshot(top)
})

does not work 😩

Dynamic data!

const snapshot = require('schema-shot')
it('returns most popular item', () => {
  const top = api.getMostPopularItem()
  snapshot(top)
})

works 🤗 ! 

Schema-shot - snapshot testing for dynamic data

https://glebbahmutov.com/blog/schema-shot/

Stores schema

schemaShot({id: 'd13ef'})

exports['returns most popular item 1'] = {
  "$schema": "http://json-schema.org/draft-04/schema#",
  "type": "object",
  "properties": {
    "id": {
      "type": "string",
      "required": true
    }
  },
  "additionalProperties": false
}

Schema-shot - snapshot testing for dynamic data

https://glebbahmutov.com/blog/schema-shot/

Validates object against schema

schemaShot({id: '7fe101'})

Schema-shot - snapshot testing for dynamic data

https://glebbahmutov.com/blog/schema-shot/

Error: schema difference
  data has additional properties
  data.id: is required
schemaShot({uuid: '66635'})

Future plans

  • file-system-shot

  • network-proxy-shot

Help wanted ...

Snapshot Testing the Hard Way

Thank you!

Snapshot Testing the Hard Way

By Gleb Bahmutov

Snapshot Testing the Hard Way

Testing is hard and does not pay. Snapshot testing is a way to increase the efficiency of your unit tests while removing a lot of boilerplate code. Recently Jest and Ava testing framework added this feature. I have written a framework-agnostic snapshot testing library and would like to share techniques and design decisions that went into this library.

  • 7,158