Snap-shot Testing

Dr. Gleb Bahmutov PhD

VP of Eng Cypress.io

Tests!!!

source: http://www.citiconstruction.net/apps/blog/show/43096160-citi-scaffolding-pte-ltd-

Tests are scaffolding

source: http://www.citiconstruction.net/apps/blog/show/43096160-citi-scaffolding-pte-ltd-

Customer does not pay for tests 😡

source: http://www.citiconstruction.net/apps/blog/show/43096160-citi-scaffolding-pte-ltd-

Once building is finished and sold...

source: http://www.citiconstruction.net/apps/blog/show/43096160-citi-scaffolding-pte-ltd-

You keep maintaining the tests! 😡😡😡

Test types

unit tests, unit tests, unit tests, unit tests, unit tests, unit tests

integration tests, integration tests

end to end tests

Test types: practice

linter, unit tests, unit tests, unit tests, crash reporting

integration tests, integration tests

end to end tests

Test types: tools

linter, unit tests, unit tests, unit tests, crash reporting

integration tests, integration tests

end to end tests

standard

mocha, ava, jest

sentry

API / browser tests

A good unit test

describe('something', () => {
  it('does this', () => {




  })
})

https://glebbahmutov.com/blog/unit-test-node-code-in-10-seconds/

    const actual = ...
    assert.equals(actual, expected)

A good API test

describe('API feature', () => {
  it('returns items', () => {
    return get(url)
      .then(list => {
        assert.equals(list, expected)
      })
  })
})

where does the expected value come from?

Save expected value

describe('API feature', () => {
  it('returns items', () => {
    return get(url)
      .then(list => {








        assert.equals(list, expected)
      })
  })
})

boilerplate!

        let expected
        if (firstTime) {
          save('returns-items.json', list)
          expected = list
        } else {
          expected = load('returns-items.json')
        }

Jest snapshots

import React from 'react';
import Link from '../Link.react';
import renderer from 'react-test-renderer';

it('renders correctly', () => {
  const tree = renderer.create(
    <Link page="http://www.facebook.com">Facebook</Link>
  ).toJSON();
  expect(tree).toMatchSnapshot();
});

Jest saves snapshot

exports[`renders correctly 1`] = `
<a
  className="normal"
  href="http://www.facebook.com"
  onMouseEnter={[Function]}
  onMouseLeave={[Function]}
>
  Facebook
</a>
`;

Jest shows difference

Ava has added snapshot testing

https://github.com/avajs/ava/releases/tag/v0.18.0

Separate jest-snapshot

https://github.com/facebook/jest/issues/2497

My favorite is Mocha 😞

https://www.javascripting.com/view/mocha

snap-shot

https://github.com/bahmutov/snap-shot

const snapshot = require('snap-shot')
it('is 42', () => {
  snapshot(42)
})

Any testing framework, async, transpiled code, multiple snapshots per test, anonymous tests

*

Object snapshots

it('compares objects', () => {
  const o = {
    inner: {
      a: 10,
      b: 20
    },
    foo: 'foo (changed)',
    newProperty: 'here'
  }
  snapshot(o)
})

changed value

new value

Text snapshots

it('compares multi line strings', () => {
  snapshot(`
    this is a line
    and a second line (changed)
    with number 42
  `)
})

changed line

Implementation details

I don't want to depend on specific test framework

Implementation details

const snapshot = require('snap-shot')
it('is 42', () => {
  snapshot(42)
})
// snap-shot/index.js
function snapshot(what) {
  // hmm, which test called me?
}

Jest / Ava / ...

const snapshot = require('snap-shot')
it('is 42', () => {
  snapshot(42)
})
// other frameworks
function snapshot(what) {
  const testName = this.currentText.name
}

snap-shot

const snapshot = require('snap-shot')
it('is 42', () => {
  snapshot(42)
})
// snap-shot/index.js
function snapshot(what) {
  const stack = (new Error('')).stack
  // snapshot (node_modules/snap-shot/index.js:20:10)
  // anonymous (my-spec.js:3:2)
  // ...
}

use any name!

Abstract Syntax Tree

location around

call to "snapshot"

const falafel = require('falafel')
const callLocation = {line: 3}
falafel(source, options, node => {
  if (node.type === 'CallExpression') {
    // if node is around callLocation
    // found it!
  }
})

Different uses of Abstract Syntax Tree in "real world" https://glebbahmutov.com/blog/tags/ast/

Parsing and traversing AST is simple with https://github.com/substack/node-falafel

Once we know the test name

Saving / loading /diffing values is simple

const snapShot = require('snap-shot-core')
const what // my object / value
const out = snapShot({
  what,
  file: __filename,
  specName: 'my test',
  store,
  compare: compareFn,
  ext: '.test'
})

Dynamic data?

const snapshot = require('snap-shot')
it('returns most popular item', () => {
  const top = api.getMostPopularItem()
  snapshot(top)
})

does not work 😩

Dynamic data!

const snapshot = require('schema-shot')
it('returns most popular item', () => {
  const top = api.getMostPopularItem()
  snapshot(top)
})

works 🤗 ! 

Schema-shot - snapshot testing for dynamic data

https://glebbahmutov.com/blog/schema-shot/

Stores schema

schemaShot({id: 'd13ef'})

exports['returns most popular item 1'] = {
  "$schema": "http://json-schema.org/draft-04/schema#",
  "type": "object",
  "properties": {
    "id": {
      "type": "string",
      "required": true
    }
  },
  "additionalProperties": false
}

Schema-shot - snapshot testing for dynamic data

https://glebbahmutov.com/blog/schema-shot/

Validates object against schema

schemaShot({id: '7fe101'})

Schema-shot - snapshot testing for dynamic data

https://glebbahmutov.com/blog/schema-shot/

Error: schema difference
  data has additional properties
  data.id: is required
schemaShot({uuid: '66635'})

snap-shot

snap-shot-core

schema-shot

snap-shot-jest-test

snap-shot-ava-test

snap-shot-vue-test

semantic release

with dont-crack

semantic release

with dont-crack

...

Dependencies structure

OSS Community

by Mike Calvanese

OSS Community

by Mike Calvanese

Add data-driven testing

// checks if n is prime
const isPrime = n => ...
it('tests prime', () => {
  snapshot(isPrime, 1, 2, 3, 4, 5, 6, 7, 8, 9)
})

pure function

inputs

Add data-driven testing

// snapshot file
exports['tests prime 1'] = {
  "name": "isPrime",
  "behavior": [
    {
      "given": 1,
      "expect": false
    },
    {
      "given": 2,
      "expect": true
    },
    {
      "given": 3,
      "expect": true
    },
    ...
  ]
}

OSS Community

Remix good ideas you see around you.

Perseverance

https://github.com/bahmutov/snap-shot

const snapshot = require('snap-shot')
it('is 42', () => {
  snapshot(42)
})

Any testing framework, async, transpiled code, multiple snapshots per test, anonymous tests

*

Single promise stack trace is missing

#35

Unexpected snapshot id when using with Sazerac

#40

Can this work with TypeScript

#72

Does not handle one liners

#80

It is hard to match source code to runtime frame

AND

it is hard to match transpiled code to original source

Solution: overwrite global.it function and grab test callback

Solution: grab current test at runtime

const snapshot = require('snap-shot-it')
it('works', () => {
  snapshot(42)
})
// snap-shot-it.js
const core = require('snap-shot-core')
beforeEach(function () {
  global.currentTest = this.test
})
function snapshot(value) {
  // use global.currentTest.title
}

Most testing frameworks, async, transpiled code, multiple snapshots per test, anonymous tests, data-driven testing

Deprecated

Perseverance

It takes a few attempts to get something right

I wrote <name of a project> because existing tools were missing <feature X>. I refined my solution because <reason Y and Z>. I plan to add <something cool> because there is limitation due to <the reason>.

My Interview

Thank you!

$ npm i -D snap-shot-it

Snap-shot Testing

Snapshot Testing

By Gleb Bahmutov

Snapshot Testing

Snapshot Testing is a recent technique in unit testing and automated testing in general that's been gaining some traction, thanks to a new excellent test runner called Jest. Gleb Bahmutov from Cypress.io will explain to us what snapshot testing is, and how to use his family of NPM modules snap-shot-* to go beyond static data. Presented at Atlanta JS Meetup, video at https://www.youtube.com/watch?v=MAKRJJ06Nw4

  • 5,350