Snapshot Testing the Hard Way
Gleb Bahmutov aka "the sub"
KENSHO
Technology that brings transparency to complex systems
Harvard Sq, WTC NYC
Tests!!!
source: http://www.citiconstruction.net/apps/blog/show/43096160-citi-scaffolding-pte-ltd-
Tests are scaffolding
source: http://www.citiconstruction.net/apps/blog/show/43096160-citi-scaffolding-pte-ltd-
Customer does not pay for tests 😡
source: http://www.citiconstruction.net/apps/blog/show/43096160-citi-scaffolding-pte-ltd-
Once building is finished and sold...
source: http://www.citiconstruction.net/apps/blog/show/43096160-citi-scaffolding-pte-ltd-
You keep maintaining the tests! 😡😡😡
Test types
unit tests, unit tests, unit tests, unit tests, unit tests, unit tests
integration tests, integration tests
end to end tests
Test types: practice
linter, unit tests, unit tests, unit tests, crash reporting
integration tests, integration tests
end to end tests
Test types: tools
linter, unit tests, unit tests, unit tests, crash reporting
integration tests, integration tests
end to end tests
standard
mocha, ava, jest
sentry
API / browser tests
A good unit test
describe('something', () => {
it('does this', () => {
})
})https://glebbahmutov.com/blog/unit-test-node-code-in-10-seconds/
const actual = ...
assert.equals(actual, expected)A good API test
describe('API feature', () => {
it('returns items', () => {
return get(url)
.then(list => {
assert.equals(list, expected)
})
})
})https://glebbahmutov.com/blog/rest-api-testing-made-easy/
where does the expected value come from?
Save expected value
describe('API feature', () => {
it('returns items', () => {
return get(url)
.then(list => {
assert.equals(list, expected)
})
})
})boilerplate!
let expected
if (firstTime) {
save('returns-items.json', list)
expected = list
} else {
expected = load('returns-items.json')
}Jest snapshots
describe('API feature', () => {
test('returns items', () => {
return get(url)
.then(list => {
expect(list).toMatchSnapshot()
})
})
})https://facebook.github.io/jest/blog/2016/07/27/jest-14.html
Jest snapshots
// __snapshots__/test.snapshot.js
exports[`returns items 1`] = `
['foo', 'bar', 'baz']
`https://facebook.github.io/jest/blog/2016/07/27/jest-14.html
Ava has added snapshot testing
https://github.com/avajs/ava/releases/tag/v0.18.0
Separate jest-snapshot
https://github.com/facebook/jest/issues/2497
My favorite is Mocha 😞
https://www.javascripting.com/view/mocha
snap-shot
https://github.com/bahmutov/snap-shot
const snapshot = require('snap-shot')
it('is 42', () => {
snapshot(42)
})Any testing framework, async, transpiled code, multiple snapshots per test, anonymous tests
Object snapshots
it('compares objects', () => {
const o = {
inner: {
a: 10,
b: 20
},
foo: 'foo (changed)',
newProperty: 'here'
}
snapshot(o)
})changed value
new value
Text snapshots
it('compares multi line strings', () => {
snapshot(`
this is a line
and a second line (changed)
with number 42
`)
})changed line
Implementation details
const snapshot = require('snap-shot')
it('is 42', () => {
snapshot(42)
})// snap-shot/index.js
function snapshot(what) {
// hmm, which test called me?
}Jest / Ava / ...
const snapshot = require('snap-shot')
it('is 42', () => {
snapshot(42)
})// other frameworks
function snapshot(what) {
const testName = this.currentText.name
}snap-shot
const snapshot = require('snap-shot')
it('is 42', () => {
snapshot(42)
})// snap-shot/index.js
function snapshot(what) {
const stack = (new Error('')).stack
// snapshot (node_modules/snap-shot/index.js:20:10)
// anonymous (my-spec.js:3:2)
// ...
}use any name!
Abstract Syntax Tree
location around
call to "snapshot"
const falafel = require('falafel')
const callLocation = {line: 3}
falafel(source, options, node => {
if (node.type === 'CallExpression') {
// if node is around callLocation
// found it!
}
})Different uses of Abstract Syntax Tree in "real world" https://glebbahmutov.com/blog/tags/ast/
Parsing and traversing AST is simple with https://github.com/substack/node-falafel
Once we know the test name
Saving / loading /diffing values is simple
const snapShot = require('snap-shot-core')
const what // my object / value
const out = snapShot({
what,
file: __filename,
specName: 'my test',
store,
compare: compareFn,
ext: '.test'
})Dynamic data?
const snapshot = require('snap-shot')
it('returns most popular item', () => {
const top = api.getMostPopularItem()
snapshot(top)
})does not work 😩
Dynamic data!
const snapshot = require('schema-shot')
it('returns most popular item', () => {
const top = api.getMostPopularItem()
snapshot(top)
})works 🤗 !
Schema-shot - snapshot testing for dynamic data
Stores schema
schemaShot({id: 'd13ef'})
exports['returns most popular item 1'] = {
"$schema": "http://json-schema.org/draft-04/schema#",
"type": "object",
"properties": {
"id": {
"type": "string",
"required": true
}
},
"additionalProperties": false
}Schema-shot - snapshot testing for dynamic data
Validates object against schema
schemaShot({id: '7fe101'})Schema-shot - snapshot testing for dynamic data
Error: schema difference data has additional properties data.id: is required
schemaShot({uuid: '66635'})✅
Future plans
-
file-system-shot
-
network-proxy-shot
Help wanted ...
Snapshot Testing the Hard Way