Gleb Bahmutov PRO
JavaScript ninja, image processing expert, software quality fanatic
Snapshot Testing the Hard Way
Gleb Bahmutov aka "the sub"
KENSHO
Technology that brings transparency to complex systems
Harvard Sq, WTC NYC
Tests!!!
source: http://www.citiconstruction.net/apps/blog/show/43096160-citi-scaffolding-pte-ltd-
Tests are scaffolding
source: http://www.citiconstruction.net/apps/blog/show/43096160-citi-scaffolding-pte-ltd-
Customer does not pay for tests 😡
source: http://www.citiconstruction.net/apps/blog/show/43096160-citi-scaffolding-pte-ltd-
Once building is finished and sold...
source: http://www.citiconstruction.net/apps/blog/show/43096160-citi-scaffolding-pte-ltd-
You keep maintaining the tests! 😡😡😡
Test types
unit tests, unit tests, unit tests, unit tests, unit tests, unit tests
integration tests, integration tests
end to end tests
Test types: practice
linter, unit tests, unit tests, unit tests, crash reporting
integration tests, integration tests
end to end tests
Test types: tools
linter, unit tests, unit tests, unit tests, crash reporting
integration tests, integration tests
end to end tests
standard
mocha, ava, jest
sentry
API / browser tests
A good unit test
describe('something', () => {
it('does this', () => {
})
})https://glebbahmutov.com/blog/unit-test-node-code-in-10-seconds/
const actual = ...
assert.equals(actual, expected)A good API test
describe('API feature', () => {
it('returns items', () => {
return get(url)
.then(list => {
assert.equals(list, expected)
})
})
})https://glebbahmutov.com/blog/rest-api-testing-made-easy/
where does the expected value come from?
Save expected value
describe('API feature', () => {
it('returns items', () => {
return get(url)
.then(list => {
assert.equals(list, expected)
})
})
})boilerplate!
let expected
if (firstTime) {
save('returns-items.json', list)
expected = list
} else {
expected = load('returns-items.json')
}Jest snapshots
describe('API feature', () => {
test('returns items', () => {
return get(url)
.then(list => {
expect(list).toMatchSnapshot()
})
})
})https://facebook.github.io/jest/blog/2016/07/27/jest-14.html
Jest snapshots
// __snapshots__/test.snapshot.js
exports[`returns items 1`] = `
['foo', 'bar', 'baz']
`https://facebook.github.io/jest/blog/2016/07/27/jest-14.html
Ava has added snapshot testing
https://github.com/avajs/ava/releases/tag/v0.18.0
Separate jest-snapshot
https://github.com/facebook/jest/issues/2497
My favorite is Mocha 😞
https://www.javascripting.com/view/mocha
snap-shot
https://github.com/bahmutov/snap-shot
const snapshot = require('snap-shot')
it('is 42', () => {
snapshot(42)
})Any testing framework, async, transpiled code, multiple snapshots per test, anonymous tests
Object snapshots
it('compares objects', () => {
const o = {
inner: {
a: 10,
b: 20
},
foo: 'foo (changed)',
newProperty: 'here'
}
snapshot(o)
})changed value
new value
Text snapshots
it('compares multi line strings', () => {
snapshot(`
this is a line
and a second line (changed)
with number 42
`)
})changed line
Implementation details
const snapshot = require('snap-shot')
it('is 42', () => {
snapshot(42)
})// snap-shot/index.js
function snapshot(what) {
// hmm, which test called me?
}Jest / Ava / ...
const snapshot = require('snap-shot')
it('is 42', () => {
snapshot(42)
})// other frameworks
function snapshot(what) {
const testName = this.currentText.name
}snap-shot
const snapshot = require('snap-shot')
it('is 42', () => {
snapshot(42)
})// snap-shot/index.js
function snapshot(what) {
const stack = (new Error('')).stack
// snapshot (node_modules/snap-shot/index.js:20:10)
// anonymous (my-spec.js:3:2)
// ...
}use any name!
Abstract Syntax Tree
location around
call to "snapshot"
const falafel = require('falafel')
const callLocation = {line: 3}
falafel(source, options, node => {
if (node.type === 'CallExpression') {
// if node is around callLocation
// found it!
}
})Different uses of Abstract Syntax Tree in "real world" https://glebbahmutov.com/blog/tags/ast/
Parsing and traversing AST is simple with https://github.com/substack/node-falafel
Once we know the test name
Saving / loading /diffing values is simple
const snapShot = require('snap-shot-core')
const what // my object / value
const out = snapShot({
what,
file: __filename,
specName: 'my test',
store,
compare: compareFn,
ext: '.test'
})Dynamic data?
const snapshot = require('snap-shot')
it('returns most popular item', () => {
const top = api.getMostPopularItem()
snapshot(top)
})does not work 😩
Dynamic data!
const snapshot = require('schema-shot')
it('returns most popular item', () => {
const top = api.getMostPopularItem()
snapshot(top)
})works 🤗 !
Schema-shot - snapshot testing for dynamic data
Stores schema
schemaShot({id: 'd13ef'})
exports['returns most popular item 1'] = {
"$schema": "http://json-schema.org/draft-04/schema#",
"type": "object",
"properties": {
"id": {
"type": "string",
"required": true
}
},
"additionalProperties": false
}Schema-shot - snapshot testing for dynamic data
Validates object against schema
schemaShot({id: '7fe101'})Schema-shot - snapshot testing for dynamic data
Error: schema difference data has additional properties data.id: is required
schemaShot({uuid: '66635'})✅
Future plans
-
file-system-shot
-
network-proxy-shot
Help wanted ...
Snapshot Testing the Hard Way
Thank you!
Snapshot Testing the Hard Way
By Gleb Bahmutov
Snapshot Testing the Hard Way
Testing is hard and does not pay. Snapshot testing is a way to increase the efficiency of your unit tests while removing a lot of boilerplate code. Recently Jest and Ava testing framework added this feature. I have written a framework-agnostic snapshot testing library and would like to share techniques and design decisions that went into this library.
