Gleb Bahmutov PRO
JavaScript ninja, image processing expert, software quality fanatic
Snap-shot Testing
Dr. Gleb Bahmutov PhD
VP of Eng Cypress.io
Tests!!!
source: http://www.citiconstruction.net/apps/blog/show/43096160-citi-scaffolding-pte-ltd-
Tests are scaffolding
source: http://www.citiconstruction.net/apps/blog/show/43096160-citi-scaffolding-pte-ltd-
Customer does not pay for tests 😡
source: http://www.citiconstruction.net/apps/blog/show/43096160-citi-scaffolding-pte-ltd-
Once building is finished and sold...
source: http://www.citiconstruction.net/apps/blog/show/43096160-citi-scaffolding-pte-ltd-
You keep maintaining the tests! 😡😡😡
Test types
unit tests, unit tests, unit tests, unit tests, unit tests, unit tests
integration tests, integration tests
end to end tests
Test types: practice
linter, unit tests, unit tests, unit tests, crash reporting
integration tests, integration tests
end to end tests
Test types: tools
linter, unit tests, unit tests, unit tests, crash reporting
integration tests, integration tests
end to end tests
standard
mocha, ava, jest
sentry
API / browser tests
A good unit test
describe('something', () => {
it('does this', () => {
})
})https://glebbahmutov.com/blog/unit-test-node-code-in-10-seconds/
const actual = ...
assert.equals(actual, expected)A good API test
describe('API feature', () => {
it('returns items', () => {
return get(url)
.then(list => {
assert.equals(list, expected)
})
})
})where does the expected value come from?
Save expected value
describe('API feature', () => {
it('returns items', () => {
return get(url)
.then(list => {
assert.equals(list, expected)
})
})
})boilerplate!
let expected
if (firstTime) {
save('returns-items.json', list)
expected = list
} else {
expected = load('returns-items.json')
}Jest snapshots
import React from 'react';
import Link from '../Link.react';
import renderer from 'react-test-renderer';
it('renders correctly', () => {
const tree = renderer.create(
<Link page="http://www.facebook.com">Facebook</Link>
).toJSON();
expect(tree).toMatchSnapshot();
});Jest saves snapshot
exports[`renders correctly 1`] = `
<a
className="normal"
href="http://www.facebook.com"
onMouseEnter={[Function]}
onMouseLeave={[Function]}
>
Facebook
</a>
`;Jest shows difference
Ava has added snapshot testing
https://github.com/avajs/ava/releases/tag/v0.18.0
Separate jest-snapshot
https://github.com/facebook/jest/issues/2497
My favorite is Mocha 😞
https://www.javascripting.com/view/mocha
snap-shot
https://github.com/bahmutov/snap-shot
const snapshot = require('snap-shot')
it('is 42', () => {
snapshot(42)
})Any testing framework, async, transpiled code, multiple snapshots per test, anonymous tests
*
Object snapshots
it('compares objects', () => {
const o = {
inner: {
a: 10,
b: 20
},
foo: 'foo (changed)',
newProperty: 'here'
}
snapshot(o)
})changed value
new value
Text snapshots
it('compares multi line strings', () => {
snapshot(`
this is a line
and a second line (changed)
with number 42
`)
})changed line
Implementation details
I don't want to depend on specific test framework
Implementation details
const snapshot = require('snap-shot')
it('is 42', () => {
snapshot(42)
})// snap-shot/index.js
function snapshot(what) {
// hmm, which test called me?
}Jest / Ava / ...
const snapshot = require('snap-shot')
it('is 42', () => {
snapshot(42)
})// other frameworks
function snapshot(what) {
const testName = this.currentText.name
}snap-shot
const snapshot = require('snap-shot')
it('is 42', () => {
snapshot(42)
})// snap-shot/index.js
function snapshot(what) {
const stack = (new Error('')).stack
// snapshot (node_modules/snap-shot/index.js:20:10)
// anonymous (my-spec.js:3:2)
// ...
}use any name!
Abstract Syntax Tree
location around
call to "snapshot"
const falafel = require('falafel')
const callLocation = {line: 3}
falafel(source, options, node => {
if (node.type === 'CallExpression') {
// if node is around callLocation
// found it!
}
})Different uses of Abstract Syntax Tree in "real world" https://glebbahmutov.com/blog/tags/ast/
Parsing and traversing AST is simple with https://github.com/substack/node-falafel
Once we know the test name
Saving / loading /diffing values is simple
const snapShot = require('snap-shot-core')
const what // my object / value
const out = snapShot({
what,
file: __filename,
specName: 'my test',
store,
compare: compareFn,
ext: '.test'
})Dynamic data?
const snapshot = require('snap-shot')
it('returns most popular item', () => {
const top = api.getMostPopularItem()
snapshot(top)
})does not work 😩
Dynamic data!
const snapshot = require('schema-shot')
it('returns most popular item', () => {
const top = api.getMostPopularItem()
snapshot(top)
})works 🤗 !
Schema-shot - snapshot testing for dynamic data
Stores schema
schemaShot({id: 'd13ef'})
exports['returns most popular item 1'] = {
"$schema": "http://json-schema.org/draft-04/schema#",
"type": "object",
"properties": {
"id": {
"type": "string",
"required": true
}
},
"additionalProperties": false
}Schema-shot - snapshot testing for dynamic data
Validates object against schema
schemaShot({id: '7fe101'})Schema-shot - snapshot testing for dynamic data
Error: schema difference data has additional properties data.id: is required
schemaShot({uuid: '66635'})✅
snap-shot
snap-shot-core
schema-shot
snap-shot-jest-test
snap-shot-ava-test
snap-shot-vue-test
semantic release
with dont-crack
semantic release
with dont-crack
...
Dependencies structure
OSS Community
by Mike Calvanese
OSS Community
by Mike Calvanese
Add data-driven testing
// checks if n is prime
const isPrime = n => ...
it('tests prime', () => {
snapshot(isPrime, 1, 2, 3, 4, 5, 6, 7, 8, 9)
})pure function
inputs
Add data-driven testing
// snapshot file
exports['tests prime 1'] = {
"name": "isPrime",
"behavior": [
{
"given": 1,
"expect": false
},
{
"given": 2,
"expect": true
},
{
"given": 3,
"expect": true
},
...
]
}OSS Community
Remix good ideas you see around you.
Perseverance
https://github.com/bahmutov/snap-shot
const snapshot = require('snap-shot')
it('is 42', () => {
snapshot(42)
})Any testing framework, async, transpiled code, multiple snapshots per test, anonymous tests
*
Single promise stack trace is missing
Unexpected snapshot id when using with Sazerac
Can this work with TypeScript
Does not handle one liners
It is hard to match source code to runtime frame
AND
it is hard to match transpiled code to original source
Solution: overwrite global.it function and grab test callback
Solution: grab current test at runtime
const snapshot = require('snap-shot-it')
it('works', () => {
snapshot(42)
})// snap-shot-it.js
const core = require('snap-shot-core')
beforeEach(function () {
global.currentTest = this.test
})
function snapshot(value) {
// use global.currentTest.title
}Most testing frameworks, async, transpiled code, multiple snapshots per test, anonymous tests, data-driven testing
Deprecated
Perseverance
It takes a few attempts to get something right
I wrote <name of a project> because existing tools were missing <feature X>. I refined my solution because <reason Y and Z>. I plan to add <something cool> because there is limitation due to <the reason>.
My Interview
Thank you!
$ npm i -D snap-shot-itSnap-shot Testing
Snapshot Testing
By Gleb Bahmutov
Snapshot Testing
Snapshot Testing is a recent technique in unit testing and automated testing in general that's been gaining some traction, thanks to a new excellent test runner called Jest. Gleb Bahmutov from Cypress.io will explain to us what snapshot testing is, and how to use his family of NPM modules snap-shot-* to go beyond static data. Presented at Atlanta JS Meetup, video at https://www.youtube.com/watch?v=MAKRJJ06Nw4
