JSON Schemas are Your True Testing Friend
C / C++ / C# / Java / CoffeeScript / JavaScript / Node / Angular / Vue / Cycle.js / functional
Gleb Bahmutov, PhD
VP of Engineering Cypress.io
Don't trust this guy
👉
Every time code crosses a border ...
Defensive coding: checking inputs before each computation (preconditions). Sometimes checking result value before returning (postconditions).
Paranoid coding: checking inputs before each computation, as if the caller was evil and trying to break the function on purpose.
Just crash and explain
const la = require('lazy-ass')
const is = require('check-more-types')
la(is.strings(names), 'expected list of names', names)
la(is.email(loginName), 'missing email')
la(is.version(tag) || check.sha(tag), 'invalid tag', tag)Just crash and explain
const la = require('lazy-ass')
const is = require('check-more-types')
la(is.strings(names), 'expected list of names', names)
la(is.email(loginName), 'missing email')
la(is.version(tag) || check.sha(tag), 'invalid tag', tag)predicate
Just crash and explain
const la = require('lazy-ass')
const is = require('check-more-types')
la(is.strings(names), 'expected list of names', names)
la(is.email(loginName), 'missing email')
la(is.version(tag) || check.sha(tag), 'invalid tag', tag)explanation
const la = require('lazy-ass')
const is = require('check-more-types')
la(is.strings(names), 'expected list of names', names)
la(is.email(loginName), 'missing email')
la(is.version(tag) || check.sha(tag), 'invalid tag', tag)explanation
🔑
STOP throwing Errors
START throwing Explanations
undefined is not a functionvs
What went wrong?
Where?
Why?
How do I fix this?
Real World
Cypress v3.0.0
API
Cypress 1.0.0
Cypress 1.1.0
Cypress 2.1.0
...
Dashboard web app
Cypress v3 test runner sending data to API
API
Cypress 1.0.0
Cypress 1.1.0
Cypress 2.1.0
Cypress 3.0.0
...
Dashboard web app
Completely different sequence of calls from the test runner Cypress v3
API
Cypress 1.0.0
Cypress 1.1.0
Cypress 2.1.0
Cypress 3.0.0
...
Dashboard web app
database
old and new
data
Mixture of old and new data in DB
API
Cypress 1.0.0
Cypress 1.1.0
Cypress 2.1.0
Cypress 3.0.0
...
Dashboard web app
database
old and new
data
What data is crossing these borders?
Ohh, it is in a wiki 😖
tip: manually updating wiki examples does not ⚖️
- API already exists
- Other domain objects
- Not invented here
Why not GraphQL?
- Not just the API
Why not TypeScript?
- Existing projects, code is crossing the boundary via JSON
{
"title": "PostTodoRequest",
"type": "object",
"properties": {
"text": {
"type": "string"
},
"done": {
"type": "boolean"
}
},
"required": [
"text",
"done"
],
"additionalProperties": false
}json-schema
+ descriptions
{
"title": "PostTodoRequest",
"type": "object",
"description": "Todo item sent by the client",
"properties": {
"text": {
"type": "string",
"description": "Todo text"
},
"done": {
"type": "boolean",
"description": "Is this todo item completed?"
}
},
"required": ["text", "done"],
"additionalProperties": false
}{
"version": {"major": 1, "minor": 0, "patch": 0},
"schema": {
"title": "PostTodoRequest",
"type": "object",
"description": "Todo item sent by the client",
"properties": {
"text": {
"type": "string",
"description": "Todo text"
},
"done": {
"type": "boolean",
"description": "Is this todo item completed?"
}
},
"required": ["text", "done"],
"additionalProperties": false
}
}{
"version": {"major": 1, "minor": 0, "patch": 0},
"schema": {
"title": "PostTodoRequest",
"type": "object",
"description": "Todo item sent by the client",
"properties": {
"text": {
"type": "string",
"description": "Todo text"
},
"done": {
"type": "boolean",
"description": "Is this todo item completed?"
}
},
"required": ["text", "done"],
"additionalProperties": false
},
"example": {"text": "write test", "done": false}
}import { assertSchema } from '@cypress/schema-tools'
import { schemas } from '../schemas'
const assertTodoRequest = assertSchema(schemas)
('postTodoRequest', '1.0.0')
const todo = {
text: 'use schemas',
done: true,
}
assertTodoRequest(todo)
// all goodvalid object
import { assertSchema } from '@cypress/schema-tools'
import { schemas } from '../schemas'
const assertTodoRequest = assertSchema(schemas)
('postTodoRequest', '1.0.0')
const todo = {
done: true,
}
assertTodoRequest(todo)
// Errorbad object
import { assertSchema } from '@cypress/schema-tools'
import { schemas } from '../schemas'
const assertTodoRequest = assertSchema(schemas)
('postTodoRequest', '1.0.0')
const todo = {
done: true,
}
assertTodoRequest(todo)
// Errorbad object
// Explanation
Schema postTodoRequest@1.0.0 violated
Errors:
data.text is required
Current object:
{
"done": true
}
Expected object like this:
{
"done": false,
"text": "do something"
}message in the thrown error
Schema postTodoRequest@1.0.0 violated
Errors:
data.text is required
Current object:
{
"done": true
}
Expected object like this:
{
"done": false,
"text": "do something"
}message in the thrown error
This for real world objects is 100% 💣
I'm in my happy place
= @cypress/schema-tools
some data is hard to deal with
const data = {
"done": false,
"id": 2,
"text": "do something",
"uuid": "3372137d-b582-4e32-807d-af3021112efa"
}
assertTodo(data) // ok
expect(data).toMatchSnapshot() // nope, dynamic valueconst data = {
"done": false,
"id": 2,
"text": "do something",
"uuid": "3372137d-b582-4e32-807d-af3021112efa"
}
assertTodo(data) // ok
expect(data).toMatchSnapshot({
uuid: expect.any(String)
}) // okconst data = {
"done": false,
"id": 2,
"text": "do something",
"uuid": "3372137d-b582-4e32-807d-af3021112efa"
}
assertTodo(data) // ok
expect(data).toMatchSnapshot({
uuid: expect.any(String)
}) // okThis information is already part of the schema!!!
const data = {
"done": false,
"id": 2,
"text": "do something",
"uuid": "3372137d-b582-4e32-807d-af3021112efa"
}
expect(
sanitizeTodo(assertTodo(data))
).toMatchSnapshot()exports = {
"done": false,
"id": 2,
"text": "do something",
"uuid": "ffffffff-ffff-ffff-ffff-ffffffffffff"
}saved snapshot
getBuilds(@user, { page: 2 })
.expect(200)
.expectResSchema("getRunResponse@1.0.0", {
snapshot: 'second page of builds'
})an object with 30 properties, uuids and timestamps sanitized
I am in my happy place and will never leave
JSON Schemas are Your True Testing Friend