Gleb Bahmutov PRO
JavaScript ninja, image processing expert, software quality fanatic
Gleb Bahmutov, PhD
VP of Engineering, Cypress.io
JSON Schemas Update
This.JavaScript - State of Testing
About me
C / C++ / C# / Java / CoffeeScript / JavaScript / Node / Angular / Vue / Cycle.js / functional
these slides
18 people. Atlanta, Philly, Boston, Chicago, NYC
Fast, easy and reliable testing for anything that runs in a browser
API
Cypress 1.0.0
Cypress 1.1.0
Cypress 2.1.0
...
Dashboard web app
Cypress v1,v2 test runners sending data to API
Let's run Cypress tests in parallel!
API
Cypress 1.0.0
Cypress 1.1.0
Cypress 2.1.0
Cypress 3.0.0
...
Dashboard web app
Completely different sequence of calls from the test runner Cypress v3
API
Cypress 1.0.0
Cypress 1.1.0
Cypress 2.1.0
Cypress 3.0.0
...
Dashboard web app
database
old and new
data
Mixture of old and new data in DB
API
Cypress 1.0.0
Cypress 1.1.0
Cypress 2.1.0
Cypress 3.0.0
...
Dashboard web app
database
old and new
data
What data is crossing these borders?
Ohh, it is in a wiki 😖
tip: manually updating wiki examples does not ⚖️
Every time data crosses a border ...
Let's design a good solution for validating complex objects
{
"title": "PostTodoRequest",
"type": "object",
"properties": {
"text": {
"type": "string"
},
"done": {
"type": "boolean"
}
},
"required": [
"text",
"done"
],
"additionalProperties": false
}json-schema
+ descriptions
{
"title": "PostTodoRequest",
"type": "object",
"description": "Todo item sent by the client",
"properties": {
"text": {
"type": "string",
"description": "Todo text"
},
"done": {
"type": "boolean",
"description": "Is this todo item completed?"
}
},
"required": ["text", "done"],
"additionalProperties": false
}{
"version": {"major": 1, "minor": 0, "patch": 0},
"schema": {
"title": "PostTodoRequest",
"type": "object",
"description": "Todo item sent by the client",
"properties": {
"text": {
"type": "string",
"description": "Todo text"
},
"done": {
"type": "boolean",
"description": "Is this todo item completed?"
}
},
"required": ["text", "done"],
"additionalProperties": false
}
}{
"version": {"major": 1, "minor": 0, "patch": 0},
"schema": {
"title": "PostTodoRequest",
"type": "object",
"description": "Todo item sent by the client",
"properties": {
"text": {
"type": "string",
"description": "Todo text"
},
"done": {
"type": "boolean",
"description": "Is this todo item completed?"
}
},
"required": ["text", "done"],
"additionalProperties": false
},
"example": {"text": "write test", "done": false}
}import { assertSchema } from '@cypress/schema-tools'
import { schemas } from '../schemas'
const assertTodoRequest = assertSchema(schemas)
('postTodoRequest', '1.0.0')helper curried assert function
import { assertSchema } from '@cypress/schema-tools'
import { schemas } from '../schemas'
const assertTodoRequest = assertSchema(schemas)
('postTodoRequest', '1.0.0')
const todo = {
text: 'use schemas',
done: true,
}
assertTodoRequest(todo)
// all goodvalid object
import { assertSchema } from '@cypress/schema-tools'
import { schemas } from '../schemas'
const assertTodoRequest = assertSchema(schemas)
('postTodoRequest', '1.0.0')
const todo = {
done: true,
}
assertTodoRequest(todo)
// Errorinvalid object
import { assertSchema } from '@cypress/schema-tools'
import { schemas } from '../schemas'
const assertTodoRequest = assertSchema(schemas)
('postTodoRequest', '1.0.0')
const todo = {
done: true,
}
assertTodoRequest(todo)
// Error// Explanation
invalid object
Schema postTodoRequest@1.0.0 violated
Errors:
data.text is required
Current object:
{
"done": true
}
Expected object like this:
{
"done": false,
"text": "do something"
}message in the thrown error
Schema postTodoRequest@1.0.0 violated
Errors:
data.text is required
Current object:
{
"done": true
}
Expected object like this:
{
"done": false,
"text": "do something"
}message in the thrown error
What went wrong?
Schema postTodoRequest@1.0.0 violated
Errors:
data.text is required
Current object:
{
"done": true
}
Expected object like this:
{
"done": false,
"text": "do something"
}message in the thrown error
Why?
Schema postTodoRequest@1.0.0 violated
Errors:
data.text is required
Current object:
{
"done": true
}
Expected object like this:
{
"done": false,
"text": "do something"
}message in the thrown error
How do I fix this?
Schema postTodoRequest@1.0.0 violated
Errors:
data.text is required
Current object:
{
"done": true
}
Expected object like this:
{
"done": false,
"text": "do something"
}message in the thrown error
This for real world objects is 100% 💣
= @cypress/schema-tools
-
Make schemas
-
Publish as NPM package
-
Use from API and from any client
no more manual Wiki edits
API
Cypress 1.0.0
Cypress 1.1.0
Cypress 2.1.0
Cypress 3.0.0
...
Dashboard web app
database
old and new
data
Check data crossing these borders
req: PostXRequest@1.0.0
res: PostXResponse@2.1.0
JSON Schemas in
End-to-end tests
-
Validate data fixtures
-
Check network requests
JSON Schemas in
End-to-end tests
it('has todo fixture matching schema', () => {
// is our fixture file correct?
cy.fixture('todo')
.then(api.assertSchema('PostTodoRequest', '1.0.0'))
})
it('returns new item matching schema', () => {
cy.server()
cy.route('POST', '/todos').as('post')
cy.visit('/')
cy.get('.new-todo').type('Use schemas{enter}')
cy.wait('@post')
.its('response.body')
.then(api.assertSchema('PostTodoResponse', '1.0.0'))
})JSON Schemas in
End-to-end tests: GUI + Network
some data is hard to deal with
Dynamic data
const data = {
"done": false,
"id": 2,
"text": "do something",
"uuid": "3372137d-b582-4e32-807d-af3021112efa"
}
assertTodo(data) // ok
expect(data).toMatchSnapshot() // nope, dynamic valueJest test
Cannot snapshot random UUID 😕
❌
const data = {
"done": false,
"id": 2,
"text": "do something",
"uuid": "3372137d-b582-4e32-807d-af3021112efa"
}
assertTodo(data) // ok
expect(data).toMatchSnapshot({
uuid: expect.any(String)
}) // okJest test
const data = {
"done": false,
"id": 2,
"text": "do something",
"uuid": "3372137d-b582-4e32-807d-af3021112efa"
}
assertTodo(data) // ok
expect(data).toMatchSnapshot({
uuid: expect.any(String)
}) // okThis information is already part of the schema!!!
Jest test
import { CustomFormat, CustomFormats }
from '@cypress/schema-tools'
const uuid: CustomFormat = {
name: 'uuid', // the name
description: 'GUID used through the system',
detect: /^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/,
// (optional) replace actual value with this default value
// when using to sanitize an object
defaultValue: 'ffffffff-ffff-ffff-ffff-ffffffffffff',
}describe custom formats in json-schema,
give it default value
schema: {
type: 'object',
title: 'Employee',
properties: {
id: {
type: 'string',
format: 'uuid',
},
},
},
example: {
id: 'a368dbfd-08e4-4698-b9a3-b2b660a11835',
}const data = {
"done": false,
"id": 2,
"text": "do something",
"uuid": "3372137d-b582-4e32-807d-af3021112efa"
}
expect(
sanitizeTodo(assertTodo(data))
).toMatchSnapshot()exports = {
"done": false,
"id": 2,
"text": "do something",
"uuid": "ffffffff-ffff-ffff-ffff-ffffffffffff"
}saved snapshot
I'm in my happy place
JSON schemas
A portable, serializable way to describe properties of objects
JSON schemas
Useful to document and validate objects flowing through the system
Cypress REST API v2 ➡ v3 went without errors. And we still support v1 and v2 clients
JSON schemas
Can produce beautiful and helpful error messages
More information
Cypress GraphQL API
Declarative, Code-First GraphQL Schemas for JavaScript/TypeScript
Tim Griesser @tgriesser, Software Developer at Cypress.io
Gleb Bahmutov, PhD
VP of Engineering
Thank you
github.com/cypress-io/cypress ⭐️
JSON Schemas: State of Testing
By Gleb Bahmutov
JSON Schemas: State of Testing
JSON schemas is a portable declarative way to describe the shape of an object. In end-to-end testing, we can use these schemas to validate data fixtures, confirm server responses and sanitize data flowing through the application and tests for simpler assertions. In blog post form at https://www.cypress.io/blog/2018/07/10/json-schemas-are-your-true-testing-friend/
