Everything I know about writing

quality software

Gleb Bahmutov, PhD

C / C++ / C# / Java / CoffeeScript / JavaScript / Node / Angular / Vue / Cycle.js / functional

JavaScript ninja, image processing expert, software quality fanatic, Microsoft MVP

tip: reload page to get new funny definition of "NPM"

11 people. Atlanta, Philly, Boston, LA

Fast, easy and reliable testing for anything that runs in a browser

want to work with me? Ok, not with me, but at Cypress? https://www.cypress.io/jobs

Why is all software broken?

Will Klein

Quality software behaves the way users expect it to behave

We going to need some tests

E2E

integration

unit

Smallest pieces

Testing Pyramid β–³

Unit tests pass...

E2E

integration

unit

Component

(enzyme with full rendering)

E2E

integration

unit

Website / API

E2E

integration

unit

Really important to users

Really important to developers

function add (a, b) {
  return a + b
}
function add (a, b) {
  return a + b
}
if (!module.parent) {
  console.log(add(2, 3))
}
// node ./add
// 5
function add (a, b) {
  return a + b
}
module.exports = add
if (!module.parent) {
  console.log(add(2, 3))
}
// node ./add
// 5
test('2 + 3', () => {
  const add = require('./add')
  expect(add(1, 2)).toBe(3)
})

test.js

$ npm t

> add@1.0.0 test /add
> jest

 PASS  ./test.js
  βœ“ 2 + 3 (35ms)

Test Suites: 1 passed, 1 total
Tests:       1 passed, 1 total
Snapshots:   0 total
Time:        0.96s
Ran all test suites.
$ npm t -- --coverage

> add@1.0.0 test /Users/gleb/git/training/javascript/add
> jest "--coverage"

 PASS  ./test.js
  βœ“ 2 + 3 (341ms)

----------|----------|----------|----------|----------|-------------------|
File      |  % Stmts | % Branch |  % Funcs |  % Lines | Uncovered Line #s |
----------|----------|----------|----------|----------|-------------------|
All files |       75 |       50 |      100 |       75 |                   |
 add.js   |       75 |       50 |      100 |       75 |                 6 |
----------|----------|----------|----------|----------|-------------------|
Test Suites: 1 passed, 1 total
Tests:       1 passed, 1 total
Snapshots:   0 total
Time:        1.167s
Ran all test suites.
open coverage/lcov-report/index.html

100% code coverage

require = require('really-need')
test('2 + 3', () => {
  const add = require('./add', {
    parent: undefined
  })
  expect(add(1, 2)).toBe(3)
})

Trick a module to think it has no parent

require = require('really-need')
test('2 + 3', () => {
  const add = require('./add', {
    parent: undefined
  })
  expect(add(1, 2)).toBe(3)
})

Trick a module to think it has no parent

require = require('really-need')
test('2 + 3', () => {
  const add = require('./add', {
    parent: undefined
  })
  expect(add(1, 2)).toBe(3)
})

Trick a module to think it has no parent

works in Node < v4.2.2 πŸ›‘

const exec = require('execa-wrap')
test('2 + 3', () => {
  return exec('node', ['./add'])
  .then(result => {
    expect(result).toContain('5')
  })
})
command: node ./add
code: 0
failed: false
killed: false
signal: null
timedOut: false

stdout:
-------
5
-------
stderr:
-------

-------

βœ…

E2E

integration

unit

E2E

unit

expect(add(1, 2)).toBe(3)
exec('node', ['./add'])
.then(result => {
  expect(result).toContain('5')
})

E2E

integration

unit

E2E

unit

expect(add(1, 2)).toBe(3)
exec('node', ['./add'])
.then(result => {
  expect(result).toContain('5')
})

developer view

user view

Test the software the way

the user would use it

$ npm t -- --coverage

> add@1.0.0 test /Users/gleb/git/training/javascript/add
> jest "--coverage"

 PASS  ./test.js
  βœ“ 2 + 3 (77ms)

5
----------|----------|----------|----------|----------|-------------------|
File      |  % Stmts | % Branch |  % Funcs |  % Lines | Uncovered Line #s |
----------|----------|----------|----------|----------|-------------------|
All files |        0 |        0 |        0 |        0 |                   |
----------|----------|----------|----------|----------|-------------------|
Test Suites: 1 passed, 1 total
Tests:       1 passed, 1 total
Snapshots:   0 total
Time:        1.323s
Ran all test suites.

πŸ˜•

// change test to
return exec('nyc', 
  ['--reporter=lcov', 'node', './add'])
// npm i -D nyc

test.js

Coverage is hard

Code coverage

is tricky

const isEmail = (s) =>
  /^\w+@\w+\.\w{3,4}$/.test(s)

// 1 test = 100% code coverage
​(?:[a-z0-9!#$%&'*+/=?^_`{|}~-]+(?:\.[a-z0-9!#$%&'*+/=?^_`{|}~-]+)*|"(?:[\x01-\x08\x0b\x0c\x0e-\x1f\x21\x23-\x5b\x5d-\x7f]|\\[\x01-\x09\x0b\x0c\x0e-\x7f])*")@(?:(?:[a-z0-9](?:[a-z0-9-]*[a-z0-9])?\.)+[a-z0-9](?:[a-z0-9-]*[a-z0-9])?|\[(?:(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.){3}(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?|[a-z0-9-]*[a-z0-9]:(?:[\x01-\x08\x0b\x0c\x0e-\x1f\x21-\x5a\x53-\x7f]|\\[\x01-\x09\x0b\x0c\x0e-\x7f])+)\])

Code coverage

vs

Data coverage

mocha -r data-cover spec.js

Direct your code coverage effort by complexity

Let's test a web app the way a user would use it

$ npm install -D cypress
it('opens the page', () => {
  cy.visit('http://localhost:3000')
  cy.get('.new-todo')
    .should('be.visible')
})
it('adds 2 todos', () => {
  cy.visit('http://localhost:3000')
  cy.get('.new-todo')
    .type('learn testing{enter}')
    .type('be cool{enter}')
  cy.get('.todo-list li')
    .should('have.length', 2)
})

Cypress demo

  • Opening a page
  • Typical test
  • Failing test
  • API and documentation
  • Recording tests
  • CI setup

To learn more:

Egghead.io Cypress Course

Free course (same Egghead.io author!)

Cypress documentation

You want quality?

You better be a good writer!

Crashes will happen

image source: http://ktla.com/2016/04/02/small-plane-crashes-into-suv-on-15-freeway-in-san-diego-county/

undefined is not a function

Sentry.io crash service

if (process.env.NODE_ENV === 'production') {
    var raven = require('raven');
    var SENTRY_DSN = 'https://<DSN>@app.getsentry.com/...';
    var client = new raven.Client(SENTRY_DSN);
    client.patchGlobal();
}
foo.bar // this Error will be reported
npm install raven --save

Defensive coding: checking inputs before each computation (preconditions). Sometimes checking result value before returning (postconditions).

Crash early and often

Paranoid coding: checking inputs before each computation, as if the caller was evil and trying to break the function on purpose.

Crash early and often

Crash early and often

const la = require('lazy-ass')
const is = require('check-more-types')
la(is.strings(names), 'expected list of names', names)
la(is.email(loginName), 'missing email')
la(is.version(tag) || check.sha(tag), 'invalid tag', tag)

unit / E2E tests for happy path

error handling

tests based on crashes

50%

20%

30%

Writing tests

User stories?!

Tips for less testing

TypeScript

TypeScript

Try to connect separate pieces of software together. ...

This is also why I don't use type system like Typescript - it does not let me hack things together!

I was wrong

TypeScript

Go ahead, try it

Tips for less testing

Functional programming

Ramda / Lodash-fp

function getAddress() {
  // "address" is nested inside the result object
  return db.fetch(...)
    .then(R.path(['customer', 'address']))
    .then(Maybe.fromNullable)
    .then(R.map(objectToCamelCase))
}

Tips for less testing

json-schemas

{
  "title": "Todo",
  "type": "object",
  "properties": {
     text: {
       type: 'string',
       description: 'Todo text, like "clean room"',
     },
     done: {
       type: 'boolean',
       description: 'Is this todo item completed?',
     }
  },
  "required": ["text", "done"]
}

json-schema

const Todo100: ObjectSchema = {
  version: '1.0.0',
  schema: {
    title: 'Todo', type: 'object',
    description: 'Todo item sent by the client',
    properties: {
      text: {
        type: 'string',
        description: 'Todo text, like "clean room"',
      },
      done: {
        type: 'boolean',
        description: 'Is this todo item completed?',
      },
    },
    required: true, additionalProperties: false,
  },
  example: {
    text: 'do something',
    done: false
  }
}

versioned schema

const {assertSchema} = require('@cypress/schema-tools')
const assertTodoRequest = 
  assertSchema(schemas)('Todo', '1.0.0')
assertTodoRequest({
  done: true
})
  • create
  • document
  • validate
  • sanitize
Schema Todo@1.0.0 violated

Errors:
data.text is required

Current object:
{
  done: true
}

Expected object like this:
{
  done: false,
  text: "do something"
}

really verbose error message

Please, invest time in error messages!

api.assertSchema('Todo', '1.0.0')(response.body)
expect(
  api.sanitize('Todo', '1.0.0)(response.body)
).toMatchSnapshot()

Sanitize dynamic data based on schema

Follow https://www.cypress.io/blogΒ for blog posts

βœ… Unit test your code (choices!)

πŸ”₯ Crash service is a compass

🌲 Cypress does E2E testing

βœ‚οΈ Write fewer tests

Software

Tools

> Software

People

> Tools

> Software

πŸ‘©β€πŸ’»πŸ‘¨β€πŸ’» Trust your team mates

πŸ“š Learn and teach

🐝 Attend BuzzJS βš‘οΈπŸ”‹

@bahmutov

cypress.io

Everything I know about writing quality software

By Gleb Bahmutov

Everything I know about writing quality software

In this presentation I will show all the tools I use to write software that does not break but keeps the users happy. Static types, exception monitoring, unit tests with snapshots, randomized testing, code and data coverage, code complexity metrics and awesome end to end testing tools - the list is long and keeps on growing! The techniques I plan to show are applicable to every framework and environment. Presented at BuzzJS 2018, video at https://youtu.be/1PMxLTfh6lo?t=1

  • 4,866