TDD: Test-Driven Documentation

Gleb Bahmutov

Distinguished Engineer

Cypress.io

@bahmutov

https://slides.com/bahmutov/test-driven-documentation

our planet is in imminent danger

https://lizkeogh.com/2019/07/02/off-the-charts/

+3 degrees Celsius will be the end.

@bahmutov

survival is possible* but we need to act now

change your life
quit fossil fuel bank
join an organization

@bahmutov

rebellion.global 350.org

AGENDA

Horses, bicycles, and cars
Types of docs and common mistakes
Tested Markdown
Tests as docs for your apps

@bahmutov

source: https://blog.crisp.se/2016/01/25/henrikkniberg/making-sense-of-mvp

If all you know is

🏇

😲

😐

🤮

😳

🤔

State of the art "technology" - the horse

the bicycle

@bahmutov

// WHY DOESN'T THIS WORK!!?
const text = await cy.get('#inner').text
expect(text).to.equal('Submit')

cy.get('#inner')
  .should('have.text', 'Submit')

Why?!!! How is this better?!!!

cy.get('#inner')
  .should('have.text', 'Submit')
  .and('have.css', 'color', 'rgb(255, 0, 0)')
  .click()

Makes the tests human

Build a successful business at each step of the journey

👶

👦

😁

🤘📢

👩🧑🧔

Full set of training, documentation, materials

📚

A complex system that works is invariably found to have evolved from a simple system that worked. A complex system designed from scratch never works and cannot be patched up to make it work. You have to start over, beginning with a working simple system.

- John Gall

https://en.wikipedia.org/wiki/John_Gall_(author)#Gall's_law

A complex system that works is invariably found to have evolved from a simple well-documented system that worked.

"Gleb's Law"

👁

🥇

Fully lock yourself before attempting the next climb

Types of docs

Marketing copy
Guides
Reference
Tutorials and recipes
Example applications

Blog posts
Webinars
Conference talks
~~Comparisons to other tools~~

@bahmutov

Problems with Marketing

Selling the drills and not the holes

Fast, easy and reliable testing for anything that runs in a browser.

Cross-platform e2e test runner

Problems with Guides 1/2

Tell me why I should read it

npm install ...

🙁

Problems with Guides 2/2

Not stating the requirements clearly

This guide assumes you know X and have Y and Z

Open the app and ...

🤔

Problems with Reference Pages

Lack of examples
Examples out of date
Examples are all "foo" and "bar"
Non-uniform page structure
Missing history
Not linking to larger guides and tutorials

Problems with Reference Pages

Problems with Tutorials

Tutorials are out of date
Missing source code
Missing CI
Expectations not stated

This is how you drill a hole...

Problems with Blog posts

Missing the published date

When was this written?

Is this even relevant?

Problems with Blog posts

Missing version numbers

Problems with Blog posts

Missing version numbers

Problems with Webinars and Conference Talks

No links from the webinar to the rest of documentation
No links from the rest of the documentation to the webinar or talk

https://www.cypress.io/blog/2019/11/13/self-testing-jam-pages/

https://self-testing-jam.netlify.app/reactiveconf.html

Index of the talk with direct links to the individual sections
Scrapable and searchable
Can be linked from other pages

📛 Obstacles to good documentation

Private code or project
Diffuse responsibility
Hard to maintain

📛 Often missing from the docs

Repetition: the common themes should be repeated through the documentation

💡Tip: Let People Learn More

💡Tip: Teach Users to Search

https://slides.com/bahmutov/find-me-if-you-can https://slides.com/bahmutov/testing-docs

💡Tip: Teach Users to Search

Tests should be closer to their targets

@bahmutov

Keeping Examples Correct and Up-to-date

If I see a question on a forum, GitHub, Twitter, etc

How do I toggle a checkbox?

App HTML

Test code

Markdown file

https://github.com/bahmutov/cypress-markdown-preprocessor

Markdown is king

Markdown spec also becomes a static docs page

Every example is scraped into the docs index

💡 Do NOT answer support questions

💡💡💡 Especially for private support

@bahmutov

💡 Do NOT answer support questions

Instead, update the documentation, or create an example, or write a blog post.

https://glebbahmutov.com/blog/multiple-benefits/

Then answer with a link

@bahmutov

💡 The Support Team

Should eliminate their own jobs by writing more and more documentation until the users find everything themselves

@bahmutov

and double their salaries

Tutorials / recipes / example apps

Do not let dependencies fall behind
Communicate versions used

@bahmutov

https://glebbahmutov.com/blog/keep-examples-up-to-date/

{
  "extends": [
    "config:base"
  ],
  "automerge": true,
  "masterIssue": true
}

renovate.json

https://www.whitesourcesoftware.com/free-developer-tools/renovate

https://glebbahmutov.com/blog/keep-examples-up-to-date/

A pull request is opened automatically based on the schedule
If all tests pass, the PR is merged by RenovateBot

name: badges
# update README badge only if the README file changes
# or if the package.json file changes, or this file changes
on:
  push:
    branches:
      - master
    paths:
      - README.md
      - package.json
      - .github/workflows/badges.yml

jobs:
  build:
    name: Badges
    runs-on: ubuntu-latest
    steps:
      - name: Checkout 🛎
        uses: actions/checkout@v1

      - name: Update version badges 🏷
        run: |
          npx -p dependency-version-badge update-badge cypress cypress-react-unit-test

      # commit any changed files
      # https://github.com/mikeal/publish-to-github-action
      - name: Push any changes to repo 📤
        uses: mikeal/publish-to-github-action@master
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

.github/workflows/badges.yml