TDD: Test-Driven Documentation

Gleb Bahmutov

Distinguished Engineer

Cypress.io 

@bahmutov

our planet is in imminent danger

https://lizkeogh.com/2019/07/02/off-the-charts/

+3 degrees Celsius will be the end.

survival is possible* but we need to act now

  • change your life
  • quit fossil fuel bank
  • join an organization

rebellion.global          350.org

AGENDA

  1. Horses, bicycles, and cars

  2. Types of docs and common mistakes

  3. Tested Markdown

  4. Tests as docs for your apps

If all you know is

🏇

😲

😐

🤮

😳

🤔

State of the art "technology" - the horse

the bicycle

// WHY DOESN'T THIS WORK!!?
const text = await cy.get('#inner').text
expect(text).to.equal('Submit')
cy.get('#inner')
  .should('have.text', 'Submit')

Why?!!! How is this better?!!!

cy.get('#inner')
  .should('have.text', 'Submit')
  .and('have.css', 'color', 'rgb(255, 0, 0)')
  .click()

Makes the tests human

Build a successful business at each step of the journey

👶

👦

😁

🤘📢

👩🧑🧔

Full set of training, documentation, materials

📚

📚

📚

📚

📚

📚

📚

📚

📚

A complex system that works is invariably found to have evolved from a simple system that worked. A complex system designed from scratch never works and cannot be patched up to make it work. You have to start over, beginning with a working simple system.

- John Gall

A complex system that works is invariably found to have evolved from a simple well-documented system that worked.

"Gleb's Law"

👁

🥇

Fully lock yourself before attempting the next climb

Types of docs

  • Marketing copy

  • Guides

  • Reference

  • Tutorials and recipes

  • Example applications

  • Blog posts

  • Webinars

  • Conference talks

  • Comparisons to other tools

Problems with Marketing

Selling the drills and not the holes

Fast, easy and reliable testing for anything that runs in a browser.

Cross-platform e2e test runner

Problems with Guides 1/2

Tell me why I should read it

npm install ...

🙁

Problems with Guides 2/2

Not stating the requirements clearly

This guide assumes you know X and have Y and Z

Open the app and ... 

🤔

Problems with Reference Pages

Problems with Reference Pages

  • Lack of examples
  • Examples out of date
  • Examples are all "foo" and "bar"
  • Non-uniform page structure
  • Missing history
  • Not linking to larger guides and tutorials

Problems with Reference Pages

Problems with Tutorials

  • Tutorials are out of date
  • Missing source code
  • Missing CI
  • Expectations not stated

This is how you drill a hole...

Problems with Blog posts

Missing the published date

When was this written?

Is this even relevant?

Problems with Blog posts

Missing version numbers

Problems with Blog posts

Missing version numbers

Problems with Webinars and Conference Talks

  • No links from the webinar to the rest of documentation
  • No links from the rest of the documentation to the webinar or talk
  • Index of the talk with direct links to the individual sections
  • Scrapable and searchable
  • Can be linked from other pages

📛 Obstacles to good documentation

  • Private code or project

  • Diffuse responsibility

  • Hard to maintain

📛 Often missing from the docs

Repetition: the common themes should be repeated through the documentation

 💡Tip: Let People Learn More

 💡Tip: Let People Learn More

 💡Tip: Let People Learn More

 💡Tip: Teach Users to Search

 💡Tip: Teach Users to Search

Tests should be closer to their targets

Keeping Examples Correct and Up-to-date

If I see a question on a forum, GitHub, Twitter, etc

How do I toggle a checkbox?

App HTML

Test code

Markdown file

Markdown file

Markdown is king

Markdown spec also becomes a static docs page

Every example is scraped into the docs index

💡 Do NOT answer support questions

💡💡💡 Especially for private support

💡 Do NOT answer support questions

Instead, update the documentation, or create an example, or write a blog post.

Then answer with a link

💡 The Support Team

Should eliminate their own jobs by writing more and more documentation until the users find everything themselves

and double their salaries

Tutorials / recipes / example apps

  • Do not let dependencies fall behind
  • Communicate versions used
{
  "extends": [
    "config:base"
  ],
  "automerge": true,
  "masterIssue": true
}

renovate.json

  • A pull request is opened automatically based on the schedule
  • If all tests pass, the PR is merged by RenovateBot
name: badges
# update README badge only if the README file changes
# or if the package.json file changes, or this file changes
on:
  push:
    branches:
      - master
    paths:
      - README.md
      - package.json
      - .github/workflows/badges.yml

jobs:
  build:
    name: Badges
    runs-on: ubuntu-latest
    steps:
      - name: Checkout 🛎
        uses: actions/checkout@v1

      - name: Update version badges 🏷
        run: |
          npx -p dependency-version-badge update-badge cypress cypress-react-unit-test

      # commit any changed files
      # https://github.com/mikeal/publish-to-github-action
      - name: Push any changes to repo 📤
        uses: mikeal/publish-to-github-action@master
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

.github/workflows/badges.yml

💡 Use Example Apps to Test New Releases

"Testing" README.md

"Testing" README.md

  • Install and use your library / app as if you were a novice user blindly following the README instructions
  • Even better to have someone else do it for you

🖥 What About My Apps?

Create and embed app screenshots from tests into your documentation

🖥 What About My Apps?

Tests become the status of your project

🖥 Tests are Demos

Document what you have

Now

Make writing correct documentation easy

You are not there with the user

But your documentation is.

Gleb Bahmutov

@bahmutov

youtube.com/glebbahmutov

Thank you 👏

TDD: Test-Driven Documentation

By Gleb Bahmutov

TDD: Test-Driven Documentation

If you are working on the web application, and try to document it for the users, what do you do? Do you copy / paste screenshots into the docs? Does your documentation slowly diverge from the application? What if there was a better way? In this talk, I will show how to use the tests to document the web-application behavior, ensuring the docs are always correct. Presented at CityJS 2021

  • 1,254
Loading comments...

More from Gleb Bahmutov