Find Me If You Can

Algolia Search is Easy

JS Monthly Online #07

Gleb Bahmutov

VP of Engineering

Cypress.io

@bahmutov

gleb.dev

github.com/bahmutov

glebbahmutov.com/blog

https://slides.com/bahmutov/find-me-if-you-can

video at https://www.youtube.com/watch?v=z_RutLs0W70

our planet is in imminent danger

https://lizkeogh.com/2019/07/02/off-the-charts/

+3 degrees Celsius will be the end.

@bahmutov

survival is possible. but we need to act now

change your life
join an organization

@bahmutov

rebellion.global www.greenpeace.org.uk www.theclimatecoalition.org

Talk Contents

The funnel
The documentation
The search
- simple example
- complex documentation
Tips, Conclusions, Q & A

https://slides.com/bahmutov/find-me-if-you-can

The Funnel

aka "You are always losing customers"

@bahmutov

The Funnel

aka "You are always losing customers"

💵

💰

👀

🧑‍💻

🦸

@bahmutov

The Funnel

aka "You are always losing customers"

10-20%

@bahmutov

The Funnel

aka "You are always losing customers"

10-20%

@bahmutov

The Funnel

aka "You are always losing customers"

10-20%

@bahmutov

The Funnel

aka "You are always losing customers"

This tool does not do what I need

It does not work on my platform

Valid reasons

@bahmutov

The Funnel

aka "You are always losing customers"

This tool does not do what I need

It does not work on my platform

Valid reasons

This tool does not do what I need (but it does)

Does not work on my platform (but it does work)

Wrong reasons

@bahmutov

The Funnel

aka "You are always losing customers"

This tool does not run on my CI

Valid reasons

I don't know how to make this tool work on my CI

Wrong reasons

@bahmutov

The Funnel

aka "You are always losing customers"

I cannot afford this tool

Valid reasons

I don't know what value this tool gives me

Wrong reasons

The Funnel

aka "You are always losing customers"

I am happy with this tool, that's enough for me

Valid reasons

I did not know this tool could do that

Wrong reasons

The Funnel

aka "You are always losing customers"

Market fit

Features

Bug fixes

@bahmutov

The Funnel

aka "You are always losing customers"

Market fit

Features

Bug fixes

Bad UI

Bad docs

Lack of examples

@bahmutov

The Funnel

aka "You are always losing customers"

Market fit

Features

Bug fixes

Bad UI

Bad docs

Lack of examples

Goal: drive the "bad" area to zero

@bahmutov

The Funnel

aka "You are always losing customers"

Bad UI

Bad docs

Lack of examples

@bahmutov

Goal: drive the "bad" area to zero

Features vs Docs

Bad UI

Bad docs

Lack of examples

⚖️

Market fit

Features

Bug fixes

So
Much
Easier

@bahmutov

Improve the Docs: Personas

@bahmutov

🔑 Visitor / User should always find the right answer

First time visitor

What does this tool / service do?
Will it work for me?

@bahmutov

🔑 Visitor / User should always find the right answer

First time user

How do I do X?
How do I do Y?
What do I do for error A?

First time user

🔑 Visitor / User should always find the right answer

Repeat user

How do I do X and Y and Z?
What do I do for error A, B, C?
How do I run using ...?
How do I learn more?

Repeat user

🔑 Visitor / User should always find the right answer

Paying customer

same as repeat user +
Is there training?
I got stuck, is there help?

Paying customer

🔑 Visitor / User should always find the right answer

Promoter

When are the new releases?
What else can this tool do?
Can I suggest a feature?
Is there an email list or webinars?

Docs keep growing

Information for every user persona

@bahmutov

How to measure how good your documentation is ... and start crying

https://gitter.im/cypress-io/cypress

Is there a way to disable command snapshots?

How to preserve session cookie?

How to check the length of text?

How to measure how good your documentation is ... and start crying

https://gitter.im/cypress-io/cypress

Every question here is a failure of the documentation

Every question here is probably a:

a support issue 💵
maybe a lost customer 💸

Documentation miss percentage

https://gitter.im/cypress-io/cypress

all well-defined questions across chat, support emails, github issues

questions that have answer in your documentation

* 100%

Documentation miss percentage

< 50% write more docs!

≥ 50%

docs have a problem

(many things are not documented)

(users cannot find the right documentation)

You Want Users to Find Answers to Their Questions by Themselves

@bahmutov

The documentation demands for different user personas are contradictory

Why Good Docs Are Hard?

@bahmutov

First time visitor

First time user

Repeat user

Paying customer

Promoter

💥

"show me Hello, World!"

"show me the changelog diff from version X to Y"

"show me a tutorial"

"show me how to do X"

"how do I solve my issue or bug?"

👀 Optimize documentation structure for beginners

"Hello World" example front and center
There are 10 beginners for every 1 advanced user. Every user starts as a beginner

@bahmutov

👀 Optimize documentation structure for beginners

Then add all your documentation to the site reachable from the index

@bahmutov

💡 Make a powerful docs search for every persona

https://docs.cypress.io/

💡 Do NOT answer support questions

💡💡💡 Especially for private support

@bahmutov

💡 Do NOT answer support questions

Instead, update the documentation, or create an example, or write a blog post.

https://glebbahmutov.com/blog/multiple-benefits/

Then answer with a link

@bahmutov

💡 The Support Team

Should eliminate their own jobs by writing more and more documentation until the users find everything themselves

https://glebbahmutov.com/blog/multiple-benefits/

@bahmutov

Talk Contents

The funnel
The documentation
The search
- simple example
- complex documentation
Tips, Conclusions, Q & A

Sample App

https://glebbahmutov.com/blog/scrape-static-site-with-algolia/

https://glebbahmutov.com/triple-tested/

Why not use the built-in client-side indexing?

Hint: look at the search results for "about" query

Algolia

https://docsearch.algolia.com/

I am unaffiliated, just a user
Generous free plan
Good documentation, scraping tools, nice UI

@bahmutov

Algolia in action

@bahmutov

Create Algolia App

https://www.algolia.com/account/applications

@bahmutov

Create Search Index

https://www.algolia.com/account/applications

@bahmutov

Workflow

After every deploy 🔼
1. Scrape the site 🕷
Set search widget to point at the index using Algolia SDKs

@bahmutov

API Keys

to add records to the index

(keep private)

@bahmutov

API Keys

to search index from site

(public)

@bahmutov

Scraping the Site

@bahmutov

Document Structure

{
  "index_name": "scrape-test",
  "start_urls": ["https://glebbahmutov.com/triple-tested/"],
  "selectors": {
    "lvl0": {
      "selector": ".site-name",
      "global": true
    },
    "lvl1": ".content__default h1",
    "lvl2": ".content__default h2",
    "lvl3": ".content__default h3",
    "lvl4": ".content__default h4",
    "lvl5": ".content__default h5",
    "text": ".content__default p, .content__default li"
  }
}

Algolia config (JSON)

Scraping

# when scraping the site, inject secrets as environment variables
# then pass their values into the Docker container using "-e" syntax
# and inject config.json contents as another variable
- name: scrape the site 🧽
  env:
    APPLICATION_ID: ${{ secrets.APPLICATION_ID }}
    API_KEY: ${{ secrets.API_KEY }}
  run: |
    docker run \
    -e APPLICATION_ID -e API_KEY \
    -e CONFIG="$(cat config.json)" \
    algolia/docsearch-scraper:v1.6.0

use Algolia Docker image

https://github.com/bahmutov/scrape-test

@bahmutov

Scraping

# when scraping the site, inject secrets as environment variables
# then pass their values into the Docker container using "-e" syntax
# and inject config.json contents as another variable
- name: scrape the site 🧽
  env:
    APPLICATION_ID: ${{ secrets.APPLICATION_ID }}
    API_KEY: ${{ secrets.API_KEY }}
  run: |
    docker run \
    -e APPLICATION_ID -e API_KEY \
    -e CONFIG="$(cat config.json)" \
    algolia/docsearch-scraper:v1.6.0

use Algolia Docker image

https://github.com/bahmutov/scrape-test

@bahmutov

Searching Widget

module.exports = {
  title: 'Triple Tested',
  description: 'Example static site',
  base: '/triple-tested/',
  themeConfig: {
    algolia: {
      // DANGER 🧨💀: ONLY USE ALGOLIA PUBLIC SEARCH-ONLY API KEY
      apiKey: 'a2f5586dd739087cabead4b20ae0c2bc',
      indexName: 'scrape-test',
      appId: 'XVIJU4G89W'
    }
  }
}

https://github.com/bahmutov/triple-tested

using built-in VuePress Algolia Widget

@bahmutov

Markup and operation

@bahmutov

Larger sites: Multiple roots

blog

examples

Larger sites: Multiple roots

{
  "index_name": "cypress",
  "start_urls": [
    {
      "url": "https://docs.cypress.io/",
      "page_rank": 10
    },
    {
      "url": "https://example.cypress.io/",
      "selectors_key": "kitchensink",
      "tags": ["example"],
      "page_rank": 2
    },
    {
      "url": "https://www.cypress.io/blog/",
      "selectors_key": "blog",
      "tags": ["blog post"],
      "page_rank": 1
    }
  ],
  "stop_urls": [
    "^https://docs.cypress.io/ja/",
    "^https://docs.cypress.io/zh-cn/",
    "^https://docs.cypress.io/pt-br/",
    "^https://docs.cypress.io/ru/"
  ],
  "selectors_exclude": [],
  "selectors": {
    "default": {
      "lvl0": "article h1.article-title",
      "lvl1": "article h1.article-heading",
      "lvl2": "article h2.article-heading",
      "text": "article .article-content p, article .article-content tr, article .article-content li, article .article-content pre"
    },
    "blog": {
      "lvl0": "article h1",
      "lvl1": "article h2",
      "text": "article p, article pre"
    },
    "kitchensink": {
      "lvl0": ".container h1",
      "lvl1": ".container h4",
      "text": ".container p, .container pre"
    }
  },
  "nb_hits": 0,
  "min_indexed_level": 2
}

Algolia config (JSON)

Larger sites: Multiple roots

2 minutes to scrape, creates ~ 21k records

@bahmutov

Larger sites: Multiple roots

@bahmutov

Talk Contents

The funnel
The documentation
The search
- simple example
- complex documentation
Tips, Conclusions, Q & A

💡 Include code comments in the index

https://github.com/bahmutov/cypress-examples

@bahmutov

💡 Include code comments in the index

{
  "index_name": "cypress-examples",
  "start_urls": ["https://glebbahmutov.com/cypress-examples/"],
  "selectors": {
    "lvl0": {
      "selector": ".site-name",
      "global": true
    },
    "lvl1": ".content__default h1",
    "lvl2": ".content__default h2",
    "lvl3": ".content__default h3",
    "lvl4": ".content__default h4",
    "lvl5": ".content__default h5",
    "text": ".content__default p, .content__default li, .content__default pre .comment"
  }
}

https://github.com/bahmutov/cypress-examples

algolia-config.json

@bahmutov

💡 Include code comments in the index

{
  "index_name": "cypress-examples",
  "start_urls": ["https://glebbahmutov.com/cypress-examples/"],
  "selectors": {
    "lvl0": {
      "selector": ".site-name",
      "global": true
    },
    "lvl1": ".content__default h1",
    "lvl2": ".content__default h2",
    "lvl3": ".content__default h3",
    "lvl4": ".content__default h4",
    "lvl5": ".content__default h5",
    "text": ".content__default p, .content__default li, .content__default pre .comment"
  }
}

https://github.com/bahmutov/cypress-examples

algolia-config.json

💡 Look at searches with no results

⚠️ free Algolia plans only get weekly emails with such queries

write docs for these searches!

@bahmutov

💡 Search from CLI

https://github.com/bahmutov/cy-search

@bahmutov

Final Thoughts

Good documentation is essential to your project's success

@bahmutov

Final Thoughts

Creating an ideal documentation structure is hard and might be impossible

@bahmutov

Final Thoughts

Creating a good enough text search using Algolia is pretty simple 🎉

@bahmutov

Thank you & More Info

🔎 https://docs.cypress.io/ search in action
🔎 https://www.algolia.com/doc/

🌍 https://rebellion.global/

🦉 @bahmutov

Find Me If You Can

JS Monthly Online #07

Gleb Bahmutov

VP of Engineering

Cypress.io

@bahmutov

our planet is in imminent danger

survival is possible. but we need to act now

Talk Contents

The funnel

The documentation

The search

simple example

complex documentation

Tips, Conclusions, Q & A

The Funnel

The Funnel

The Funnel

The Funnel

The Funnel

The Funnel

The Funnel

The Funnel

The Funnel

The Funnel

The Funnel

The Funnel

The Funnel

The Funnel

Features vs Docs

⚖️

Improve the Docs: Personas

🔑 Visitor / User should always find the right answer

First time visitor

What does this tool / service do?

Will it work for me?

🔑 Visitor / User should always find the right answer

First time user

How do I do X?

How do I do Y?

What do I do for error A?

First time user

🔑 Visitor / User should always find the right answer

Repeat user

How do I do X and Y and Z?

What do I do for error A, B, C?

How do I run using ...?

How do I learn more?

Repeat user

🔑 Visitor / User should always find the right answer

Paying customer

same as repeat user +

Is there training?

I got stuck, is there help?

Paying customer

🔑 Visitor / User should always find the right answer

Promoter

When are the new releases?

What else can this tool do?

Can I suggest a feature?

Is there an email list or webinars?

Docs keep growing

How to measure how good your documentation is ... and start crying

How to measure how good your documentation is ... and start crying

Documentation miss percentage

Documentation miss percentage

< 50% write more docs!

≥ 50%

docs have a problem

You Want Users to Find Answers to Their Questions by Themselves

The documentation demands for different user personas are contradictory

Why Good Docs Are Hard?

First time visitor

First time user

Repeat user

Paying customer

Promoter

👀 Optimize documentation structure for beginners

👀 Optimize documentation structure for beginners

💡 Make a powerful docs search for every persona