Find Me If You Can

Algolia Search is Easy

JS Monthly Online #07

Gleb Bahmutov

VP of Engineering

Cypress.io 

@bahmutov

gleb.dev

github.com/bahmutov

glebbahmutov.com/blog

https://slides.com/bahmutov/find-me-if-you-can

video at https://www.youtube.com/watch?v=z_RutLs0W70

our planet is in imminent danger

https://lizkeogh.com/2019/07/02/off-the-charts/

+3 degrees Celsius will be the end.

@bahmutov

survival is possible. but we need to act now

  • change your life
  • join an organization

@bahmutov

rebellion.global   www.greenpeace.org.uk   www.theclimatecoalition.org

Talk Contents

  • The funnel 

  • The documentation

  • The search

    • simple example

    • complex documentation

  • Tips, Conclusions, Q & A

https://slides.com/bahmutov/find-me-if-you-can

The Funnel

aka "You are always losing customers"

funnel of visitors to paying customers

@bahmutov

The Funnel

aka "You are always losing customers"

funnel of visitors to paying customers

💵

💰

👀

🧑‍💻

🦸

@bahmutov

The Funnel

aka "You are always losing customers"

funnel of visitors to paying customers

10-20%

@bahmutov

The Funnel

aka "You are always losing customers"

funnel of visitors to paying customers

10-20%

What if you converted all first time visitors to paying customers?

@bahmutov

The Funnel

aka "You are always losing customers"

funnel of visitors to paying customers

10-20%

What if you converted all first time visitors to paying customers?

You would increase your revenue 5 or 10 times!

@bahmutov

The Funnel

aka "You are always losing customers"

funnel of visitors to paying customers
Why the drop outs?

This tool does not do what I need

It does not work on my platform

Valid reasons

@bahmutov

The Funnel

aka "You are always losing customers"

funnel of visitors to paying customers
Why the drop outs?

This tool does not do what I need

It does not work on my platform

Valid reasons

This tool does not do what I need (but it does)

Does not work on my platform (but it does work)

Wrong reasons

@bahmutov

The Funnel

aka "You are always losing customers"

funnel of visitors to paying customers
Why the drop outs?

This tool does not run on my CI

Valid reasons

I don't know how to make this tool work on my CI

Wrong reasons

@bahmutov

The Funnel

aka "You are always losing customers"

funnel of visitors to paying customers
Why the drop outs?

I cannot afford this tool

Valid reasons

I don't know what value this tool gives me

Wrong reasons

The Funnel

aka "You are always losing customers"

funnel of visitors to paying customers
Why the drop outs?

I am happy with this tool, that's enough for me

Valid reasons

I did not know this tool could do that

Wrong reasons

The Funnel

aka "You are always losing customers"

funnel of visitors to paying customers
Why the drop outs?

Market fit

Features

Bug fixes

@bahmutov

The Funnel

aka "You are always losing customers"

funnel of visitors to paying customers
Why the drop outs?

Market fit

Features

Bug fixes

Bad UI

Bad docs

Lack of examples

@bahmutov

The Funnel

aka "You are always losing customers"

funnel of visitors to paying customers
Why the drop outs?

Market fit

Features

Bug fixes

Bad UI

Bad docs

Lack of examples

Goal: drive the "bad" area to zero

@bahmutov

The Funnel

aka "You are always losing customers"

funnel of visitors to paying customers

If you stop losing users in red area, you would increase your revenue 2 or 5 times!

Bad UI

Bad docs

Lack of examples

@bahmutov

Goal: drive the "bad" area to zero

Features vs Docs

Bad UI

Bad docs

Lack of examples

⚖️

Market fit

Features

Bug fixes

  • So
  • Much
  • Easier

@bahmutov

Improve the Docs: Personas

@bahmutov

🔑 Visitor / User should always find the right answer

First time visitor

  • What does this tool / service do?

  • Will it work for me?

@bahmutov

🔑 Visitor / User should always find the right answer

First time user

  • How do I do X?

  • How do I do Y?

  • What do I do for error A?

First time user

🔑 Visitor / User should always find the right answer

Repeat user

  • How do I do X and Y and Z?

  • What do I do for error A, B, C?

  • How do I run using ...?

  • How do I learn more?

Repeat user

🔑 Visitor / User should always find the right answer

Paying customer

  • same as repeat user +

  • Is there training?

  • I got stuck, is there help?

Paying customer

🔑 Visitor / User should always find the right answer

Promoter

  • When are the new releases?

  • What else can this tool do?

  • Can I suggest a feature?

  • Is there an email list or webinars?

Docs keep growing

Information for every user persona

@bahmutov

How to measure how good your documentation is ... and start crying

https://gitter.im/cypress-io/cypress

Is there a way to disable command snapshots?

How to preserve session cookie?

How to check the length of text?

How to measure how good your documentation is ... and start crying

https://gitter.im/cypress-io/cypress

Every question here is a failure of the documentation

Every question here is probably a:

  1. a support issue 💵
  2. maybe a lost customer 💸

Documentation miss percentage

https://gitter.im/cypress-io/cypress

all well-defined questions across chat, support emails, github issues
questions that have answer in your documentation
* 100%

Documentation miss percentage

< 50%       write more docs!

≥ 50%

docs have a problem

(many things are not documented)

(users cannot find the right documentation)

You Want Users to Find Answers to Their Questions by Themselves

@bahmutov

The documentation demands for different user personas are contradictory

Why Good Docs Are Hard?

@bahmutov

First time visitor

First time user

Repeat user

Paying customer

Promoter

💥

"show me Hello, World!"

"show me the changelog diff from version X to Y"

"show me a tutorial"

"show me how to do X"

"how do I solve my issue or bug?"

👀 Optimize documentation structure for beginners

  • "Hello World" example front and center
  • There are 10 beginners for every 1 advanced user. Every user starts as a beginner

@bahmutov

👀 Optimize documentation structure for beginners

Then add all your documentation to the site reachable from the index

@bahmutov

💡 Make a powerful docs search for every persona

https://docs.cypress.io/

💡 Do NOT answer support questions

💡💡💡 Especially for private support

@bahmutov

💡 Do NOT answer support questions

Instead, update the documentation, or create an example, or write a blog post.

https://glebbahmutov.com/blog/multiple-benefits/

Then answer with a link

@bahmutov

💡 The Support Team

Should eliminate their own jobs by writing more and more documentation until the users find everything themselves

https://glebbahmutov.com/blog/multiple-benefits/

@bahmutov

Talk Contents

  • The funnel

  • The documentation

  • The search

    • simple example

    • complex documentation

  • Tips, Conclusions, Q & A

Sample App

https://glebbahmutov.com/blog/scrape-static-site-with-algolia/

https://glebbahmutov.com/triple-tested/

Why not use the built-in client-side indexing?

Hint: look at the search results for "about" query

Algolia

https://docsearch.algolia.com/

  • I am unaffiliated, just a user

  • Generous free plan

  • Good documentation, scraping tools, nice UI

@bahmutov

Algolia in action

@bahmutov

Create Algolia App

https://www.algolia.com/account/applications

@bahmutov

Create Search Index

https://www.algolia.com/account/applications

@bahmutov

Workflow

  1. After every deploy 🔼
    1. Scrape the site 🕷
  2. Set search widget to point at the index using Algolia SDKs

@bahmutov

API Keys

to add records to the index

(keep private)

@bahmutov

API Keys

to search index from site

(public)

@bahmutov

Scraping the Site

@bahmutov

Document Structure

{
  "index_name": "scrape-test",
  "start_urls": ["https://glebbahmutov.com/triple-tested/"],
  "selectors": {
    "lvl0": {
      "selector": ".site-name",
      "global": true
    },
    "lvl1": ".content__default h1",
    "lvl2": ".content__default h2",
    "lvl3": ".content__default h3",
    "lvl4": ".content__default h4",
    "lvl5": ".content__default h5",
    "text": ".content__default p, .content__default li"
  }
}

Algolia config (JSON)

Scraping

# when scraping the site, inject secrets as environment variables
# then pass their values into the Docker container using "-e" syntax
# and inject config.json contents as another variable
- name: scrape the site 🧽
  env:
    APPLICATION_ID: ${{ secrets.APPLICATION_ID }}
    API_KEY: ${{ secrets.API_KEY }}
  run: |
    docker run \
    -e APPLICATION_ID -e API_KEY \
    -e CONFIG="$(cat config.json)" \
    algolia/docsearch-scraper:v1.6.0

use Algolia Docker image

https://github.com/bahmutov/scrape-test

@bahmutov

Scraping

# when scraping the site, inject secrets as environment variables
# then pass their values into the Docker container using "-e" syntax
# and inject config.json contents as another variable
- name: scrape the site 🧽
  env:
    APPLICATION_ID: ${{ secrets.APPLICATION_ID }}
    API_KEY: ${{ secrets.API_KEY }}
  run: |
    docker run \
    -e APPLICATION_ID -e API_KEY \
    -e CONFIG="$(cat config.json)" \
    algolia/docsearch-scraper:v1.6.0

use Algolia Docker image

https://github.com/bahmutov/scrape-test

@bahmutov

Searching Widget

module.exports = {
  title: 'Triple Tested',
  description: 'Example static site',
  base: '/triple-tested/',
  themeConfig: {
    algolia: {
      // DANGER 🧨💀: ONLY USE ALGOLIA PUBLIC SEARCH-ONLY API KEY
      apiKey: 'a2f5586dd739087cabead4b20ae0c2bc',
      indexName: 'scrape-test',
      appId: 'XVIJU4G89W'
    }
  }
}

https://github.com/bahmutov/triple-tested

using built-in VuePress Algolia Widget

@bahmutov

Markup and operation

@bahmutov

Larger sites: Multiple roots

blog

examples

Larger sites: Multiple roots

{
  "index_name": "cypress",
  "start_urls": [
    {
      "url": "https://docs.cypress.io/",
      "page_rank": 10
    },
    {
      "url": "https://example.cypress.io/",
      "selectors_key": "kitchensink",
      "tags": ["example"],
      "page_rank": 2
    },
    {
      "url": "https://www.cypress.io/blog/",
      "selectors_key": "blog",
      "tags": ["blog post"],
      "page_rank": 1
    }
  ],
  "stop_urls": [
    "^https://docs.cypress.io/ja/",
    "^https://docs.cypress.io/zh-cn/",
    "^https://docs.cypress.io/pt-br/",
    "^https://docs.cypress.io/ru/"
  ],
  "selectors_exclude": [],
  "selectors": {
    "default": {
      "lvl0": "article h1.article-title",
      "lvl1": "article h1.article-heading",
      "lvl2": "article h2.article-heading",
      "text": "article .article-content p, article .article-content tr, article .article-content li, article .article-content pre"
    },
    "blog": {
      "lvl0": "article h1",
      "lvl1": "article h2",
      "text": "article p, article pre"
    },
    "kitchensink": {
      "lvl0": ".container h1",
      "lvl1": ".container h4",
      "text": ".container p, .container pre"
    }
  },
  "nb_hits": 0,
  "min_indexed_level": 2
}

Algolia config (JSON)

Larger sites: Multiple roots

2 minutes to scrape, creates ~ 21k records

@bahmutov

Larger sites: Multiple roots

@bahmutov

Talk Contents

  • The funnel

  • The documentation

  • The search

    • simple example

    • complex documentation

  • Tips, Conclusions, Q & A

💡 Include code comments in the index

https://github.com/bahmutov/cypress-examples

@bahmutov

💡 Include code comments in the index

{
  "index_name": "cypress-examples",
  "start_urls": ["https://glebbahmutov.com/cypress-examples/"],
  "selectors": {
    "lvl0": {
      "selector": ".site-name",
      "global": true
    },
    "lvl1": ".content__default h1",
    "lvl2": ".content__default h2",
    "lvl3": ".content__default h3",
    "lvl4": ".content__default h4",
    "lvl5": ".content__default h5",
    "text": ".content__default p, .content__default li, .content__default pre .comment"
  }
}

https://github.com/bahmutov/cypress-examples

algolia-config.json

@bahmutov

💡 Include code comments in the index

{
  "index_name": "cypress-examples",
  "start_urls": ["https://glebbahmutov.com/cypress-examples/"],
  "selectors": {
    "lvl0": {
      "selector": ".site-name",
      "global": true
    },
    "lvl1": ".content__default h1",
    "lvl2": ".content__default h2",
    "lvl3": ".content__default h3",
    "lvl4": ".content__default h4",
    "lvl5": ".content__default h5",
    "text": ".content__default p, .content__default li, .content__default pre .comment"
  }
}

https://github.com/bahmutov/cypress-examples

algolia-config.json

💡 Look at searches with no results

⚠️ free Algolia plans only get weekly emails with such queries

write docs for these searches!

@bahmutov

💡 Search from CLI

https://github.com/bahmutov/cy-search

@bahmutov

Final Thoughts

Good documentation is essential to your project's success

@bahmutov

Final Thoughts

Creating an ideal documentation structure is hard and might be impossible

@bahmutov

Final Thoughts

Creating a good enough text search using Algolia is pretty simple 🎉

@bahmutov

Thank you & More Info

Tweet at me when you get 10x users from a better docs search!

Find Me If You Can

By Gleb Bahmutov

Find Me If You Can

Having a good text search is the key to successful documentation, especially as the number of pages, examples, code recipes, answers, and other user-facing text snippets grow. In this presentation, I will show how to set up the Algolia search index and use it from a static site to make sure your users find _everything_ they are looking for. Video at https://www.youtube.com/watch?v=z_RutLs0W70

  • 3,432

More from Gleb Bahmutov