Find Me If You Can

Algolia Search is Easy

JS Monthly Online #07

Gleb Bahmutov

VP of Engineering

Cypress.io 

@bahmutov

our planet is in imminent danger

https://lizkeogh.com/2019/07/02/off-the-charts/

+3 degrees Celsius will be the end.

survival is possible. but we need to act now

  • change your life
  • join an organization

Talk Contents

  • The funnel 

  • The documentation

  • The search

    • simple example

    • complex documentation

  • Tips, Conclusions, Q & A

The Funnel

aka "You are always losing customers"

funnel of visitors to paying customers

The Funnel

aka "You are always losing customers"

funnel of visitors to paying customers

💵

💰

👀

🧑‍💻

🦸

The Funnel

aka "You are always losing customers"

funnel of visitors to paying customers

10-20%

The Funnel

aka "You are always losing customers"

funnel of visitors to paying customers

10-20%

What if you converted all first time visitors to paying customers?

The Funnel

aka "You are always losing customers"

funnel of visitors to paying customers

10-20%

What if you converted all first time visitors to paying customers?

You would increase your revenue 5 or 10 times!

The Funnel

aka "You are always losing customers"

funnel of visitors to paying customers
Why the drop outs?

This tool does not do what I need

It does not work on my platform

Valid reasons

The Funnel

aka "You are always losing customers"

funnel of visitors to paying customers
Why the drop outs?

This tool does not do what I need

It does not work on my platform

Valid reasons

This tool does not do what I need (but it does)

Does not work on my platform (but it does work)

Wrong reasons

The Funnel

aka "You are always losing customers"

funnel of visitors to paying customers
Why the drop outs?

This tool does not run on my CI

Valid reasons

I don't know how to make this tool work on my CI

Wrong reasons

The Funnel

aka "You are always losing customers"

funnel of visitors to paying customers
Why the drop outs?

I cannot afford this tool

Valid reasons

I don't know what value this tool gives me

Wrong reasons

The Funnel

aka "You are always losing customers"

funnel of visitors to paying customers
Why the drop outs?

I am happy with this tool, that's enough for me

Valid reasons

I did not know this tool could do that

Wrong reasons

The Funnel

aka "You are always losing customers"

funnel of visitors to paying customers
Why the drop outs?

Market fit

Features

Bug fixes

The Funnel

aka "You are always losing customers"

funnel of visitors to paying customers
Why the drop outs?

Market fit

Features

Bug fixes

Bad UI

Bad docs

Lack of examples

The Funnel

aka "You are always losing customers"

funnel of visitors to paying customers
Why the drop outs?

Market fit

Features

Bug fixes

Bad UI

Bad docs

Lack of examples

Goal: drive the "bad" area to zero

The Funnel

aka "You are always losing customers"

funnel of visitors to paying customers

If you stop losing users in red area, you would increase your revenue 2 or 5 times!

Bad UI

Bad docs

Lack of examples

Goal: drive the "bad" area to zero

Features vs Docs

Bad UI

Bad docs

Lack of examples

⚖️

Market fit

Features

Bug fixes

  • So
  • Much
  • Easier

Improve the Docs: Personas

🔑 Visitor / User should always find the right answer

First time visitor

  • What does this tool / service do?

  • Will it work for me?

🔑 Visitor / User should always find the right answer

First time user

  • How do I do X?

  • How do I do Y?

  • What do I do for error A?

First time user

🔑 Visitor / User should always find the right answer

Repeat user

  • How do I do X and Y and Z?

  • What do I do for error A, B, C?

  • How do I run using ...?

  • How do I learn more?

Repeat user

🔑 Visitor / User should always find the right answer

Paying customer

  • same as repeat user +

  • Is there training?

  • I got stuck, is there help?

Paying customer

🔑 Visitor / User should always find the right answer

Promoter

  • When are the new releases?

  • What else can this tool do?

  • Can I suggest a feature?

  • Is there an email list or webinars?

Docs keep growing

Information for every user persona

How to measure how good your documentation is ... and start crying

Is there a way to disable command snapshots?

How to preserve session cookie?

How to check the length of text?

How to measure how good your documentation is ... and start crying

Every question here is a failure of the documentation

Every question here is probably a:

  1. a support issue 💵
  2. maybe a lost customer 💸

Documentation miss percentage

all well-defined questions across chat, support emails, github issues
questions that have answer in your documentation
* 100%

Documentation miss percentage

< 50%       write more docs!

≥ 50%

docs have a problem

(many things are not documented)

(users cannot find the right documentation)

You Want Users to Find Answers to Their Questions by Themselves

The documentation demands for different user personas are contradictory

Why Good Docs Are Hard?

First time visitor

First time user

Repeat user

Paying customer

Promoter

💥

"show me Hello, World!"

"show me the changelog diff from version X to Y"

"show me a tutorial"

"show me how to do X"

"how do I solve my issue or bug?"

👀 Optimize documentation structure for beginners

  • "Hello World" example front and center
  • There are 10 beginners for every 1 advanced user. Every user starts as a beginner

👀 Optimize documentation structure for beginners

Then add all your documentation to the site reachable from the index

💡 Make a powerful docs search for every persona

💡 Do NOT answer support questions

💡💡💡 Especially for private support

💡 Do NOT answer support questions

Instead, update the documentation, or create an example, or write a blog post.

Then answer with a link

💡 The Support Team

Should eliminate their own jobs by writing more and more documentation until the users find everything themselves

Talk Contents

  • The funnel

  • The documentation

  • The search

    • simple example

    • complex documentation

  • Tips, Conclusions, Q & A

Sample App

Why not use the built-in client-side indexing?

Hint: look at the search results for "about" query

Algolia

  • I am unaffiliated, just a user

  • Generous free plan

  • Good documentation, scraping tools, nice UI

Algolia in action

Create Algolia App

Create Search Index

Workflow

  1. After every deploy 🔼
    1. Scrape the site 🕷
  2. Set search widget to point at the index using Algolia SDKs

API Keys

to add records to the index

(keep private)

API Keys

to search index from site

(public)

Scraping the Site

Document Structure

{
  "index_name": "scrape-test",
  "start_urls": ["https://glebbahmutov.com/triple-tested/"],
  "selectors": {
    "lvl0": {
      "selector": ".site-name",
      "global": true
    },
    "lvl1": ".content__default h1",
    "lvl2": ".content__default h2",
    "lvl3": ".content__default h3",
    "lvl4": ".content__default h4",
    "lvl5": ".content__default h5",
    "text": ".content__default p, .content__default li"
  }
}

Algolia config (JSON)

Scraping

# when scraping the site, inject secrets as environment variables
# then pass their values into the Docker container using "-e" syntax
# and inject config.json contents as another variable
- name: scrape the site 🧽
  env:
    APPLICATION_ID: ${{ secrets.APPLICATION_ID }}
    API_KEY: ${{ secrets.API_KEY }}
  run: |
    docker run \
    -e APPLICATION_ID -e API_KEY \
    -e CONFIG="$(cat config.json)" \
    algolia/docsearch-scraper:v1.6.0

use Algolia Docker image

Scraping

# when scraping the site, inject secrets as environment variables
# then pass their values into the Docker container using "-e" syntax
# and inject config.json contents as another variable
- name: scrape the site 🧽
  env:
    APPLICATION_ID: ${{ secrets.APPLICATION_ID }}
    API_KEY: ${{ secrets.API_KEY }}
  run: |
    docker run \
    -e APPLICATION_ID -e API_KEY \
    -e CONFIG="$(cat config.json)" \
    algolia/docsearch-scraper:v1.6.0

use Algolia Docker image

Searching Widget

module.exports = {
  title: 'Triple Tested',
  description: 'Example static site',
  base: '/triple-tested/',
  themeConfig: {
    algolia: {
      // DANGER 🧨💀: ONLY USE ALGOLIA PUBLIC SEARCH-ONLY API KEY
      apiKey: 'a2f5586dd739087cabead4b20ae0c2bc',
      indexName: 'scrape-test',
      appId: 'XVIJU4G89W'
    }
  }
}

using built-in VuePress Algolia Widget

Markup and operation

Larger sites: Multiple roots

blog

examples

Larger sites: Multiple roots

{
  "index_name": "cypress",
  "start_urls": [
    {
      "url": "https://docs.cypress.io/",
      "page_rank": 10
    },
    {
      "url": "https://example.cypress.io/",
      "selectors_key": "kitchensink",
      "tags": ["example"],
      "page_rank": 2
    },
    {
      "url": "https://www.cypress.io/blog/",
      "selectors_key": "blog",
      "tags": ["blog post"],
      "page_rank": 1
    }
  ],
  "stop_urls": [
    "^https://docs.cypress.io/ja/",
    "^https://docs.cypress.io/zh-cn/",
    "^https://docs.cypress.io/pt-br/",
    "^https://docs.cypress.io/ru/"
  ],
  "selectors_exclude": [],
  "selectors": {
    "default": {
      "lvl0": "article h1.article-title",
      "lvl1": "article h1.article-heading",
      "lvl2": "article h2.article-heading",
      "text": "article .article-content p, article .article-content tr, article .article-content li, article .article-content pre"
    },
    "blog": {
      "lvl0": "article h1",
      "lvl1": "article h2",
      "text": "article p, article pre"
    },
    "kitchensink": {
      "lvl0": ".container h1",
      "lvl1": ".container h4",
      "text": ".container p, .container pre"
    }
  },
  "nb_hits": 0,
  "min_indexed_level": 2
}

Algolia config (JSON)

Larger sites: Multiple roots

2 minutes to scrape, creates ~ 21k records

Larger sites: Multiple roots

Talk Contents

  • The funnel

  • The documentation

  • The search

    • simple example

    • complex documentation

  • Tips, Conclusions, Q & A

💡 Include code comments in the index

💡 Include code comments in the index

{
  "index_name": "cypress-examples",
  "start_urls": ["https://glebbahmutov.com/cypress-examples/"],
  "selectors": {
    "lvl0": {
      "selector": ".site-name",
      "global": true
    },
    "lvl1": ".content__default h1",
    "lvl2": ".content__default h2",
    "lvl3": ".content__default h3",
    "lvl4": ".content__default h4",
    "lvl5": ".content__default h5",
    "text": ".content__default p, .content__default li, .content__default pre .comment"
  }
}

algolia-config.json

💡 Include code comments in the index

{
  "index_name": "cypress-examples",
  "start_urls": ["https://glebbahmutov.com/cypress-examples/"],
  "selectors": {
    "lvl0": {
      "selector": ".site-name",
      "global": true
    },
    "lvl1": ".content__default h1",
    "lvl2": ".content__default h2",
    "lvl3": ".content__default h3",
    "lvl4": ".content__default h4",
    "lvl5": ".content__default h5",
    "text": ".content__default p, .content__default li, .content__default pre .comment"
  }
}

algolia-config.json

💡 Look at searches with no results

⚠️ free Algolia plans only get weekly emails with such queries

write docs for these searches!

💡 Search from CLI

Final Thoughts

Good documentation is essential to your project's success

Final Thoughts

Creating an ideal documentation structure is hard and might be impossible

Final Thoughts

Creating a good enough text search using Algolia is pretty simple 🎉

Thank you & More Info

Tweet at me when you get 10x users from a better docs search!