Percolation in ElasticSearch

Jason Foster, Orion Healthcare

Like, for Reverse Search...

• Normally we index multiple documents, then run individual searches to find matching documents
• Percolation indexes multiple queries, then runs individual searches on a single document to find matching queries

Percolation Use Cases

• Providing alerts based on customizable "interests"
• Categorizing and/or summarizing incoming data
• Workflow & routing

How It Works

• Add queries to percolator

curl -XPUT 'localhost:9200/my-index/.percolator/1' -d '{
    "query" : {
        "match" : {
            "message" : "bonsai tree"
        }
    }
}'

• Match documents to the added queries

curl -XGET 'localhost:9200/my-index/message/_percolate' -d '{
    "doc" : {
        "message" : "A new bonsai tree in the office"
    }
}'

• Interrogate/act-on response

Sample Response

{
    "took" : 19,
    "_shards" : {
        "total" : 5,
        "successful" : 5,
        "failed" : 0
    },
    "total" : 1,
    "matches" : [
        {
             "_index" : "my-index",
             "_id" : "1"
        }
    ]
}

Other Cool Stuff

• Queries are just documents - so they can have other attributes that can be used when matches are found (like a user id)
• Works in batch and streaming
• Can do just counts
• Percolate existing documents, not just new ones
• Multi-Percolate

Under the Hood

• All percolate queries are kept in memory
• Utilizes an optimized 'single-document, in-memory index' 
• Executes queries against that single document

What Can't It Do

• Deal with child documents
• Execute wildcard/regexp queries
• Substitute for the Flux Capacitor

Demo