Structured Query Engine

with Movie Recommendation System demo

Search Engine Architecture

NYU

 

Amanpreet Singh

Karthik Venkatesan

Simranjyot Singh Gill

WHY?

  • Complex Queries (for e.g. Get all the articles for sports by New York Times combined with those from CNN)

  • Handling user generated random queries (for e.g User wants movies from Brad Pitt with IMDb score >7)

  • Apply different kind of analyzing for different kind of fields in documents

  • Realtime addition, updation and deletion of documents

 

WHAT?

  • ​Based on ElasticSearch
  • Supports mappings and settings
  • Realtime Distributed
  • Supports querying documents vai Query DSL similar to ElasticSearch
{
  "movie": {
    "properties": {
      "director_name": {
        "type": "text",
        "analyzer": "n_gram",
        "search_analyzer": "standard"
      },
      "actor_names": {
        "type": "text",
        "analyzer": "n_gram",
        "search_analyzer": "standard"
      },
      "genres": {
        "type": "text"
      },
      "movie_title": {
        "type": "text",
        "analyzer": "n_gram",
        "search_analyzer": "standard"
      },
      "plot_keywords": {
        "type": "text"
      },
      "movie_imdb_link": {
        "type": "keyword",
        "index": false
      },
      "imdb_score": {
        "type": "float"
      }
    }
  }
}

Architecture

RESTful API

  • Info Controller
  • Index Controller
  • Index Query Controller

Realtime Distributed Indexer

  • Inverted Index
  • Document Store
  • Operations
    • Add
    • Delete
    • Update
    • Get

Information Retriever

Structured Query

  • Leaf Query
    • Term Query
    • Match Query
  • Compound Query (Bool)
    • Should Clause
    • Range Filter
  • Boost Query Class
     

Movie Recommendation System

Movie Recommendation System

  • Built using React
  • SPA, uses axios to make AJAX requests
  • Provides simple UI to search for your favorite movie and then get recommendations

Architecture

Search Query

{"query": 
  {"bool": 
    {"should":
       [{"match": {"director_name": "Shane Black"}},
       {"match": {"plot_keywords": {"query": "human bomb", "boost": 2.0}}},
       {"match": {"actor_names": "Robert Downey Jr."}},
       {"term": {"actor_names": "Cheadle"}},
       {"match": {"genres": {"query": "Action", "boost": 2.0}}},
       {"match": {"genres": {"query": "Sci-Fi", "boost": 2.0}}}],
    "filter":
       [{"range": {"imdb_score": {"gte": 6.0}}}]
    }
  }
}
  • search_analyzer

Recommendations

  • Boosting

Demo

?

Thank You!

Structured Query Engine

By Amanpreet Singh

Structured Query Engine

  • 1,173