Picking What to Watch Next

build a recommendation system

Cheuk Ting Ho

Cheukting

@cheukting_ho

https://cheuk.dev

Project repo: github.com/Cheukting/knn_recommender

* start downloading the datasets *

Overview

30 mins - understand recommenders

10 mins - have a look at our project:

https://github.com/Cheukting/knn_recommender

50 mins - work on data preparation

50 mins - work on model

10 mins - try it out

What is a recommender?

  • Based on the data about the users and items
  • Make personalized recommendations
  • Important to online business when there's no/ less human involved

Type of recommenders

  • content-based systems
  • collaborative filtering systems
  • hybrid systems
    (just the combination of the 2 👍🏻)

Content-based systems

  • If you have watched Cowboy Bebop the anime
  • You probably would like Cowboy Bebop the live-action series
  • (not always true 🤦🏻‍♀️)

Collaborative filtering systems

  • If both you and I have watched a lot of anime
  • If I watched Cowboy Bebop the anime
  • You may like Cowboy Bebop the anime
  • (there are popularity bias 🤷🏻‍♀️)

In our workshop

Data: ratings.csv and movies.csv from MovieLens Datasets (full version)

Method: collaborative filtering

Problem: Unpopular movies, inactive users -> sparse data points

Preparing the data

  • there are long tails in the data
  • many movies received only a few ratings (unpopular)
  • many users only rate a few movies (inactive)
  • we only want to use the popular movie and active users to reduce bias

Preparing the data

  • do it in _prep_data
  • create a filter for popular movies
  • create a filter for active users
  • only use those that are popular and active
  • filter data that satisfied both filters

Preparing the data

  • create a m x n array
  • m: number of movies, n: number of users
  • create a mapper from movie title to index
  • the m x n array is sparse (not every user rate every movies)
  • transform array to SciPy sparse matrix

Our model

  • collaborative filtering
  • user-based or item-based
  • KNN model: similar movies are clustered together
  • similar means received similar ratings from users
  • feed the m x n array (sparse matrix) into KNN model
  • inference and get the top n nearest neighbours

Our model

  • using SciKit-learn NearestNeighbor:
    self.model = NearestNeighbors()
  • parameters are set in set_model_params
  • in _inference get the distances and indices of the top n_neighbours
  • try different settings

Run our model in CLI

  • run the Python script as a module with options
  • Example:
    python knn_recommender.py --movie_name "Iron Man" --top_n 10
  • Details about the options: see project repo
  • after the data is processed it will be pickled for faster rerun (and used in PyScript version)

Run our model in browser (PyScript)

  • make sure you have the pickle files (hashmap.p and movie_user_mat_sparse.p)
  • start a local server: python -m http.server
  • Open http://0.0.0.0:8000 and select knn_recommender.html from there

What have we learnt

  • Know about the different types of recommenders
  • Building a simple recommender
  • See what data quality issue we may have
    • popularity bias: bias toward popular items
    • cold-start problem: new items do not get enough reviews

References and Credits

Kevin Laio (KevinLiao159) for original code and blog post

 

PyScript: https://pyscript.net/

 

Project repo: https://github.com/Cheukting/knn_recommender