Recommender Systems

Users need a next item, and a recommender has to predict which one. The page starts with the recommendation task itself (content-based methods, collaborative filtering, and matrix factorization), then asks how to evaluate those systems, and ends with the tools that implement them.

The same notes are in Embedding, Multi Armed Bandits, SIMILARITY, SVD, and TF-IDF.

The recommendation task

The task has two classic answers: recommend items similar to what the user liked, or recommend what similar users liked. The first is content-based. Beginner guide is Shuvayan's guide to content based recommender systems, their workings, applications, and limitations, with a step-by-step build. The second is collaborative filtering, and Real python on CF walks the steps involved in it. Intro to, using item-item or user-item is Ethan Rosenthal's introduction, written because he found Andrew Ng's course lacking on recommender systems, which he sees as completing the triumvirate of machine learning pillars for data science after regression and classification.

The simplest content-based version is Tfidf cosine similarity, and countvec cosine is the DataCamp beginner tutorial on recommender systems in Python that shows it. Various implementations of CF; a serious review of algorithms is listed in the tools section below. Collaborative filtering, SVD is HackerNoon's introduction, which defines a recommender system as one that predicts the future preference of a set of items for a user and recommends the top items, needed because the Internet gives people too many options; the same piece is listed as Part1, of the series. The DataCamp tutorial is also A general tutorial, has a nice intro.

Movies are the standard playground. Medium on Movies was a three-part series (Part 1 matrix factorization in movies, users vs movies; Part 2 using collaborative filtering using open ai; Part 3 using col-filtering with neural nets), and its sources are kept at the end of the page. The Medium series on collaborative filtering and embeddings Part 1, part 2 is also gone, but its git repo remains. Movie recommender systems is a Kaggle notebook using data from The Movies Dataset, and the basics of recommendation models are On git.

Matrix factorization is the next step past neighbours; its summary is kept at the end of the page. Real data is often binary rather than rated: Collaborative filtering with binary countvec data, item-item, didnt work well on another domain is a simple, non state of the art write-up of item-item collaborative filtering with binary or unary data, since most examples use the MovieLens 1–5 star ratings. The Netflix competition, matrix factorization over classical algorithms, a survey paper, and Movie similarity based on genre used to sit here and are kept at the end of the page.

Similarity at scale is its own problem. Similar entities, matrix multiplication high sparsity: comparing very large feature vectors usually means a sparse matrix multiplication followed by selecting the top-n results, and a customized Cython function improves speed by about 40% over SciPy and NumPy. The distance itself also breaks down: Euclidean distance with high sparse data is the question of comparing objects built from texts whose length varies a lot.

To build one yourself, Excel & fastai Excel fastai git is collaborative filtering from scratch on MovieLens, implemented with fastai and pytorch. CF for movie recommendation is Bin Wang's Python implementation of baseline item-based collaborative filtering, and its related article is the Comparison item vs user cf. The Real Python piece in full is build a recommendation engine with collaborative filtering.

Evaluating Recommender Systems

Once recommenders exist, the question is which one is better, for the business and for accuracy.

The same notes are in Evaluation Metrics.

An exhaustive list of methods to evaluate used to open this list and is kept at the end of the page. For the business view, Choosing the best for your business starts from the rise of Software-as-a-Service recommender systems, which are no longer a privilege of large companies building their own in-house. Evaluating is Saumyadeepta Sen's post on the metrics used for recommender systems, after earlier posts on classification metrics. The accuracy side is formalized in the survey of accuracy eval metrics for RS by Microsoft. Metrics then need a process: Building a validation framework is Dimitris Apostolopoulos on building a validation framework for a recommender system. Evaluation Metrics for RS is kept at the end of the page. The hardest case is having no history at all: offline vs online validation answers how to validate a recommendation system without prior user interaction data, including a detailed rebuttal of a paper on how much traffic recommenders actually cause. Evaluating RS is the paper for a designer choosing between candidate approaches, where the first step is deciding which properties of the application matter.

TOOLS

After evaluation, the work is implementation, and the tools here are Surprise, Grover Prince’s repo, and python-recsys.

Surprise is a Python scikit for building and analyzing recommender systems, and its docs FAQ shows how to get the top-N recommendations for each user. The related article for Grover Prince's notes for machine learning is Grover prince. Recsys is a python library for implementing a recommender system.

Various implementations of CF are compared in the Towards Data Science article https://towardsdatascience.com/various-implementations-of-collaborative-filtering-100385c6dfe0, which is also the related article for the repos above: https://towardsdatascience.com/various-implementations-of-collaborative-filtering-100385c6dfe0

Body Transformation Product Recommendation Using Gen-AI (August 2025) is a recommender built as a product.

Deprecated links

These links and images no longer work. The original wording is kept here. A same-resource copy, when one was checked, is used above.

• Tfidf cosine similarity. This address no longer opens: https://towardsdatascience.com/recommender-engine-under-the-hood-7869d5eab072

• Spotlight, item2vec, Neural nets for Recommender systems. This address no longer opens: https://towardsdatascience.com/introduction-to-recommender-system-part-2-adoption-of-neural-network-831972c4cbf7

• Part 1 matrix factorization in movies, users vs movies. This address no longer opens: https://towardsdatascience.com/fast-ai-season-1-episode-5-1-movie-recommendation-using-fastai-a53ed8e41269

• Part 2 using collaborative filtering. This address no longer opens: https://towardsdatascience.com/fast-ai-season-1-episode-5-2-collaborative-filtering-from-scratch-1877640f514a

• Part 3 using col-filtering with neural nets. This address no longer opens: https://towardsdatascience.com/fast-ai-season-1-episode-5-3-collaborative-filtering-using-neural-network-48e49d7f9b36

• Medium series on collaborative filtering and embeddings Part 1. This address no longer opens: https://towardsdatascience.com/collaborative-filtering-and-embeddings-part-1-63b00b9739ce

• part 2. This address no longer opens: https://towardsdatascience.com/collaborative-filtering-and-embeddings-part-2-919da17ecefb

• Matrix factorization. This address no longer opens: https://towardsdatascience.com/paper-summary-matrix-factorization-techniques-for-recommender-systems-82d1a7ace74

• Netflix competition, matrix factorization over classical algorithms, a survey paper. This address no longer opens: https://towardsdatascience.com/paper-summary-matrix-factorization-techniques-for-recommender-systems-82d1a7ace74

• Movie similarity based on genre. This address no longer opens: https://towardsdatascience.com/content-based-recommender-systems-28a1dbd858f5

• An exhaustive list of methods to evaluate. This address no longer opens: https://towardsdatascience.com/an-exhaustive-list-of-methods-to-evaluate-recommender-systems-a70c05e121de

• Evaluation Metrics for RS. This address no longer opens: https://towardsdatascience.com/evaluation-metrics-for-recommender-systems-df56c6611093

• (how to find product relations, important: creating negative samples). This address no longer opens: https://eugeneyan.com/2020/01/06/recommender-systems-beyond-the-user-item-matrix