Recommendation Systems: How Netflix and Amazon Suggest Items

Recommendation systems decide much of what people see online: the next film to watch, the products shown beside the one you are viewing, the songs in a generated playlist. They look sophisticated from outside, and the largest ones are. But their core ideas are simple, and a working recommender fits in a few dozen lines of Python. This guide explains the main approaches, builds one on a real online shop's transactions, and evaluates it honestly, which turns out to be the most instructive part.

Two famous examples

Amazon and item-to-item collaborative filtering. A 2003 paper by Amazon researchers, Amazon.com Recommendations: Item-to-Item Collaborative Filtering, described relating items to each other through purchase histories rather than finding customers similar to you. For each item a customer bought, the system pulls a list of related items, and items appearing repeatedly across those lists become recommendations. Because item relationships can be computed ahead of time, the approach scales far better than comparing every customer with every other (Amazon Science: history of Amazon's recommendation algorithm).

Netflix and the Netflix Prize. From 2006 to 2009, Netflix ran a public competition to predict how users would rate films. To win the 1,000,000 dollar grand prize, a team had to reduce the prediction error of Netflix's own system, Cinematch, by 10% (Wikipedia: Netflix Prize). The competition drew tens of thousands of teams and made rating prediction the best-known recommendation problem.

The main approaches

• Popularity: recommend what is selling or trending. No personalisation, but a strong baseline and the fallback for new users.
• Content-based: recommend items whose attributes, such as genre, brand or text, resemble what the user liked.
• Collaborative filtering: use behaviour alone. User-based versions find similar users; item-based versions, like Amazon's, find items bought or watched by the same people.
• Matrix factorisation and embeddings: learn a vector for every user and item so that their similarity predicts interest. Neural recommenders extend this idea.
• Hybrids and ranking models: combine many signals, such as history, context, popularity and item attributes, in a model trained to rank candidates.

Building item-to-item recommendations on real data

The data is every transaction of a UK online gift retailer between December 2010 and December 2011, many of whose customers are wholesalers (UCI: Online Retail). Purchases before November 2011 are used to build the recommender; purchases in November and December are used to test it, just as a real system learns from the past and is judged on the future.

import io
import urllib.request
import zipfile
from itertools import islice
import numpy as np
import pandas as pd
from scipy import sparse
from sklearn.preprocessing import normalize

url = "https://archive.ics.uci.edu/static/public/352/online+retail.zip"
with zipfile.ZipFile(io.BytesIO(urllib.request.urlopen(url).read())) as archive:
    retail = pd.read_excel(archive.open("Online Retail.xlsx"))

retail = retail[retail["CustomerID"].notna() & (retail["Quantity"] > 0)
                & ~retail["InvoiceNo"].astype(str).str.startswith("C")]           # drop cancellations
retail = retail.assign(customer=retail["CustomerID"].astype(int), item=retail["StockCode"].astype(str))
cutoff = pd.Timestamp("2011-11-01")
train, test = retail[retail["InvoiceDate"] < cutoff], retail[retail["InvoiceDate"] >= cutoff]
print(f"{len(retail)} purchase lines | {train['customer'].nunique()} customers and "
      f"{train['item'].nunique()} products before {cutoff.date()}")

397924 purchase lines | 3974 customers and 3613 products before 2011-11-01

Next, a customer-by-product matrix records who bought what. Each product becomes a vector of its buyers, and the cosine similarity between two products measures how much their buyers overlap:

customers = pd.Index(sorted(train["customer"].unique()))
items = pd.Index(sorted(train["item"].unique()))
item_names = items.to_numpy()
pairs = train[["customer", "item"]].drop_duplicates()
bought = sparse.csr_matrix((np.ones(len(pairs)), (customers.get_indexer(pairs["customer"]), items.get_indexer(pairs["item"]))),
                           shape=(len(customers), len(items)))
item_vectors = normalize(bought.T.tocsr())              # one row per item: who bought it
similarity = (item_vectors @ item_vectors.T).tocsr()     # cosine similarity between items
similarity.setdiag(0)

names = train.drop_duplicates("item").set_index("item")["Description"]
anchor = items.get_loc("85123A")
print("customers who bought", names["85123A"], "also bought:")
for j in np.argsort(-similarity[anchor].toarray().ravel())[:5]:
    print(f"  {similarity[anchor, j]:.2f}  {names[items[j]]}")

customers who bought WHITE HANGING HEART T-LIGHT HOLDER also bought:
  0.52  RED HANGING HEART T-LIGHT HOLDER
  0.50  CANDLEHOLDER PINK HANGING HEART
  0.41  HEART OF WICKER LARGE
  0.39  HEART OF WICKER SMALL
  0.37  WOODEN PICTURE FRAME WHITE FINISH

No product descriptions were used, only who bought what, yet the neighbours of a white hanging-heart candle holder are its red and pink versions and other heart-themed decorations. That is collaborative filtering: behaviour reveals relationships between items. The matrix is very sparse, since each customer buys a small fraction of the catalogue, which is why sparse matrices are essential at scale.

Evaluating honestly: compare with simple baselines

A recommender is only useful if it beats something simpler. Four strategies, each recommending 10 products, are scored on what the 1,544 returning customers actually bought in November and December. Precision@10 is the share of the 10 recommendations the customer went on to buy:

last_month = train[train["InvoiceDate"] >= cutoff - pd.Timedelta(days=30)]
bestsellers = last_month.groupby("item")["customer"].nunique().sort_values(ascending=False).index
purchase_counts = train.groupby(["customer", "item"])["InvoiceNo"].nunique()
future = test[test["customer"].isin(customers)].groupby("customer")["item"].apply(set)

def score(recommend):
    precision = [len(set(recommend(c)) & truth) / 10 for c, truth in future.items()]
    return f"precision@10 {np.mean(precision):.3f}, at least one hit for {np.mean(np.array(precision) > 0):.1%} of customers"

def top_new(ranked, c, k=10):                          # first k items the customer has never bought
    have = set(items[bought[customers.get_loc(c)].indices])
    return list(islice((i for i in ranked if i not in have), k))

def item_to_item(c):
    scores = (bought[customers.get_loc(c)] @ similarity).toarray().ravel()
    return item_names[np.argsort(-scores)]

print(len(future), "customers bought again in November-December")
print("bestsellers:           ", score(lambda c: bestsellers[:10]))
print("buy again:             ", score(lambda c: purchase_counts.loc[c].nlargest(10).index))
print("bestsellers, new only: ", score(lambda c: top_new(bestsellers, c)))
print("item-to-item, new only:", score(lambda c: top_new(item_to_item(c), c)))

1544 customers bought again in November-December
bestsellers:            precision@10 0.117, at least one hit for 59.1% of customers
buy again:              precision@10 0.271, at least one hit for 79.5% of customers
bestsellers, new only:  precision@10 0.062, at least one hit for 38.2% of customers
item-to-item, new only: precision@10 0.055, at least one hit for 32.2% of customers

The results are humbling, and typical:

• "Buy again" wins by a wide margin. Simply recommending each customer's own most frequently bought products gets 27% of recommendations right. For a shop whose customers are largely wholesalers restocking the same lines, that is exactly the job a recommender should do.
• For discovering new products, the bestseller list slightly beats item-to-item filtering. Personalisation did not pay off here, at least not in this simple form.

Strong simple baselines are not a quirk of this dataset. Amazon Science recounts an experiment in which a simple bestseller list outperformed a far more complex autoencoder recommender, so badly that the researchers re-ran the experiments several times to be sure (Amazon Science: history of Amazon's recommendation algorithm). The lesson: always report baselines, and let the business problem, restocking or discovery, decide what to optimise.

Improving the discovery case usually means combining signals: blending similarity with recent popularity, weighting recent purchases more, using product attributes, and training a ranking model on those features. Each change must beat the baselines above on the same time-split test.

Beyond accuracy

Offline precision is only a first filter. Real recommenders also need:

• Coverage and diversity: a system that only ever shows bestsellers leaves most of the catalogue unseen.
• Cold start handling: new users and new products have no history; popularity and content features fill the gap.
• Awareness of feedback loops: items that are recommended get bought, which makes them look even better next time, so popular items can crowd out everything else.
• Online testing: the final judge is a controlled experiment on live traffic. A/B testing for data scientists covers how to run one.

A checklist for building one

How to judge ranking results is covered in evaluation metrics explained. A recommendation system is one of the projects in the Data Science course.

Sources

• The history of Amazon's recommendation algorithm (Amazon Science) — Amazon Science
• Netflix Prize (Wikipedia) — Wikipedia
• Online Retail (UCI Machine Learning Repository) — UCI Machine Learning Repository