Vector Databases Explained: Pinecone, FAISS and ChromaDB

An embedding model turns each of your documents into a list of numbers. Searching by meaning then becomes a geometric problem: given the vector for a question, find the stored vectors closest to it. With a thousand documents you compare against all of them and nobody notices. With fifty million you cannot.

A vector database is the tool for that second situation. It stores vectors, keeps an index that finds near neighbours without checking every one, and wraps that in the things a database provides: filtering, updates, persistence. See embeddings explained for where the vectors come from.

The core problem

Finding the exact nearest neighbours means computing a distance to every stored vector. That is correct, simple and linear in the size of the collection. Every index beyond it makes the same bargain: look at fewer vectors, and accept that now and then a true neighbour is missed. This is approximate nearest neighbour search, and the share of true neighbours it still finds is called recall.

These are the names used in FAISS, a library from Meta built for exactly this job (Faiss wiki: Faiss indexes). HNSW, the graph method, comes from a 2016 paper and is offered by many vector databases, pgvector among them (Malkov and Yashunin: HNSW).

Measuring the bargain

The script takes 35,764 real embeddings: one for each paragraph of the public-domain Python Enhancement Proposals (peps.python.org), made with a small open embedding model. It searches for the ten nearest neighbours of 500 of them among the rest, three ways, and counts how many vectors each method actually compared.

import faiss
import numpy as np

faiss.omp_set_num_threads(1)                           # one thread, so the run is repeatable
vectors = np.load("../../genai/pep_paragraph_vectors.npy")   # one embedding for each paragraph
data, queries = vectors[:-500], vectors[-500:]         # search for the last 500 among the rest
n, d = data.shape
k = 10

flat = faiss.IndexFlatIP(d)                            # exact: compares a query with every vector
flat.add(data)
_, truth = flat.search(queries, k)


def recall(found):
    return np.mean([len(set(f) & set(t)) / k for f, t in zip(found, truth)])


print(f"{n:,} vectors of {d} numbers = {n * d * 4 / 1e6:.0f} MB as 32-bit floats")
print()
print("index              setting        vectors compared per query   recall@10")
print(f"flat (exact)       -              {n:>26,}   {1.0:>9.2f}")

ivf = faiss.IndexIVFFlat(faiss.IndexFlatIP(d), d, 256, faiss.METRIC_INNER_PRODUCT)
ivf.train(data)                                        # k-means splits the vectors into 256 groups
ivf.add(data)
for nprobe in (1, 4, 16, 64):
    ivf.nprobe = nprobe                                # how many groups to look in
    faiss.cvar.indexIVF_stats.reset()
    _, found = ivf.search(queries, k)
    compared = faiss.cvar.indexIVF_stats.ndis / len(queries)
    print(f"IVF, 256 groups    nprobe={nprobe:<6} {compared:>26,.0f}   {recall(found):>9.2f}")

hnsw = faiss.IndexHNSWFlat(d, 32, faiss.METRIC_INNER_PRODUCT)
hnsw.add(data)                                         # builds a graph linking each vector to near ones
for ef in (16, 64, 256):
    hnsw.hnsw.efSearch = ef                            # how widely to explore the graph
    faiss.cvar.hnsw_stats.reset()
    _, found = hnsw.search(queries, k)
    compared = faiss.cvar.hnsw_stats.ndis / len(queries)
    print(f"HNSW, 32 links     efSearch={ef:<4} {compared:>26,.0f}   {recall(found):>9.2f}")

35,264 vectors of 384 numbers = 54 MB as 32-bit floats

index              setting        vectors compared per query   recall@10
flat (exact)       -                                  35,264        1.00
IVF, 256 groups    nprobe=1                             167        0.55
IVF, 256 groups    nprobe=4                             721        0.83
IVF, 256 groups    nprobe=16                          2,765        0.94
IVF, 256 groups    nprobe=64                          9,933        0.99
HNSW, 32 links     efSearch=16                          536        0.92
HNSW, 32 links     efSearch=64                        1,267        0.97
HNSW, 32 links     efSearch=256                       3,690        0.98

• Exact search compared every query with all 35,264 vectors. That is the baseline the others are measured against.
• IVF with one cluster looked at 167 vectors and found 55% of the true neighbours. Cheap, and wrong nearly half the time.
• Searching more clusters buys recall. At 16 clusters, 2,765 comparisons gave 94%. At 64, almost 10,000 comparisons gave 99%.
• HNSW reached 92% with 536 comparisons, and 97% with 1,267. On this data it got more recall for less work than IVF.
• Neither reached 100%. That is the nature of an approximate index.
• The vectors themselves take 54 MB. At four bytes a number, memory grows with both the count and the size of the vectors.

Two settings did all of that, nprobe for IVF and efSearch for HNSW. Every vector database has an equivalent dial, often with a default nobody has looked at. If a retrieval system misses things it should find, check the index setting before blaming the embedding model.

The pgvector documentation describes the same trade-off between the two index types in general terms: HNSW has the better balance of speed and recall, and is slower to build and uses more memory (pgvector).

Four kinds of tool

The products usually compared in one breath are not the same kind of thing.

FAISS is the engine without the car. Its own paper calls vector search a core function of vector databases and describes FAISS as a toolkit of indexing methods (Douze et al.: The Faiss library). It gives you the indexes shown above and nothing else: no storage of the original text, no filtering by metadata, no server. It suits experiments, batch jobs and systems that build their own storage around it.

Chroma is an open-source database that keeps documents, metadata and embeddings together and can filter by metadata while searching. It runs inside a Python program for a prototype and can be self-hosted or used as a cloud service later (Chroma: introduction). It suits getting a retrieval application working quickly.

Pinecone is a hosted service. You create an index, send records and query it, and the vendor runs the machines. Its indexes hold dense vectors, sparse vectors and text, and usage is priced by reads and writes (Pinecone: key terms). It suits teams that would sooner pay than operate.

pgvector adds a vector column and vector indexes to Postgres. Search is exact by default, and an approximate index can be added. Its attraction is everything else Postgres already does: transactions, joins, backups and the rest of your data in the same place. It suits applications that already have a Postgres database and a moderate number of vectors.

Do you need one

1. A few thousand vectors? Keep them in memory and compare against all of them. It is exact, and it is fast enough.
2. Already using Postgres, with up to a few million vectors? Try pgvector before adding a new system.
3. Building a prototype? An embedded database such as Chroma gets you storage, metadata and search in a few lines.
4. Large scale, or no wish to run infrastructure? A hosted service.
5. Unusual requirements, or research? A library such as FAISS, and build around it.

Whatever you choose, three things decide whether it works in practice:

• Filtering. Real queries are rarely "nearest overall". They are "nearest among this customer's documents from this year". Check how the tool combines a metadata filter with vector search.
• Updates. Some indexes handle frequent inserts and deletes better than others.
• Hybrid search. Vector search misses exact terms. Several of these tools can combine it with keyword search. See semantic search vs keyword search.

Common mistakes

• Choosing the database before measuring retrieval quality. The embedding model and the way documents are split matter more. See chunking strategies for RAG.
• Leaving the recall setting at its default and never testing it against exact search.
• Comparing products on speed alone. A fast index with 80% recall is answering a different question.
• Adding a vector database for 5,000 documents.
• Forgetting that changing the embedding model means re-embedding and re-indexing everything.
• Storing vectors without the text or an identifier to get back to it.

For how the retrieved passages are used, see RAG architecture explained. For measuring whether retrieval is good enough, see retrieval evaluation.

Vector databases, including FAISS, Pinecone and Chroma, are covered in the Vector Databases and Retrieval module of the Generative AI course.

Sources

• The Faiss library (arXiv) — Douze et al.
• Faiss indexes (Faiss wiki on GitHub) — Meta Research
• Efficient and robust approximate nearest neighbor search using Hierarchical Navigable Small World graphs (arXiv) — Malkov and Yashunin
• pgvector: open-source vector similarity search for Postgres (GitHub) — pgvector
• Introduction (Chroma Docs) — Chroma
• Pinecone key terms (Pinecone Docs) — Pinecone
• Python Enhancement Proposals (Python Software Foundation) — Python Software Foundation