FAISS: Efficient Vector Similarity Search at Scale

FAISS (Facebook AI Similarity Search) is a C++ library with Python bindings for indexing and searching dense vectors. When your application needs to find the nearest neighbours of an embedding among millions or billions of vectors, FAISS gives you fine-grained control over the speed, memory, and recall trade-offs that a managed vector database hides behind an API. This tutorial focuses on the library itself: its index types, how each one works internally, and how to use them in production.

Who this tutorial is for

This is not another generic "build a semantic search engine" walkthrough. We assume you already understand embeddings and cosine similarity at a high level. Instead we go deep on the FAISS index zoo: exact flat indexes, inverted-file (IVF) indexes, product quantization (PQ), HNSW graphs, ID mapping, persistence, and GPU offload. By the end you will be able to choose the right index for your dataset size and latency budget, and tune it deliberately rather than by guesswork.

What FAISS is and what it is not

FAISS solves one problem extremely well: given a query vector, return the k most similar vectors from a collection, using either L2 (Euclidean) distance or inner product. It is a library, not a service. There is no network layer, no authentication, no metadata filtering engine, and no built-in persistence beyond reading and writing a single index file.

That minimalism is the point. FAISS lets you:

• Decide exactly how vectors are stored (full precision, quantized, on disk).
• Trade recall for speed by changing a single parameter at query time.
• Run the same index on CPU or GPU with a one-line move.
• Embed the index directly inside your own process, avoiding network round-trips.

When to use FAISS vs a managed vector database

Reach for FAISS when:

• You want a single library embedded in your service with no extra infrastructure.
• You need precise control over the index structure and memory footprint.
• Your vectors are mostly static, or you rebuild the index on a schedule.
• You are doing research or batch similarity computation.

Reach for a managed vector database (Qdrant, Milvus, Weaviate, pgvector, Pinecone) when:

• You need rich metadata filtering combined with vector search.
• You require frequent inserts, updates, and deletes with durability guarantees.
• You want horizontal scaling, replication, and an HTTP/gRPC API out of the box.
• Operating a stateful service is acceptable and you would rather not build it yourself.

Many of those databases actually use FAISS-like algorithms (IVF, HNSW, PQ) under the hood, so understanding FAISS makes you better at tuning them too.

Installation

FAISS ships as two mutually exclusive packages. Install exactly one.

# CPU-only build (works everywhere, good default)
pip install faiss-cpu

GPU build (requires a CUDA-capable GPU and matching CUDA runtime)
pip install faiss-gpu

For the examples we also use sentence-transformers to produce embeddings and numpy for array handling.

pip install sentence-transformers numpy

A quick sanity check:

import faiss
import numpy as np

print("FAISS version:", faiss.version)
print("Number of GPUs visible to FAISS:", faiss.getnumgpus())

If getnumgpus() returns 0 you are on the CPU build, which is fine for everything except the GPU section near the end.

Creating embeddings

FAISS works with float32 NumPy arrays of shape (nvectors, dimension). It does not generate embeddings itself; you bring your own. Here we use a small sentence-transformer model.

from sentencetransformers import SentenceTransformer
import numpy as np

model = SentenceTransformer("all-MiniLM-L6-v2")  # 384-dimensional output

documents = [
    "FAISS performs nearest-neighbour search over dense vectors.",
    "Product quantization compresses vectors to save memory.",
    "An inverted file index partitions the vector space into cells.",
    "HNSW builds a navigable small-world graph for fast search.",
    "Cosine similarity is inner product on normalized vectors.",
    "GPU indexes can accelerate search by an order of magnitude.",
]

embeddings = model.encode(documents, converttonumpy=True)
embeddings = embeddings.astype("float32")  # FAISS requires float32

print(embeddings.shape)  # (6, 384)
print(embeddings.dtype)  # float32

Two rules to internalize early:

d = embeddings.shape[1]  # 384

Exact search: IndexFlatL2 and IndexFlatIP

The flat indexes store every vector verbatim and compare the query against all of them. They give 100% recall (they are exhaustive) but scale linearly with the number of vectors. They are the right choice for small collections and the baseline you measure approximate indexes against.

import faiss

index = faiss.IndexFlatL2(d)
print("Trained?", index.istrained)   # True - flat indexes need no training

index.add(embeddings)                 # store all vectors
print("Vectors in index:", index.ntotal)  # 6

Searching returns two arrays: D (distances) and I (indices into the index), each of shape (nqueries, k).

query = model.encode(["how to compress vectors"], converttonumpy=True).astype("float32")
k = 3
D, I = index.search(query, k)

print("Distances:", D[0])   # ascending for L2
print("Indices:", I[0])
for rank, idx in enumerate(I[0]):
    print(rank, documents[idx])

indexip = faiss.IndexFlatIP(d)
indexip.add(embeddings)
D, I = indexip.search(query, k)

print(D[0])  # descending: higher is better

Cosine similarity equals the inner product of L2-normalized vectors. FAISS has no dedicated cosine index, so you normalize both the database and the query vectors and then use IndexFlatIP. The helper faiss.normalizeL2 does this in place.

import numpy as np

emb = embeddings.copy()
faiss.normalizeL2(emb)        # in-place L2 normalization


cosindex = faiss.IndexFlatIP(d)
cosindex.add(emb)


q = query.copy()
faiss.normalizeL2(q)          # normalize the query the same way
D, I = cosindex.search(q, k)

print("Cosine scores:", D[0])  # now in [-1, 1], higher is more similar

Constructing complex indexes by hand is verbose. faiss.indexfactory builds them from a short string description, which is also how you would store an index recipe in a config file.

# Equivalent to IndexFlatL2(d)
index = faiss.indexfactory(d, "Flat")


IVF with 100 cells, flat storage, L2 metric
index = faiss.indexfactory(d, "IVF100,Flat")

IVF with PQ compression: 100 cells, 16 sub-quantizers of 8 bits each
index = faiss.indexfactory(d, "IVF100,PQ16")


HNSW with 32 neighbours per node
index = faiss.indexfactory(d, "HNSW32")

Cosine similarity variant: pass the metric argument
index = faiss.indexfactory(d, "Flat", faiss.METRICINNERPRODUCT)

import numpy as np

Build a larger synthetic dataset to make IVF worthwhile
np.random.seed(42)
n = 100000
xb = np.random.random((n, d)).astype("float32")
xq = np.random.random((5, d)).astype("float32")

nlist = 256                       # number of Voronoi cells
quantizer = faiss.IndexFlatL2(d)  # assigns vectors to cells
index = faiss.IndexIVFFlat(quantizer, d, nlist, faiss.METRICL2)


print("Trained before training?", index.istrained)  # False
index.train(xb)                   # learn the cell centroids (k-means)
print("Trained after training?", index.istrained)   # True


index.add(xb)
print("Vectors:", index.ntotal)

index.nprobe = 1
D, I = index.search(xq, k=5)   # fast, lower recall

index.nprobe = 16
D, I = index.search(xq, k=5)   # slower, higher recall

# Ground truth from exact search
flat = faiss.IndexFlatL2(d)
flat.add(xb)
, gt = flat.search(xq, 5)

def recallatk(approxI, truthI, k=5):
    hits = 0
    for arow, trow in zip(approxI, truthI):
        hits += len(set(arow[:k]) & set(trow[:k]))
    return hits / (len(truthI)  k)

for nprobe in (1, 4, 16, 64):
    index.nprobe = nprobe
    , I = index.search(xq, 5)
    print(f"nprobe={nprobe:3d}  recall@5={recallatk(I, gt):.3f}")

This sweep is the workflow you should run on every dataset: it turns the abstract "speed vs recall" trade-off into concrete numbers for your data.

Memory compression with IndexIVFPQ

Flat storage keeps every vector at full precision: n d 4 bytes. One million 384-dim vectors cost about 1.5 GB. Product quantization (PQ) compresses each vector by splitting it into m sub-vectors and encoding each sub-vector with nbits bits, replacing it with the nearest entry in a learned codebook.

A vector then occupies only m * nbits / 8 bytes regardless of d. With m = 16 and nbits = 8 each vector is 16 bytes, roughly a 24x reduction versus float32 for 384 dimensions.

m = 16        # number of sub-quantizers; d must be divisible by m
nbits = 8     # bits per sub-quantizer code (8 => 256-entry codebooks)
nlist = 256

quantizer = faiss.IndexFlatL2(d)
index = faiss.IndexIVFPQ(quantizer, d, nlist, m, nbits)

index.train(xb)   # trains both the IVF centroids and the PQ codebooks
index.add(xb)

index.nprobe = 16
D, I = index.search(xq, 5)

The cost of PQ is approximation error: distances are computed against reconstructed (lossy) vectors, so recall drops compared with IVFFlat at the same nprobe. Constraints and tuning:

• d must be divisible by m. For d = 384, valid m values include 8, 12, 16, 24, 32, 48, 96.
• Larger m preserves more detail (higher recall) but uses more memory and is slower.
• nbits = 8 is by far the most common; it keeps codebook lookups cache-friendly.
• For an extra accuracy boost you can re-rank PQ candidates with exact distances using IndexIVFPQR or an IndexRefineFlat wrapper.

Use IVFPQ when the full-precision index does not fit in RAM. If it does fit, IVFFlat will give you better recall for the same query cost.

Graph-based ANN with IndexHNSWFlat

HNSW (Hierarchical Navigable Small World) builds a multi-layer graph where each vector links to its nearest neighbours. Search starts at the top layer and greedily walks toward the query, descending layers as it converges. HNSW typically delivers the best recall-at-low-latency of the CPU indexes and, unlike IVF, needs no separate training step.

M = 32                              # neighbours per node in the base layer
index = faiss.IndexHNSWFlat(d, M)   # also: indexfactory(d, "HNSW32")


index.hnsw.efConstruction = 80      # build-time search width (quality of graph)
index.add(xb)                        # building is the expensive part

index.hnsw.efSearch = 64            # query-time search width (recall vs speed)
D, I = index.search(xq, 5)

By default FAISS returns sequential positions (0, 1, 2, ...) in the I array. Real systems have their own IDs: database primary keys, document UUIDs, and so on. IndexIDMap wraps any index and lets you supply 64-bit integer IDs with addwithids, which are then returned by search.

base = faiss.IndexFlatL2(d)
index = faiss.IndexIDMap(base)

ids = np.array([1001, 1002, 1003, 1004, 1005, 1006], dtype="int64")
small = embeddings[:6]
index.addwithids(small, ids)


D, I = index.search(query, 3)
print("Your IDs:", I[0])   # e.g. [1002, 1004, 1001]

• IndexIDMap stores the mapping in a side table. It supports removeids, which is the standard way to delete vectors in FAISS.
• IndexIDMap2 additionally supports reconstruct(id) to retrieve the original vector by its ID.

# Removing vectors by ID
selector = faiss.IDSelectorArray(np.array([1002], dtype="int64"))
index.removeids(selector)

print("After removal:", index.ntotal)

Note that IVF indexes accept IDs through addwithids directly without an explicit wrapper, but IndexIDMap is the general mechanism for the flat and HNSW families.

# Persist
faiss.writeindex(index, "documents.faiss")

Restore in another process or run
index = faiss.readindex("documents.faiss")

print("Reloaded vectors:", index.ntotal)

The file is self-contained but does not store your raw text or metadata. A practical pattern is to keep the FAISS index alongside a separate store (a SQLite table, a Parquet file, or a JSON sidecar) keyed by the same integer IDs you fed into addwithids.

res = faiss.StandardGpuResources()          # manages GPU memory

cpuindex = faiss.IndexFlatL2(d)
cpuindex.add(xb)


gpuindex = faiss.indexcputogpu(res, 0, cpuindex)  # device 0
D, I = gpuindex.search(xq, 5)


Bring results back to CPU for saving (writeindex needs a CPU index)
finalcpu = faiss.indexcputoallgpus  # for multi-GPU; see below

backoncpu = faiss.indexgputocpu(gpuindex)

faiss.writeindex(backoncpu, "documents.faiss")

To shard an index across every visible GPU, use indexcputoallgpus.

gpuindex = faiss.indexcputoallgpus(cpuindex)

GPU notes worth remembering:

• Not every index type is GPU-supported; IndexFlat, IndexIVFFlat, and IndexIVFPQ are. HNSW is CPU-only.
• The GPU shines on large query batches and large datasets. For a handful of queries the CPU/GPU transfer overhead can dominate.
• You must move a GPU index back to CPU with indexgputocpu before calling writeindex.

End-to-end example: a searchable document index

This section ties everything together into a small, reusable component: embed documents, build an IVFFlat index with explicit IDs, persist it, reload it, and query. We keep document text in a parallel dictionary keyed by the same IDs.

import faiss
import numpy as np
from sentencetransformers import SentenceTransformer



class DocumentIndex:
    def init(self, modelname="all-MiniLM-L6-v2", nlist=64):
        self.model = SentenceTransformer(modelname)

        self.d = self.model.getsentenceembeddingdimension()
        self.nlist = nlist
        self.index = None
        self.docs = {}   # id -> original text

    def embed(self, texts):

        emb = self.model.encode(texts, converttonumpy=True).astype("float32")

        faiss.normalizeL2(emb)   # cosine similarity via inner product
        return emb

    def build(self, texts, ids):
        emb = self.embed(texts)

        quantizer = faiss.IndexFlatIP(self.d)
        ivf = faiss.IndexIVFFlat(quantizer, self.d, self.nlist,
                                 faiss.METRICINNERPRODUCT)

        ivf.train(emb)
        base = faiss.IndexIDMap(ivf)
        base.addwithids(emb, np.array(ids, dtype="int64"))

        self.index = base
        self.docs = {int(i): t for i, t in zip(ids, texts)}

    def search(self, query, k=5, nprobe=8):
        # nprobe lives on the wrapped IVF index
        faiss.extractindexivf(self.index).nprobe = nprobe

        q = self.embed([query])
        D, I = self.index.search(q, k)
        return [
            {"id": int(i), "score": float(s), "text": self.docs.get(int(i), "")}
            for s, i in zip(D[0], I[0]) if i != -1
        ]

    def save(self, path):
        faiss.writeindex(self.index, path)


    def load(self, path):
        self.index = faiss.readindex(path)


if name == "main":
    corpus = [
        "FAISS partitions vectors into cells with an inverted file index.",
        "Product quantization compresses embeddings to cut memory use.",
        "HNSW is a graph-based approximate nearest-neighbour algorithm.",
        "Normalize vectors to compute cosine similarity with inner product.",
        "Move an index to the GPU with indexcputogpu for throughput.",

        "Recall is measured against an exact flat-index ground truth.",
    ]
    ids = list(range(1, len(corpus) + 1))

    store = DocumentIndex(nlist=4)   # tiny nlist for a tiny demo corpus
    store.build(corpus, ids)
    store.save("documents.faiss")

    for hit in store.search("how do I reduce memory of my vectors", k=3):
        print(f"{hit['score']:.3f}  {hit['text']}")

The extractindexivf helper reaches through the IndexIDMap wrapper to set nprobe on the underlying IVF index, a detail that trips up many first-time users. For a tiny demo corpus we use a small nlist; on real data follow the sqrt(n) heuristic and tune nprobe with the recall sweep shown earlier.

• Always cast embeddings to contiguous float32 before adding them. Mismatched dtypes are the most common error.
• Build a ground-truth set with a flat index and measure recall@k whenever you switch index types or change parameters. Do not tune blind.
• Treat nprobe (IVF) and efSearch (HNSW) as runtime dials. You can change them per query to balance latency and quality without rebuilding.
• Train IVF and PQ on data drawn from the same distribution as the vectors you will add. Training on a different distribution quietly hurts recall.
• Keep your text and metadata in a separate store keyed by the same integer IDs you pass to addwithids. FAISS stores only vectors.
• Use IndexIDMap2 if you need to reconstruct original vectors or delete by ID.
• For deletions and frequent updates, remember FAISS is happiest as a mostly-static index. If your workload is write-heavy, a managed vector database may serve you better.
• Pin your FAISS version. Index file formats are stable across minor versions, but verify before upgrading in production.

• Flat indexes (IndexFlatL2, IndexFlatIP) are exact, simple, and the right baseline. Normalize vectors and use inner product for cosine similarity.
• IVF (IndexIVFFlat) accelerates search by probing only the nearest cells; nlist shapes the partition and nprobe trades speed for recall at query time.
• PQ (IndexIVFPQ) compresses vectors dramatically for memory-bound deployments, at the cost of some recall.
• HNSW (IndexHNSWFlat) gives excellent recall at low latency with no training, in exchange for high memory and append-only construction.
• IndexIDMap maps results to your own IDs and enables deletion; writeindex/readindex persist the entire index; indexcputogpu offloads search to the GPU.

Choose your index from your dataset size and latency budget, measure recall against an exact baseline, and tune the runtime knobs deliberately. That disciplined loop is what separates a FAISS deployment that scales from one that merely runs.