FAISS Deep Dive

This is the hands-on post, and the cover sets the contract: you'll go from raw sentences to a saved, queryable index and back to the actual text, with every step copy-pasteable. The emphasis on 'type it out, don't just read' is deliberate — FAISS's gotchas (dtype, training, id mapping) are the kind you only internalize by hitting them once yourself.

Step zero is the embedding step that FAISS depends on but doesn't do. We install faiss-cpu and sentence-transformers, load the small all-MiniLM-L6-v2 model (384 dimensions, fast, good enough to demonstrate), and encode four short documents. The result, xb, is a (4, 384) array.

The .astype('float32') is not optional decoration — FAISS requires float32, and sentence-transformers can return float32 already, but being explicit guards against subtle dtype drift. From here on, every vector that touches FAISS — documents and queries alike — comes from this same model so dimensions and semantics line up.

Step one builds the simplest possible index and proves it works. IndexFlatL2(d) is exact and needs no training, so you can add() immediately. index.ntotal confirms four vectors are stored. Then we embed a brand-new query sentence — note it never appeared in the documents — and search for its two nearest neighbors.

The returned I array gives positions into our document list. The comment shows the intuitive result: 'a small cat sleeping' lands nearest 'kittens nap' and 'cats purr', because the embedding model places semantically related sentences close together. This is the payoff that makes vector search feel like magic, demonstrated in eight lines.

Step two handles the thing beginners constantly trip over: FAISS gives you back ids and distances, never your data. The I array contains positions like 3 and 0; it's on you to translate those into the original sentences. Here we zip the ids and distances together and index back into the docs list to reconstruct human-readable results.

This is the single most important operational habit with FAISS. The index and your id→text mapping are two halves of one system. If they ever drift apart — because you reordered, rebuilt, or reloaded one without the other — every result silently points to the wrong content. Building the mapping step explicitly here is meant to burn that lesson in early.

Step three upgrades to an approximate index to show the training requirement in action. We build an IndexIVFPQ with a flat quantizer, a tiny nlist=2 (real systems use roughly sqrt(n) cells), 8 PQ subvectors, and 8 bits per code. The pivotal line is ivf.train(xb): unlike the flat index, IVF and PQ must learn their centroids and codebooks from data before you can add anything.

We then add the vectors and set nprobe=2, which for this toy two-cell index means searching both cells — effectively exhaustive, so recall stays high. In a real index with hundreds of cells, nprobe would be the dial you tune. The search returns the same (D, I) shape as the flat index, underscoring that switching index types is mostly transparent to your calling code.

Step four covers persistence, which FAISS supports but doesn't automate. write_index serializes the index to a file; read_index loads it back, potentially in a completely different process. This is how you avoid re-embedding and re-building on every startup.

Two subtleties are flagged in the code. First, runtime parameters like nprobe are not always restored on load, so you set them again after read_index. Second — and this is the recurring theme — you must separately persist the docs list (or whatever maps ids to content). The .index file holds vectors and structure but not your text. Saving one without the other gives you a working index that points to nothing meaningful.

The pipeline diagram compresses the whole walkthrough into four stages: embed text into vectors, build an index (flat to start, IVF/PQ at scale), search for k nearest, then map ids back to text and save to disk. It's the loop you'll reproduce in essentially every FAISS-backed feature you build.

Keeping this shape in mind helps you place any FAISS code you encounter: whatever exotic index or tuning is involved, it still slots into embed → index → search → map+persist.

This gotcha list is the distilled pain of working with FAISS. Vectors must be float32 and C-contiguous or you get errors or silent copies. IVF and PQ indexes must be trained before adding. search() returns ids, not data, so the id→text map is mandatory. nprobe resets to its default after read_index, so re-set it. And calling train() on a flat index is harmless-looking but pointless — flat indexes ignore training, which can confuse people into thinking training 'worked' everywhere.

Each of these has cost someone hours. Reading them here is cheaper than rediscovering them under deadline.

This snippet addresses the most common metric confusion as a standalone fix because it bites so often. FAISS has no cosine-similarity index. The standard solution is to L2-normalize every vector and use IndexFlatIP (inner product): on unit-length vectors, inner product equals cosine similarity.

The critical detail is that you must normalize both the database vectors and every query. Normalizing one side but not the other corrupts the ranking subtly enough that it can pass a casual eyeball test while being wrong. With normalization done correctly, the scores fall in [-1, 1] and higher means more similar — the opposite ordering from L2 distance, where smaller means closer.

This final tips slide names the reusable loop independent of the demo specifics: embed your documents once and keep the text list; pick an index (flat to start, IVF/PQ when you scale); train if the index type requires it, then add; embed the query with the exact same model and search; and map ids back to text while persisting the index and the map together.

The phrase 'the loop you'll always write' is the point — FAISS code looks intimidating in isolation but reduces to this same five-beat rhythm every time.

The cover and CTA frame this as the practical chapter and point to its mirror image. You've now built FAISS the right way; the next post catalogues the wrong ways — the silent, confident failures like forgotten training, cosine/L2 confusion, untuned nprobe, and lost id mappings — so you can recognize them before they ship.