FAISS Deep Dive

This post opens the FAISS deep dive by fixing what FAISS is in your head before any mechanics arrive. The name — Facebook AI Similarity Search — tells you the lineage and the job: it came out of Meta's research org and it does one thing, similarity search over dense vectors, exceptionally well.

The single most important framing is that FAISS is a library, not a database. It links into your Python or C++ process and operates entirely in memory there. There is no server to talk to, no port to open, no query language. That distinction shapes every other decision you'll make with it.

This definition slide pins down the literal job. You hand FAISS a matrix of float32 embeddings and, for any query vector, it returns the k nearest neighbors by distance. The contrast with a database is the headline: it runs inside your process, with no server, no network hop, and no query language to learn.

That in-process design is what gives FAISS its speed and its constraints in one stroke. There's nothing between your code and the distance math, which is why it's fast — and why everything a database would normally hand you (storage, filtering, an API) is absent unless you build it.

An index is the only object you really need to understand to start. You construct one of a specific type (the type encodes the speed/accuracy/memory trade-off), you call add() to load your vectors into it, and you call search() to ask for the k nearest neighbors of one or more query vectors.

The reason 'learn the index zoo' is the recurring advice is that the API surface is tiny but the type catalogue is large. IndexFlatL2, IndexIVFPQ, IndexHNSWFlat, and dozens of composites all share the same add/search interface — so once you understand what each type does internally, switching between them is a one-line change.

FAISS deals in raw float32 arrays, almost always passed as NumPy matrices of shape (n, d): n vectors, each d dimensions long. It does not embed anything for you and it does not know what your vectors mean. A 384-dim vector from all-MiniLM and a 1536-dim vector from OpenAI are both just rows of floats to it.

This is why the upstream embedding step is your responsibility and your risk. FAISS will faithfully index whatever you hand it. If the numbers are wrong — wrong model, wrong normalization, wrong dimension — the search results are wrong too, and FAISS won't warn you.

The flat-versus-approximate split is the conceptual fork that the rest of the series hangs on. A flat index is exhaustive: every query is compared against every stored vector, giving exact, ground-truth nearest neighbors. It's perfect for small datasets and as a correctness baseline.

Approximate nearest neighbor (ANN) indexes give up the guarantee of finding the true nearest neighbor every time in exchange for enormous speedups. They only examine a promising subset of vectors. At a few thousand vectors the difference is irrelevant; at a hundred million it's the difference between a working product and a timeout. FAISS exists primarily to make the approximate case fast and tunable.

This diagram nests the four things you actually manipulate. The index is the top-level object. Inside it live your vectors — the float32 matrix you added. Each vector has an id, which by default is just its insertion position (0, 1, 2, …) unless you wrap the index to supply your own.

The distance metric sits at the bottom because it's a property of the index you chose at construction time, not something you pass per query. FAISS natively offers L2 (Euclidean) and inner product; cosine is achieved by normalizing and using inner product. Picking the metric to match how your embedding model was trained is a decision you make once, up front.

This snippet is deliberately the whole story in five lines, because the API really is that small. You import faiss and numpy, choose a dimension, construct a flat L2 index, add a thousand random vectors, and search with five queries for their four nearest neighbors each.

The return value is the pattern you'll see everywhere: two arrays, D and I, both shaped (n_queries, k). D holds the distances and I holds the neighbor ids. Note the .astype('float32') — FAISS is strict about dtype, and forgetting this is one of the first errors newcomers hit. Internalize this shape and the rest of FAISS is variations on it.

Calling FAISS 'a library, not a database' isn't pedantry — it's the design decision that explains its strengths and its gaps. There's no persistence you didn't write (you call write_index/read_index yourself), no authentication, no replication, no metadata filtering, and no network endpoint.

Far from a weakness, this minimalism is why FAISS is so fast and why it's embedded inside higher-level systems. Vector databases like Milvus and (historically) others wrap FAISS to get its index implementations while adding the storage, API, and operational layers on top. When you use FAISS directly you're choosing to own those layers yourself in exchange for maximum control and speed.

The comparison makes the trade explicit. On the left, FAISS gives you a library in your process, raw vectors and ids, and the widest selection of fast index types — but you handle storage and operations. On the right, a managed vector database gives you a running service with payload storage, metadata filters, auth, and an API — at the cost of fewer low-level knobs and an external dependency.

Neither column is 'better'. If you need filtered search, live updates with auth, and minimal ops, reach for Qdrant or Pinecone. If you need the absolute fastest ANN over a mostly-static billion-vector corpus and you're comfortable writing the surrounding service, FAISS is hard to beat — and it's often what those databases are running underneath anyway.

These five lines are the mental model to walk away with. FAISS is a similarity-search library in C++ with Python bindings. Its single abstraction is the index. You add vectors and search for the k nearest. Flat indexes are exact; ANN indexes are fast and approximate. And there is no server, no metadata, and no ops baked in — that's intentional.

If you remember nothing else from this post, remember that FAISS answers exactly one question — 'which stored vectors are closest to this one?' — and that everything else in a real system (embedding, storage, filtering, serving) is your job to assemble around it.

The cover and CTA bookend the concept post. Having established what FAISS is, the natural next question is why it's worth the trouble of owning all that surrounding machinery. The next post tackles the stakes: the speed and scale that make brute-force search infeasible and FAISS necessary, and the specific trade-offs you sign up for in return.