Vector Search Basics

This is the engine-room post, and the cover sets expectations honestly: the magic of 'find similar' reduces to three concrete steps plus one clever shortcut. Demystifying it this way respects the reader and prepares them to reason about performance rather than treat the system as a black box.

The three verbs — embed, measure, search smartly — map directly onto the three things that can go wrong and the three things you can tune. Establishing that vocabulary up front makes every subsequent slide easier to slot into a coherent picture.

The three-step loop is the spine of the entire post. Embedding turns data into vectors using a model; indexing organizes those vectors so they can be queried quickly; querying embeds the incoming question and finds its nearest neighbors. Every vector search system, from a 30-line script to a planet-scale service, is some elaboration of these three steps.

Stating them as a numbered list gives the reader a checklist they can map onto any tool they encounter. When they later meet FAISS, Pinecone, or pgvector, they will recognize these same three operations wearing different API names.

This pipeline diagram zooms into the indexing side, adding the chunking step that the simple loop glosses over. Real documents are too long to embed as a single vector, so they are split into chunks first; each chunk is embedded, then upserted into the index. The 'query' stage at the end shows where kNN search re-enters.

Separating chunk, embed, upsert, and query as distinct stages matters because each is a place to make decisions and mistakes. Chunk size, embedding model, and index parameters are all independent knobs, and seeing them as separate stages prepares the reader for the tuning discussion to come.

Distance metrics are where many newcomers stumble, so this slide lays out the three common choices. Cosine similarity measures the angle between vectors, ignoring magnitude — ideal for text, where direction encodes meaning and length is noise. Dot product folds in magnitude, which matters when vectors are not normalized. Euclidean (L2) measures straight-line distance.

The practical guidance is the last bullet: cosine dominates for text embeddings, and many libraries achieve it by normalizing vectors and then using dot product, which is mathematically equivalent. Knowing this prevents the confusion of seeing 'inner product' in code that is really computing cosine similarity.

This code slide makes cosine similarity tangible by implementing it from scratch in three lines: dot product divided by the product of magnitudes. Seeing the formula as runnable code, rather than notation, helps it stick — and clarifies why normalizing vectors first lets you skip the denominator.

The printed interpretation matters as much as the code: a score near 1.0 means near-identical direction (and thus meaning), while 0 means orthogonal and unrelated. Anchoring the number to a meaning prevents readers from later treating raw scores as mysterious, which is exactly the trap the mistakes post warns against.

This slide confronts why we cannot simply compute exact distances to every vector. The arithmetic is brutally clear: ten million documents means ten million distance calculations for every single query, repeated for every user. That is accurate but far too slow for interactive applications.

The resolution is a deliberate trade: give up a sliver of accuracy to gain orders of magnitude in speed. Naming this approximate nearest neighbor, or ANN, and framing it as an intentional engineering choice prepares the reader to think about recall as a tunable quantity rather than a defect.

HNSW is the dominant ANN algorithm, so explaining it in plain words is high value. The mental image is a multi-layer graph: the top layer is sparse with long-range links, letting you jump across the space quickly; lower layers are denser, letting you refine toward the true nearest neighbors. You greedily hop toward the query at each layer.

The punchline is efficiency: instead of examining millions of vectors, you touch a few hundred nodes and still land on near-exact results. Understanding the layered-graph intuition turns HNSW from a scary acronym into a navigable idea and explains why it is fast without being magic.

This stack diagram visualizes the HNSW layers described in the previous slide. The top layer offers a few long-range hops to get you into the right neighborhood; the middle layers narrow the region; the base layer contains every vector and is where final refinement happens. The reader can trace a query descending through the levels.

Pairing the verbal explanation with a layered picture reinforces the key insight: search is hierarchical, coarse-to-fine. That structure is exactly what avoids the brute-force cost, and seeing it spatially makes the speed-versus-recall tradeoff in the next slides intuitive.

This FAISS snippet shows ANN in practice with only a few lines. IndexHNSWFlat builds the layered graph; the parameter 32 sets the graph degree, which trades memory and build time for recall. Adding the document vectors constructs the index, and search returns the top-k ids and their distances.

Using real, runnable code here matters because it connects the abstract HNSW explanation to an API the reader will actually call. The comment on the graph-degree parameter previews the tuning theme: these numbers are dials, and the next slides quantify what turning them does.

This bars diagram quantifies the central tradeoff so it stops being hand-wavy. Brute force gets perfect recall but is slow. HNSW with a high ef_search setting reaches near-exact recall while being dramatically faster. Drop ef_search and you get the fastest queries at the cost of some recall.

Showing recall as a bar chart drives home that this is a continuous dial, not a binary choice. Most production systems live in the high-recall, low-latency sweet spot, and visualizing the curve helps readers understand that they are choosing a point on it rather than accepting a fixed behavior.

The recall-versus-latency slide gives the reader the precise vocabulary to tune their system. Recall is defined concretely: the fraction of the true top-k results that your approximate search actually returned. Naming ef_search and graph degree as the primary knobs tells them exactly what to adjust.

The practical target — around 95% or higher recall at single-digit milliseconds — gives a realistic benchmark rather than leaving them to guess. This grounds the abstract tradeoff in numbers they can aim for and measure against, which is what separates tuning from guessing.

The closing card promises a full runnable example next, which is the natural progression after explaining the mechanics. Having seen embed, measure, and search smartly in theory, the reader is primed to see all three executed end to end in code.

The save prompt suits a technical reference post: the distance-metric and HNSW explanations here are exactly the kind of thing engineers look up again when configuring an index, so encouraging a save matches real usage.