Vectors, Dot Products & Cosine Similarity

This is the hands-on post: we build a minimal but complete semantic search engine using nothing but NumPy. The point is demystification. Vector databases like FAISS and Pinecone, and every RAG pipeline, wrap this functionality in polished APIs, but underneath, search is just embedding, normalizing, scoring by cosine, and ranking — exactly the operations from the mechanics post.

By building it by hand and tracking shapes at each step, you'll see there's no hidden magic. If you can read and run these snippets, you understand what a production vector search actually computes, which makes the whole RAG and embedding ecosystem far less mysterious.

We're building a search with four moving parts: a query vector, a matrix of document vectors, a cosine score for each document, and a ranked top-k list. The only operations involved are matrix multiplication, vector norms, and argsort. That is genuinely the entire core of a vector search engine.

Keeping it tiny is intentional. With five documents and four dimensions, you can inspect every number and verify the result by hand. The structure scales without conceptual change — real systems just use millions of higher-dimensional vectors and add an index to find the top matches faster, but the scoring logic is identical.

Step one sets up the data. docs is a (5, 4) matrix: five documents, each represented by a four-dimensional embedding. The query is a single (4,) vector in the same space. We seed the random generator for reproducibility and use random arrays as stand-ins for real embeddings.

The shapes are the thing to watch. The query and documents must share dimensionality — both four-dimensional here — or the later dot products won't be defined. Printing docs.shape and query.shape up front is the simple discipline that catches the most common setup error before it cascades into a confusing failure downstream.

Step two implements cosine similarity for a single query-document pair, reusing the exact formula from the mechanics post: the dot product divided by the product of the two norms. Calling it on the first two documents shows scores in the [-1, 1] range, where higher means more similar to the query.

This pairwise version is correct but doesn't scale — calling it in a Python loop over a million documents would be slow. Seeing it here first establishes the ground truth, so when the next step vectorizes the computation, you can confirm the fast version produces the same scores as this transparent one.

Step three vectorizes the whole search. First we normalize the query and every document row to unit length using norm with axis=1 and keepdims=True so the division broadcasts correctly across rows. Once every vector has length 1, cosine similarity reduces to a plain dot product, so the single matmul D @ q computes all five scores at once, producing a (5,) array.

This is the key performance idea. Instead of looping and computing norms repeatedly, you normalize once and let one matrix-vector product do all the work. The shape arithmetic — (5,4) @ (4,) → (5,) — confirms you get exactly one score per document, and it's the same operation a real vector database runs at scale.

Step four turns scores into a ranked result. np.argsort returns the indices that would sort the scores in ascending order; we reverse with [::-1] to get descending (best first) and slice the top k. The loop then prints each result's rank, document index, and score.

That ordered list of indices is the search result — it's literally what 'the top 3 matches' means. The subtle point worth remembering is that argsort is ascending by default, so forgetting to reverse it would return the least similar documents. That off-by-direction bug is common enough that it reappears in the mistakes post.

The pipeline diagram shows the four stages of search end to end: embed text into vectors, normalize them to unit length, score with the cosine matmul D @ q, and rank with argsort to take the top-k. It's the mental model to carry whenever you think about vector search.

Seeing the code and the pipeline together connects the concrete NumPy lines to the conceptual flow. Every vector search system, no matter how large, is some version of these four stages. The engineering around them — indexing, sharding, approximate nearest neighbors — exists to make the score-and-rank steps fast at scale, not to change what they compute.

This slide explains the normalization trick that makes the whole thing efficient. Cosine similarity is the dot product divided by both lengths. If you pre-scale every vector to length 1, then both lengths are 1, the division disappears, and cosine is simply the dot product. So normalizing the entire matrix once converts the search into a single fast matmul.

This is exactly why many vector databases ask you to store normalized vectors, or offer a 'cosine' mode that assumes unit length. The optimization isn't a hack — it's the mathematically equivalent reformulation that lets hardware do one big matrix multiply instead of many separate norm-and-divide operations.

The flow diagram tracks the shapes through the scoring step: D is (5,4), q is (4,), the inner dimensions (both 4) match and cancel, and the result is (5,) — one score per document. This is the inner-dimension rule from linear algebra applied to search.

The habit this reinforces is predicting output shapes before running code. If you expected (5,) scores and got something else, you'd know immediately that a shape was off — perhaps the query wasn't the right dimensionality, or the matrix was transposed. Shape literacy is the fastest way to catch search bugs before they reach your results.

This slide makes the payoff explicit: swap the random arrays for real embeddings and this exact code becomes a working semantic search. Run your documents through a model like sentence-transformers or an embedding API, store the resulting vectors as the matrix, embed incoming queries the same way, and the scoring and ranking code is unchanged.

What tools like FAISS and Pinecone add is indexing to scale to millions of vectors with approximate nearest-neighbor search, plus persistence and serving infrastructure. They do not change the fundamental scoring math — it's the cosine ranking you just wrote. Understanding the NumPy version transfers directly to using and debugging those systems.

These checklist items are the practical habits that prevent the most common search bugs. Confirm the query and documents share dimensionality before scoring. Normalize with axis=1 and keepdims=True so the division broadcasts per row. Remember that once vectors are unit length, cosine equals the dot product. Recall that argsort is ascending, so reverse it for top results. And print shapes after each step.

Following this checklist turns building search from guesswork into a deterministic process. Most 'my search returns garbage and I don't know why' situations dissolve into one of these five checks, usually normalization or the argsort direction.

The teaser points to the final angle of the day: common mistakes. Now that you've built a working semantic search and seen how cleanly the scores flow when everything is correct, the next post catalogs the specific errors that quietly corrupt similarity results — skipped normalization, distance-versus-similarity confusion, zero vectors, and mismatched models — so you can avoid the silent failures they cause.