Vectors, Dot Products & Cosine Similarity

This post shifts from motivation to mechanics. The promise is that the dot product, the magnitude, and cosine similarity are each just a few arithmetic steps, and that doing them by hand once on small vectors makes the formulas permanent. We deliberately use the same example vectors throughout so the numbers connect across slides.

The teaching strategy mirrors how the topic actually clicks for people: compute the pieces concretely, then connect them back to the geometry — the angle between two arrows. By the end you should be able to take any two short vectors and produce their cosine similarity with a pencil, which is the surest sign you truly understand it.

Step one is the dot product, the atom everything else builds on. The recipe: pair up matching components, multiply each pair, and sum the products into one scalar. For a = [1, 2, 2] and b = [2, 0, 1], that's 1·2 + 2·0 + 2·1 = 2 + 0 + 2 = 4.

Notice how each component contributes: where both vectors are large and positive, the product adds a lot; where one is zero, that pair contributes nothing. The dot product is a component-by-component measure of how much the two vectors reinforce each other. This single number is the raw material; the remaining steps just rescale it into an angle.

The trace diagram walks the dot product line by line so there's no hand-waving. It lists the two input vectors, forms the elementwise products, sums them step by step, and arrives at 4. Seeing every intermediate value turns the formula into a concrete, checkable procedure.

This step-by-step habit is also a debugging tool. When a similarity score looks wrong in real code, reproducing the dot product by hand on a tiny example quickly tells you whether the problem is the math, the vector contents, or a shape issue. The trace is a template you can reuse any time a number surprises you.

Step two computes magnitude, the length of each arrow. The formula is the Pythagorean theorem generalized to any number of dimensions: take the square root of the sum of the squared components. For a, ‖a‖ = √(1² + 2² + 2²) = √9 = 3 exactly. For b, ‖b‖ = √(2² + 0² + 1²) = √5 ≈ 2.236.

These lengths are the scales we will divide out in the cosine step. The reason cosine similarity ignores size is precisely that it divides by these magnitudes. Computing them explicitly here makes the normalization in the next step concrete rather than mysterious — you can see exactly what's being cancelled.

Step three states the identity that ties algebra to geometry: a·b = ‖a‖‖b‖cos(θ). The dot product equals the product of the two lengths times the cosine of the angle between the vectors. This is the linchpin of the entire topic.

The practical consequence is that you can solve for the angle. You already computed the dot product (4) and both lengths (3 and ≈2.236); the only unknown left in the identity is cos(θ). Rearranging to isolate it is exactly what the cosine-similarity formula does. Seeing it as 'rearrange the bridge identity' makes cosine feel inevitable rather than arbitrary.

Step four finishes the calculation: divide the dot product by the product of the magnitudes. cos(θ) = 4 / (3 · 2.236) ≈ 4 / 6.708 ≈ 0.596. That value is the cosine similarity of a and b, corresponding to an angle of about 53°.

The key conceptual point is what the division accomplishes. Multiplying the two lengths together and dividing by them cancels all magnitude information, leaving a number that depends only on the angle between the vectors. A long version of a and a short version of a would give the identical cosine, because direction is all that survives the division.

The pipeline diagram condenses the whole procedure into four labeled stages: compute the dot product (4), compute the two norms (3 and 2.236), divide, and read off cos(θ) ≈ 0.596. It's a visual recipe you can run in your head for any pair of vectors.

Laying the steps out as a pipeline emphasizes that cosine similarity is a composition of simpler operations, not a monolithic formula to memorize. If you ever forget the exact expression, you can rebuild it: dot product over the product of the lengths. The pipeline is the mnemonic.

This slide explains how to interpret the score, which is where intuition pays off. A cosine of 1 means the vectors point in exactly the same direction — maximum similarity. 0 means they're perpendicular and share no direction — unrelated. -1 means they point exactly opposite. Our example's 0.596 sits in the 'somewhat similar' middle.

A useful real-world note: most embedding similarities cluster between 0 and 1 rather than spanning the full range, because trained embeddings tend to occupy a general region of the space and rarely point in truly opposite directions. So in practice you're often comparing scores like 0.6 versus 0.8, and the relative ordering matters more than the absolute value.

This code slide verifies the entire hand calculation. The dot product comes out to 4, the norms to 3.0 and approximately 2.236, and the final cosine to 0.596 — matching every step computed manually. Running it closes the loop between paper arithmetic and code.

The broader lesson is to trust but verify. Doing a calculation by hand builds understanding; confirming it in two lines of NumPy builds confidence that your mental model and the library agree. When they diverge on real data, that mismatch is exactly the signal that something — a shape, a normalization, a zero vector — is off.

This slide explains the special case that makes the angle interpretation click: when two vectors are perpendicular (90°), cos(90°) = 0, and indeed their dot product is exactly zero. Geometrically, the positive and negative contributions of the component products cancel out perfectly.

The term for this is orthogonal, and it literally means 'no shared direction.' A zero dot product is the algebraic signature of independence — the vectors point in completely unrelated directions. This is why orthogonality shows up so often as a desirable property: orthogonal features or basis vectors carry non-overlapping information, and the dot product is how you detect it.

These five rules are the portable summary of the mechanics. Dot product: multiply matching pairs, then sum. Norm: square root of the sum of squares. Cosine: the dot product divided by the product of the two norms. The range is always [-1, 1]. And a value of 0 means perpendicular, hence unrelated.

With these five lines you can compute and interpret any cosine similarity by hand. They are the practical recipe the upcoming code post scales up into a working semantic search engine.

The teaser leads into the code-heavy post. Now that you can compute cosine similarity by hand and understand each piece, the next post assembles these operations into a tiny semantic search built entirely in NumPy — embedding vectors, normalizing, scoring by cosine, and ranking the top results — so you see the math doing real work.