Vectors, Dot Products & Cosine Similarity

This post opens the day by establishing the three objects the entire topic rests on: the vector, the dot product, and cosine similarity. The goal is to replace symbol-pushing with intuition. By the end you should be able to look at two lists of numbers and reason about how aligned they are without reaching for a calculator.

The larger motivation is that these three small ideas are the machinery behind semantic search, embeddings, recommendation, and the attention mechanism in Transformers. Get the foundation concrete here, and the 'why it matters' and 'how it works' posts that follow will land much harder.

The single most useful reframe in this whole topic is that a vector is two things simultaneously. As a list of numbers, [3, 1], it's something you can store and compute with. As an arrow from the origin to the point (3, 1), it's something you can see — with a direction and a length.

Keeping both pictures in mind is what makes vector math intuitive instead of mechanical. The list view tells you how to calculate; the arrow view tells you what the calculation means. Every operation in this post has a numeric recipe and a geometric meaning, and fluency comes from holding them together.

Magnitude and direction are the two properties of the arrow view, and separating them is the key to understanding cosine similarity later. Magnitude is the arrow's length — computed as the square root of the sum of squared components, written ‖v‖. Direction is simply which way it points, independent of how long it is.

The crucial insight is that two vectors can point the same way while having very different lengths: [1, 1] and [5, 5] share a direction but differ in magnitude. This is exactly the situation where cosine similarity shines, because it measures only direction and ignores length — but to appreciate that, you first have to see the two properties as separate.

The vectors diagram makes the abstraction physical: two arrows, a and b, drawn from the origin. You can immediately see they point in somewhat different directions and have different lengths. The angle between them is the visual quantity that cosine similarity will eventually measure.

This 2-D picture is a crutch, and a good one. Real embeddings live in hundreds of dimensions where you can't draw anything, but the concepts — direction, length, angle between two arrows — extend unchanged. Train your intuition in 2-D and trust that the algebra carries it to 768-D without modification.

The dot product is the central operation. Its recipe is simple: multiply matching components and add the products into a single number. For [3,1]·[2,3] that's 3·2 + 1·3 = 9. What makes it powerful is the meaning of that number: it grows large and positive when the two vectors point the same way, lands at exactly zero when they're perpendicular, and goes negative when they point in opposing directions.

This is why the dot product is best thought of as an alignment score. It blends two influences — how aligned the directions are, and how long the vectors are — into one value. The next slides separate those two influences, but the core intuition to lock in now is: bigger dot product means more aligned (and/or larger) vectors.

This slide reveals the bridge between algebra and geometry that everything else depends on: a·b = ‖a‖‖b‖cos(θ). The same dot product you compute by multiplying and summing also equals the product of the two lengths times the cosine of the angle between the arrows.

This identity is doing enormous work. It says the purely numeric operation (multiply, sum) secretly encodes a geometric fact (the angle). If you rearrange it to solve for cos(θ), you get cosine similarity directly. Understanding that these two descriptions of the dot product are the same thing is the conceptual heart of the entire day.

Cosine similarity is what you get when you divide the dot product by both vectors' magnitudes. Algebraically, that division cancels the ‖a‖‖b‖ in the identity from the previous slide, leaving just cos(θ). The result is a clean score on a fixed scale: -1 means the vectors point in exactly opposite directions, 0 means they're perpendicular and share no direction, and 1 means they point identically.

The reason this matters is normalization. By dividing out the lengths, cosine similarity answers a focused question — 'do these point the same way?' — without being distracted by how big the vectors are. That size-independence is precisely why it's the default similarity measure for embeddings and text.

This code slide grounds all three objects in something runnable. The @ operator computes the dot product (9.0 here), np.linalg.norm computes each vector's length, and dividing gives the cosine similarity (about 0.707, which corresponds to a 45° angle). Seeing the numbers appear confirms the hand arithmetic from the earlier slides.

The habit worth forming is to compute these on tiny vectors whenever your intuition is shaky. A two-line check like this turns an abstract formula into a concrete result you can sanity-check, and it's the same code — just at larger scale — that powers production similarity search.

This slide delivers the practical punchline of the concept post: direction often matters more than raw distance. Consider a long, detailed document and a short note about the exact same topic. By straight-line (Euclidean) distance they're far apart, because one vector is much longer than the other. But they point in nearly the same direction, so their cosine similarity is high.

This is why text retrieval, semantic search, and most embedding systems prefer cosine over Euclidean distance. Document length, verbosity, and token count inflate magnitude without changing topic, and cosine deliberately ignores all of that. When you want to compare meaning rather than size, direction is the right signal.

This comparison consolidates the four ideas on one slide so the relationships are explicit. A vector is a point or arrow in space. Its magnitude is how long that arrow is. The dot product is raw alignment that is sensitive to both direction and size. Cosine is that same alignment expressed as pure angle, with size removed.

The progression matters: dot product and cosine are not different operations, they're the same operation viewed with and without magnitude. Seeing them as a pair — raw versus normalized — prevents the very common confusion of treating them as unrelated tools, and it sets up the mistakes post where mixing them up is the number-one error.

These five takeaways are the durable core to memorize before moving on. A vector is both a list and an arrow. The dot product is multiply-then-sum. The dot product also equals ‖a‖‖b‖cos(θ). Cosine similarity is the dot product divided by the lengths. And cosine always lands in [-1, 1], measuring direction alone.

If you remember only these five lines, you can reconstruct everything else in the day. They are the compact toolkit that the motivation, mechanics, code, and mistakes posts all build directly on.

The teaser points to the 'why it matters' angle. Now that the three objects are concrete, the next post shows where they actually live: semantic search, RAG retrieval, the attention mechanism, recommenders, and clustering. The aim is to make the motivation undeniable — these tiny operations aren't academic, they're the engine behind systems you use every day.