Distance Metrics: L1, L2, Cosine

This cover shifts from motivation to mechanics. Having established that L1, L2, and cosine differ and why that matters, this post derives the actual formulas and the geometry behind them. The promise is demystification: once the three formulas sit side by side, their differences become obvious rather than mysterious.

The two ingredients named in the hook — differences and dot products — are worth holding onto. L1 and L2 are built from per-coordinate differences; cosine is built from a normalized dot product. Every property explored in this post traces back to those two building blocks.

Here are the two geometric metrics stated precisely. L1 is the sum of absolute differences, Σ|xᵢ−yᵢ|. L2 is the square root of the sum of squared differences, √(Σ(xᵢ−yᵢ)²). They take the same inputs — the per-coordinate gaps — but apply different penalties.

The penalty difference is the whole story. L1 weights each gap linearly, so a coordinate off by 4 contributes 4. L2 weights each gap by its square, so that same coordinate contributes 16 before the final root. Consequently L2 reacts far more aggressively to any single large difference, making it more sensitive to outliers and large-magnitude features, while L1 spreads its attention more evenly and resists outliers better.

L1 and L2 are not two unrelated formulas but two points on a continuous family called the Minkowski distance: Lp(x,y) = (Σ|xᵢ−yᵢ|^p)^(1/p). Setting p=1 recovers Manhattan distance, p=2 recovers Euclidean, and as p grows toward infinity the formula approaches the Chebyshev distance, which is simply the single largest coordinate difference.

Seeing the family clarifies the trade-off as a dial. Small p (toward 1) spreads importance across all coordinates and resists outliers; large p (toward infinity) concentrates entirely on the worst coordinate. Most practical work lives at p=1 or p=2, but knowing the spectrum explains why those two are the standard endpoints people reach for.

This flow diagram lays the Minkowski family out as a dial from p=1 to p→∞. At p=1 you have Manhattan distance, summing absolute gaps. At p=2 you have Euclidean, the familiar straight line. Pushing p toward infinity gives Chebyshev, which ignores every coordinate except the one with the largest gap.

The value of seeing it as a single dial is conceptual economy: rather than memorizing three separate metrics, you understand one formula with a tunable exponent. Turning the dial smoothly trades robustness (low p) against sensitivity to the worst single difference (high p).

Cosine is defined as cos(x,y) = (x·y) / (‖x‖·‖y‖). The numerator, the dot product, measures how much the two vectors' magnitudes align. The denominator divides out both vectors' lengths, which is precisely the step that strips away all magnitude information and leaves only the angle between them.

It is essential to distinguish cosine similarity from cosine distance. Similarity ranges from −1 (opposite) through 0 (perpendicular) to 1 (identical direction). Distance is defined as 1 − similarity, so identical-direction vectors have distance 0 and perpendicular ones have distance 1. Keeping this conversion straight avoids a common sign confusion when plugging cosine into algorithms that expect a distance.

This snippet turns all the formulas into runnable functions, including the general Minkowski form. Each metric is a one- or two-line expression: L1 sums absolute differences, L2 roots the squared sum, Minkowski parameterizes the exponent, and cosine distance is one minus the normalized dot product.

Having them as actual functions invites experimentation. Call minkowski with p=1 and confirm it equals l1; call it with p=2 and confirm it equals l2; push p high and watch it approach the largest single coordinate gap. This is the most direct way to verify that the abstract Minkowski family really does collapse to the familiar metrics at its named values.

The unit ball — the set of all points exactly distance 1 from the origin — reveals each metric's personality geometrically. For L1, that set is a diamond (a square rotated 45 degrees) with its corners on the axes, because to stay at total absolute distance 1 you trade off between coordinates linearly. For L2, it is a perfect circle, identical in every direction, which is why L2 is rotation-invariant.

These shapes are not just decorative. The diamond shape of L1 is what gives Lasso regression its tendency to drive coefficients exactly to zero (the corners touch the axes). The circular symmetry of L2 is why Euclidean distance behaves the same regardless of how you rotate your coordinate system. The geometry of the unit ball directly explains each metric's behavior.

This is one of the most useful relationships in the whole topic: on unit-length vectors, L2 and cosine are the same comparison. If you normalize every vector to length 1, then the squared Euclidean distance between two of them equals exactly 2 − 2·cos(θ), a strictly decreasing function of the cosine similarity.

The practical consequence is profound. Ranking normalized vectors by ascending Euclidean distance produces the identical ordering as ranking by descending cosine similarity. This is why many vector databases internally normalize embeddings and then use fast Euclidean (or dot-product) search to serve cosine queries — they get cosine semantics with L2 machinery. Understanding this equivalence demystifies a lot of production retrieval design.

This pipeline diagram traces cosine's construction in three stages: start with the dot product of the two vectors, divide by the product of their magnitudes to cancel out length, and arrive at a pure measure of angle. Each stage strips away a little more magnitude information until only direction remains.

Seeing it as a pipeline makes clear that cosine is not a fundamentally different kind of operation from the dot product — it is the dot product with magnitude normalized away. That framing connects cosine to the attention mechanisms and similarity scores used throughout modern ML, which are frequently just dot products on already-normalized vectors.

A subtle but important point: not all three are technically "metrics" in the mathematical sense. A true metric must be non-negative, symmetric, zero only for identical points, and obey the triangle inequality (the direct path is never longer than a detour). L1 and L2 satisfy all four conditions and are genuine metrics.

Cosine distance is non-negative and symmetric and is zero for identical-direction vectors, but it violates the triangle inequality. That makes it a dissimilarity measure rather than a strict metric. For ranking and retrieval this distinction is harmless, but for algorithms that depend on the triangle inequality — certain metric-tree indexes and some theoretical guarantees — it matters, as the closing mistake slide warns.

These five lines compress the entire mathematical post into a recall sheet. L1 is the sum of absolute differences with a linear penalty; L2 is the root of squared differences with a squared penalty; the Lp dial spans p=1, 2, and infinity; cosine is the dot product over the lengths; and normalizing makes L2 ranking equivalent to cosine ranking.

The final bullet is the one most worth memorizing, because it ties the geometric and angular worlds together and explains a large fraction of how real retrieval systems are built. If a reader retains only that line, they understand why normalization is the bridge between the two families of metrics.

The closing mistake flags the practical danger of cosine's broken triangle inequality. Algorithms such as ball trees, certain pruning bounds, and some clustering correctness proofs assume the triangle inequality holds; feeding them raw cosine distance can produce incorrect or unstable results.

The fix is to use a true metric when one is required. Either convert to angular distance — arccos of the similarity, which restores the triangle inequality — or, more commonly, normalize your vectors and use L2, which is a proper metric and ranks identically to cosine.

The CTA closes the math post and points to the hands-on payoff. With the formulas, the Minkowski dial, and the normalization identity established, the natural next step is to run them and watch the theory play out on real numbers.

The next post is a runnable notebook: implement all three metrics from scratch, verify against scikit-learn, and build a tiny k-NN classifier that flips its prediction purely from a metric swap. It is where the math stops being symbols and becomes behavior you can see and modify.