Distance Metrics: L1, L2, Cosine

This cover reframes the metric from a math curiosity into a business-critical decision. The hook lists three concrete failure modes — bad search, wrong clusters, mismatched recommendations — and emphasizes that none of them throw an error. That silence is the recurring theme of the post.

The goal here is to convince a practitioner that the metric is not a tunable detail to leave at its default, but the lens through which every similarity system perceives the data. Change the lens and you change every result, often without any visible warning.

Nearly every similarity-based algorithm is, at its core, a ranking by distance. k-NN looks at the labels of the closest training points. k-means assigns each point to the nearest centroid. A vector database returns the embeddings closest to a query. A recommender surfaces items most similar to what a user liked. In every case, "closest" is defined entirely by the chosen metric.

This means the metric is not one component among many — it is the foundation the whole system stands on. Swapping L2 for cosine doesn't tweak the results at the margin; it can completely reorder which items count as neighbors. And because the code still runs and still returns plausible rankings, the change is invisible unless you specifically evaluate for it.

The deepest way to think about metric choice is to ask a single question: does the size of a vector mean something? L1 and L2 answer yes — a vector twice as long sits genuinely far from the original in space. Cosine answers no — it normalizes away length and keeps only direction.

So choosing a metric is really choosing what role magnitude plays. If your vectors' lengths encode real information (total spend, signal strength, raw counts), discarding it with cosine throws away signal. If the lengths are incidental (document length, embedding norm), keeping it with L2 injects noise. The right metric is the one whose blind spot matches what you genuinely don't care about.

This comparison gives a practical decision aid. Reach for L2 (or L1) when your coordinates are real, comparable measurements, when magnitude is meaningful, when features share a scale, and when you are in relatively low dimensions where Euclidean geometry still behaves intuitively.

Reach for cosine when you are working with text or learned embeddings, when only direction carries meaning, when vector lengths vary widely for reasons you don't care about, and when you are in high dimensions. These two columns cover the large majority of real decisions; the rest of the post explains the reasoning behind each row.

Text is the canonical case for cosine, and understanding why illuminates the whole idea. Represent two documents about the same subject — a long article and a short note — as bag-of-words or TF-IDF vectors. They point in nearly the same direction because they use the same vocabulary, but the article's vector is much longer simply because it contains more words.

Under cosine, the length difference vanishes and the two are judged almost identical, which matches our intuition that they are "about the same thing." Under L2, the sheer length gap makes them look far apart, conflating topic with verbosity. This is why TF-IDF retrieval, embedding search, and RAG pipelines overwhelmingly default to cosine: they want topical similarity, not length similarity.

This snippet makes the text argument tangible with three-dimensional vectors. The short note [1,1,0] and the long article [5,5,0] point in exactly the same direction — the long one is just five times the length. Cosine returns 1.0, declaring them topically identical, while L2 returns about 5.66, treating them as far apart purely because of length.

Running this is the fastest way to feel the difference. The two vectors are the "same topic" by any sensible definition, and only cosine captures that. Change the second vector's direction even slightly and watch cosine finally react, confirming that it responds to angle and angle alone.

This is the most common and most damaging L2 failure: unscaled features. Euclidean distance sums squared differences across all columns, so a column measured in large units (income around 50,000) contributes squared differences in the billions, while a column in small units (age around 40) contributes squared differences in the hundreds.

The result is that income alone determines the distance and age is effectively ignored, even if age is the more predictive feature. Your "Euclidean distance" has silently collapsed into a distance in dollars. The fix is always to standardize (zero mean, unit variance) or normalize features before any L1/L2-based algorithm — a step that is easy to forget and produces no error when omitted.

This bar chart visualizes the scaling failure concretely. With unscaled features, the squared income gap contributes the overwhelming majority of the total L2 distance, while the squared age gap is a rounding error by comparison. The bars show one feature swallowing the metric whole.

The lesson the visual drives home is that L2 is not unit-agnostic. It treats one unit of every feature as equally important, which is only sensible if the features are on comparable scales. Standardization is what restores that comparability, giving each feature a fair vote in the final distance.

High dimensionality is a subtler but equally important reason to prefer cosine. As the number of dimensions grows into the hundreds or thousands, a counterintuitive phenomenon called the curse of dimensionality sets in: the L2 distances between all pairs of points converge toward nearly the same value. The contrast between "near" and "far" erodes, and the very notion of a meaningful nearest neighbor weakens.

Cosine similarity is more robust in these regimes because it compares direction rather than absolute position, and high-dimensional embedding spaces are typically organized so that direction carries the semantic meaning. This is a major reason modern embedding-based systems, which routinely operate in 384, 768, or higher dimensions, standardize on cosine.

This slide grounds the abstract argument in the systems where metric choice has visible consequences. Vector database and RAG retrieval quality depends directly on the metric used to rank candidates. k-means produces differently shaped clusters under different metrics. k-NN's decision boundaries shift. Recommender "similar items" lists reorder. Near-duplicate detection flags different pairs.

The common thread is that in each of these, the metric is the ranking function. A reader who recognizes their own system in this list should treat the metric choice as a first-class design decision and evaluate it explicitly, not inherit whatever the library defaults to.

The closing point names what makes metric mistakes uniquely insidious. Most ML bugs are loud: a bad learning rate gives a diverging loss curve, a shape mismatch throws an exception. A wrong distance metric does neither. The system runs, returns ranked results, and those results look entirely reasonable.

But they are subtly degraded — slightly worse recommendations, clusters that don't quite cohere, retrievals that miss the best match. Because there's no error and the degradation is gradual, these problems can persist for months, quietly capping the quality of a product. That silence is exactly why understanding the metric, rather than defaulting it, matters so much.

The CTA closes the motivation post and hands off to the mechanics. Having established that the metric quietly sets the quality ceiling of every similarity system, the obvious next demand is precision: what exactly are these formulas, and how do they relate?

The next post delivers the math — the exact definitions of L1, L2, and cosine, the Minkowski family that connects them, the geometry of their unit balls, and the normalization identity that ties Euclidean distance to cosine. It turns the intuition built here into formulas you can reason about and implement.