Distance Metrics: L1, L2, Cosine

This cover frames the entire day: distance is not a single, obvious quantity but a deliberate choice among several valid answers. The hook deliberately unsettles the reader's assumption that "how far apart are these?" has one answer, because that assumption is the root of most metric-related bugs.

The three metrics in the title — L1, L2, and cosine — are the ones a practitioner meets constantly: in nearest-neighbor classifiers, in clustering, in search and retrieval, and inside every embedding-based system. By the end of this post the reader should be able to say, in one sentence, what each metric measures and why they can disagree.

A distance metric is the bridge between raw vectors and any notion of similarity. Formally it is a function d(x, y) that maps a pair of points to a single real number, with the convention that smaller means more alike and zero means identical. Everything that ranks, groups, or retrieves "similar" items is ultimately sorting by this number.

The key mindset shift is that choosing the metric is a modeling decision on the same level as choosing features or a loss function. It encodes your definition of similarity. Treating it as an incidental default — whatever the library uses — is how teams end up with systems that technically run but quietly rank the wrong things first.

L1, the Manhattan or taxicab distance, sums the absolute differences across every coordinate: |x1−y1| + |x2−y2| + … The name comes from navigating a city grid, where you can only travel along streets and never diagonally through a block. The total distance is the sum of the horizontal and vertical legs.

Because each coordinate contributes its raw absolute gap rather than its square, L1 grows only linearly with any single large difference. That makes it noticeably more robust to outliers than L2: one wildly off feature inflates the distance, but it does not dominate the way a squared term would. This robustness is why L1 shows up in regularization (Lasso) and in settings with heavy-tailed noise.

L2, the Euclidean distance, is the everyday straight-line distance from geometry: square the per-coordinate differences, sum them, and take the square root. Geometrically it is the length of the arrow connecting the two points, and it is the distance our intuition reaches for by default.

The squaring step is what gives L2 its character. A coordinate that is off by 10 contributes 100 to the sum, while one off by 1 contributes only 1, so large gaps dominate disproportionately. This makes L2 sensitive to outliers and to any unscaled feature with a wide range — a property that is both its strength (it strongly separates genuinely distant points) and its weakness (it is easily distorted by a single noisy or large-magnitude dimension).

Cosine similarity asks a fundamentally different question. Instead of "how far apart are these points in space?" it asks "do these two vectors point in the same direction?" It computes the cosine of the angle between them, which equals the dot product divided by the product of the two vector lengths. A value of 1 means the vectors are perfectly aligned, 0 means perpendicular, and −1 means opposite.

The crucial consequence is that cosine is completely blind to magnitude. Doubling a vector's length leaves its direction unchanged, so the cosine is unchanged. Cosine distance is just 1 minus the similarity, turning "perfectly aligned" into a distance of 0. This direction-only view is exactly what you want when the length of a vector is an artifact (like document length) rather than meaningful signal.

This vector diagram shows two points, A and B, that sit at the same distance from the origin but point in different directions. It is the visual anchor for the whole post: the same pair of vectors yields three different "distance" numbers depending on which metric you apply.

Looking at A and B, you can almost read off the intuition. The straight-line gap between their tips is L2. The grid-walk path (across, then up) is L1, which is longer. And the angle swept between the two arrows is what cosine measures, ignoring how long the arrows themselves are. Three different aspects of the same picture, three different metrics.

This snippet computes all three distances on the exact pair from the diagram so the abstract definitions become concrete numbers. L1 sums the absolute differences (|3−1| + |1−3| = 4). L2 is the root of the squared differences (√(4+4) ≈ 2.83). Cosine uses the dot product over the magnitudes.

Notice how compact each one is in NumPy — a metric is just a few array operations. Seeing them side by side on the same inputs is the fastest way to internalize that they are genuinely different functions, not three names for the same thing. Run it, then change a and b and watch all three numbers move in different ways.

This comparison splits the three metrics by what they fundamentally pay attention to. L1 and L2 are geometric: they care about where points actually sit in space, so moving a point changes the distance and the magnitude of vectors matters. They are the natural choice when your coordinates are real, comparable measurements.

Cosine sits in the other column because it cares only about direction. Scaling a vector — making it longer or shorter without rotating it — leaves cosine completely unchanged. This is precisely why text representations and learned embeddings, where vector length is often an incidental side effect of input length or training, almost universally use cosine.

This slide makes the stakes explicit before the deeper posts. The same two data points can be judged "close" by one metric and "far" by another, and there is no contradiction — they are simply answering different questions. A recommender built on cosine will happily match a long document to a short one if their topics align; an L2-based one would penalize the length difference and rank them apart.

The practical takeaway is that the metric is where you encode what "similar" means for your problem. Get that encoding wrong and every downstream result is subtly off, even though every individual computation is correct. This is the thread that the "Why It Matters" post picks up next.

These one-liners are the memory hooks for the whole topic. "Grid walk" for L1, "straight line" for L2, and "angle" for cosine compress each formula into an image you can recall under pressure. The fourth and fifth bullets capture the single most important distinction: L1 and L2 see magnitude, while cosine ignores it.

The goal of this slide is recall, not derivation. If a reader walks away able to reconstruct these five lines, they can re-derive everything else — the formulas, the use cases, and the failure modes — from first principles.

The closing misconception to dismantle is the belief that there is one correct distance and the others are approximations of it. That framing is backwards. L1, L2, and cosine are three legitimate, mutually incompatible definitions of similarity, each correct for the questions it is designed to answer.

The right choice depends on three things, each explored in later posts: whether the magnitude of your vectors carries real information, how many dimensions you are operating in (high dimensions favor cosine), and what "similar" concretely means for your task. Internalizing that distance is a choice — not a fact — is the single most valuable idea in this entire day.

The CTA closes the conceptual post and points toward the practical payoff. Having established what the three metrics are and that they genuinely differ, the natural next question is: so what, where does this actually bite?

The next post answers exactly that, showing how this one seemingly small choice silently sets the quality ceiling of your search, clustering, and recommendation systems — and why the failures are so dangerous precisely because nothing throws an error.