Hybrid Search (BM25 + Vector)

The mechanics post is where the hand-waving stops. By now you know hybrid runs two retrievers and fuses them; this post explains how each retriever actually scores documents and, most importantly, how the fusion step combines two scores that live on completely different scales.

The single hardest part of building hybrid search is not the retrievers — both are well-trodden — it's the fusion. Get fusion wrong and your 'hybrid' system silently collapses into whichever retriever happens to produce larger numbers. So we spend most of this post on the two dominant fusion strategies: normalized weighted sums, and Reciprocal Rank Fusion.

BM25's formula looks intimidating but encodes three plain intuitions. First, inverse document frequency: a term appearing in few documents is more discriminating than a common one, so rare query terms contribute more to the score. Second, term-frequency saturation: a document mentioning the term five times is more relevant than one mentioning it once, but not five times as relevant — the contribution flattens out. Third, length normalization: a long document shouldn't rank highly just because its size gives the term more chances to appear.

The per-term scores are summed across all query terms to give the document's total. The whole thing runs on term statistics computed from your corpus — no model, no training — which is exactly why it's fast and why it handles never-before-seen exact tokens so well.

This code slide writes the BM25 formula out explicitly so the three intuitions map to concrete factors. IDF(t) is the rarity weight. The numerator f(t,D)·(k1+1) and the denominator's f(t,D) term together produce the saturating term-frequency curve, with k1 (around 1.2) controlling how quickly extra occurrences stop helping. The b·|D|/avgdl piece inside the denominator applies length normalization, with b (around 0.75) controlling how aggressively long documents are penalized.

You rarely implement this yourself — libraries and search engines do — but seeing the formula demystifies it. The two knobs, k1 and b, are the only real tuning parameters, and their defaults are good enough for the vast majority of corpora. The point is that BM25 is a transparent, deterministic function of word counts, not a black box.

The dense side of the pipeline relies on approximate nearest-neighbor (ANN) search. Documents are embedded once, offline, and stored in a specialized index — HNSW (a navigable small-world graph) or IVF (inverted file with clustering) being the common choices. At query time you embed the query a single time, then ask the index for the closest document vectors by cosine similarity.

The word 'approximate' is load-bearing: a brute-force search comparing the query to every document is exact but scales linearly with corpus size. ANN indexes trade a tiny, tunable amount of recall for sublinear query time, which is what makes dense retrieval viable over millions of documents. For hybrid purposes, the dense retriever is essentially a black box that returns a ranked list of document IDs by semantic similarity.

This slide names the central technical obstacle of hybrid search: the two retrievers' scores are not comparable. BM25 produces unbounded positive scores that can run from 0 to 30 or more depending on query length and term rarity. Cosine similarity is bounded, typically in [0,1] for normalized embeddings. If you simply add a BM25 score of 14 to a cosine score of 0.7, the BM25 term dominates by two orders of magnitude — your 'hybrid' is BM25 with a rounding error of vectors mixed in.

There are two principled escapes. Either normalize both scores onto a common range before combining them (so neither dominates by sheer magnitude), or abandon raw scores altogether and fuse using only the rank positions, which are inherently comparable across retrievers. The next slides cover both.

The compare diagram lays out the two fusion strategies head to head. Weighted score fusion min-max normalizes each retriever's scores to [0,1], then takes a weighted sum: final = α·norm(bm25) + (1−α)·norm(vec). The weight α lets you bias toward whichever retriever works better on your data. The downside is that it's sensitive to how you normalize — outliers and score distributions can distort the result, and you have to tune α.

Reciprocal Rank Fusion takes the opposite approach: it throws away the raw scores entirely and uses only each document's rank position in each list. A document's fused score is the sum over retrievers of 1/(k+rank). It's robust, essentially parameter-free, and immune to the scale problem because ranks are always comparable. This robustness is why RRF has become the default in most engines.

This code slide implements RRF in seven lines, and the simplicity is the point. You pass in one ranked list of document IDs per retriever. For each list, you walk the documents in rank order and add 1/(k+rank) to that document's running score — so a document ranked #1 contributes 1/(k+1), #2 contributes 1/(k+2), and so on. Documents appearing in multiple lists accumulate contributions from each, which is what rewards cross-retriever agreement.

The constant k (conventionally 60) softens the influence of top ranks so that a document ranked #1 in one list doesn't completely dominate one ranked #2 in both. Crucially, RRF never touches a BM25 score or a cosine value — it only ever sees rank positions, which sidesteps the entire normalization headache. Sort by the accumulated scores, descending, and you have your fused ranking.

The pipeline diagram traces a query end to end. One user query fans out to two retrievers: BM25 returns its top-k by term matching, the vector retriever returns its top-k by ANN similarity. The fusion stage (RRF or weighted) merges those two lists into a single ranking. An optional final stage — a cross-encoder reranker — can reorder the fused top-k for extra precision.

The reranker is shown as optional because it's a separate technique (the subject of the next day), but it composes naturally with hybrid: fusion is about recall (getting the right documents into the candidate set), reranking is about precision (ordering that set correctly). Together they form the modern retrieval stack.

This slide argues for RRF as the sensible default. Its biggest practical virtue is that it needs no score normalization and almost no tuning — k=60 works across a remarkable range of corpora, so you can ship it without a tuning campaign. Because it rewards documents that rank well in either list, a document that's #1 on BM25 and entirely absent from the vector list still surfaces in the fused results, preserving the exact-match safety net.

Weighted score fusion can outperform RRF when you've carefully tuned α on a good eval set and your score distributions are well-behaved, but it's fragile and requires that investment. For most teams, RRF gives 90% of the benefit with 10% of the effort, which is exactly why search engines like Elasticsearch and vector databases like Qdrant and Weaviate ship it as their built-in hybrid mode.

The summary slide collapses the mechanics into five recallable lines: BM25 is IDF times saturating TF times a length penalty; the dense side is ANN over precomputed vectors; the raw scores from the two live on incompatible scales; you either normalize-and-weight or rank-fuse with RRF; and an optional reranker reorders the fused top-k.

The one idea to anchor on is the scale problem and its two solutions. Almost every subtle hybrid bug traces back to mishandling the fusion step — which is exactly the territory the mistakes post will map out.

This was the mechanics post in the five-part day. We demystified the BM25 formula, explained ANN-based dense retrieval, named the score-scale problem at the heart of fusion, and contrasted weighted score fusion with Reciprocal Rank Fusion.

Next, Post 4 turns all of this into a complete, runnable implementation — building both indexes, retrieving from each, and fusing with RRF in about fifty lines you can paste and point at your own documents.