Hybrid Search (BM25 + Vector)

Hybrid search is one of those ideas that sounds like a compromise but is actually the opposite — it refuses to compromise. Instead of betting your entire retrieval quality on a single notion of 'relevance,' you run two fundamentally different retrievers and let each contribute where it is strongest. BM25 treats relevance as lexical overlap weighted by rarity; dense vector search treats relevance as proximity in a learned semantic space.

This first post deliberately stays at the conceptual level. Before any formulas or code, you need a crisp mental model of the two halves and why anyone would bother running both. Everything in the next four posts — the stakes, the mechanics, the implementation, the failure modes — builds on this two-retriever picture.

The defining move of hybrid search is that you keep two ranked lists alive at once and merge them, rather than forcing every query through a single scoring function. A sparse keyword retriever (BM25) produces one ranking; a dense vector retriever produces another. A fusion step then reconciles them into a single ordered list that the rest of your pipeline consumes.

The payoff is coverage. A document can earn its place in the final results because it literally contains the query's words, because it is semantically close to the query's meaning, or because it scores moderately on both. That 'or' is the whole point — it widens what counts as a match without lowering the bar for precision.

BM25 is a probabilistic ranking function that has anchored search engines since the 1990s, and it is still shockingly hard to beat on the right queries. It scores a document by summing, over each query term, three intuitions: rare terms are more informative than common ones (inverse document frequency), more occurrences help but with diminishing returns (term-frequency saturation), and a long document shouldn't rank highly just because it has more room to contain the term (length normalization).

Crucially, BM25 needs no training, no GPU, and no embeddings — just term statistics over your corpus. That simplicity is a feature: it means BM25 works perfectly on tokens the system has never 'seen' in any semantic sense, like a brand-new product code or an internal acronym.

Dense vector retrieval is the modern counterpart. An embedding model maps both the query and every document into the same high-dimensional space, and retrieval becomes a nearest-neighbor search: which document vectors sit closest to the query vector, usually by cosine similarity. Because the space is learned from huge amounts of text, 'close' captures meaning — 'money back' lands near 'refund' even with no shared words.

The weakness is the mirror image of BM25's strength. Embedding models generalize meaning but can be unreliable on exact, low-frequency tokens like identifiers, since those carry little learnable semantics. A dense model may confidently rank a topically-related passage above the document that literally contains the ID you searched for.

This comparison is the heart of the post: the two retrievers fail and succeed on almost opposite query types. BM25 shines on exact product codes, names, acronyms, IDs, and rare domain jargon — anything where the literal token is the signal. Vector search shines on synonyms, paraphrase, intent, and cross-phrasing — anything where the meaning matters more than the spelling.

Because their strengths are complementary rather than overlapping, combining them is not redundant. You are not stacking two solutions to the same problem; you are covering two different problems with one system. That complementarity is precisely why hybrid recall can exceed either retriever's recall by a wide margin.

The flow diagram makes the dual-path structure concrete. A single query like 'reset SKU-449X' splits into two simultaneous lookups. BM25 latches onto 'SKU-449X' — the rare exact token — and surfaces the document that literally contains it. The vector retriever latches onto 'reset' and the general intent, surfacing semantically related help content.

The fusion stage then merges these two rankings into one. The key insight to carry forward is that the query never had to choose a lane: it benefits from both the exact-match path and the semantic path, and the fusion step decides how to weigh them in the final order.

This slide answers the obvious objection: if vector search is so good at meaning, why keep the old keyword retriever around? The answer is that real queries rarely fit neatly into one bucket. 'How do I fix error E-2041 on checkout' is simultaneously an exact-match query (the code E-2041) and a semantic query (the intent to fix a checkout problem).

A single retriever forces you to be good at one of these and resigned about the other. Over a stream of thousands of real queries, that resignation translates into a steady trickle of failures — queries that had a perfect answer in your corpus that simply never surfaced. Hybrid lets each query win on whichever signal happens to be strongest for it.

This code slide shows the two scoring worlds side by side so the abstraction becomes tangible. The BM25 call scores documents by literal term overlap: querying 'reset SKU-449X' gives the first document a strong positive score because it contains those exact tokens, and the second document zero. The embedding call does the opposite kind of matching: querying 'refund' scores the 'money back' document highly even though the word 'refund' never appears in it.

Notice the two outputs are not comparable as numbers — BM25 returns an unbounded score around 1.9, while cosine similarity returns 0.61 on a 0-to-1 scale. That scale mismatch is exactly the problem the fusion step in Post 3 has to solve. For now, the takeaway is that the same two documents rank in opposite orders depending on which retriever you ask.

Underneath the two retrievers are two different vector representations, and it's worth naming them. BM25 corresponds to a sparse vector: imagine one dimension for every word in your vocabulary, almost all of them zero, with nonzero weights only for the words actually present. Embeddings are dense: a few hundred floating-point numbers, essentially all nonzero, where no single dimension maps to a human-readable word.

Sparse vectors encode 'which exact words appear and how rare they are'; dense vectors encode 'what this text means.' Hybrid search is, at bottom, the decision to keep both representations rather than collapsing your documents to just one. That dual representation is what gives the system its two complementary failure modes instead of one.

The summary slide compresses the whole conceptual model into five lines you can recall under pressure. Hybrid is two retrievers fused; BM25 scores exact terms weighted by rarity; vectors score meaning by nearest neighbors; each covers the other's blind spot; and a fusion step produces the final order.

If you remember nothing else from this post, remember the complementarity: the two retrievers are strong in opposite places, which is what makes combining them worthwhile rather than redundant. The next post turns this conceptual advantage into concrete stakes — why it actually matters for production systems.

This is the conceptual foundation post in a five-part day on hybrid search. We covered what BM25 and dense vector retrieval each do, why their strengths are complementary, and what 'fusion' means at a high level.

The next post raises the stakes: it shows the specific, painful ways pure vector search and pure keyword search each fail in production, and why fusing them lifts recall, robustness, and user trust in ways neither retriever reaches alone.