Embedding Models Compared

This post opens Day 70 by anchoring the whole topic in one claim: your retrieval quality is bounded by the embedding model you choose. Before you can compare models, you have to know what they are and what 'better' even means — otherwise you're ranking numbers you don't understand.

The cover frames embedding models as the keystone of RAG, not a minor config knob. Everything in this day builds from here: why it matters, how it works, how to test it, and how comparisons go wrong.

An embedding model is a learned function from text to a fixed-length vector of real numbers. The individual numbers carry no human-readable meaning — you can't point at dimension 200 and say what it represents. What matters is relative geometry: the model is trained so that texts with similar meaning produce vectors that point in similar directions.

That property is what makes semantic search possible. Instead of matching keywords, you embed the query and the documents into the same space and look for the document vectors closest to the query vector. 'Find the answer' becomes 'find the nearest neighbors,' which is a fast, well-understood numerical operation.

The reason the embedding model is the keystone is structural, not stylistic. In a RAG pipeline, retrieval runs before generation, and the LLM only ever sees the chunks retrieval returned. If the embedding model places the true answer far from the question, that chunk never enters the candidate set, and no downstream step can rescue it.

This is why teams that obsess over prompt engineering while ignoring the embedding model often hit a wall. The prompt can only shape how the model uses the context it was given; it cannot conjure context that retrieval failed to fetch. The embedding model literally defines what 'relevant' means for your system.

The vectors diagram is a deliberately simple picture of the core idea. A query sits at one point; a good model places the true matching passage nearby (small angle, high cosine similarity), while a poorly fit model scatters that same passage far away, where it won't be retrieved.

In reality these spaces have hundreds or thousands of dimensions, not two, so you can't literally see them. But the intuition holds: comparing embedding models is really about comparing how faithfully each one places questions near their true answers and away from distractors.

Dimension — the length of the output vector — is the first concrete tradeoff in any comparison. Higher dimension gives the model more room to encode fine distinctions, which can lift accuracy. But it also multiplies storage, slows nearest-neighbor search, and inflates the index size, since every chunk you ever store carries that full vector.

The key point is that more dimensions only help if the model actually uses them well. A well-trained 384-dim model frequently matches or beats a poorly-trained 1536-dim one. So dimension is a cost you should justify with measured accuracy gains on your data, not assume buys you quality.

Embedding models come in three broad families, and knowing them keeps comparisons honest. Dense models output a single vector per text and dominate most RAG setups because they capture meaning and search fast with approximate nearest neighbors. MiniLM, BGE, E5, and the OpenAI embedding models are all dense.

Sparse models like SPLADE produce high-dimensional term-weighted vectors and excel when exact terms matter (codes, names, IDs). Late-interaction models like ColBERT keep one vector per token and compare them at query time — very accurate, but heavier to store and search. Most comparisons in practice are between dense models, but it's worth knowing the alternatives exist before declaring a winner.

'Better' is not one number, and this slide lists the axes that a real comparison spans. Retrieval quality is the headline — does the model surface the right chunk — but it never stands alone. Domain fit determines whether benchmark strength transfers to your legal, medical, code, or multilingual text.

Cost and latency are operational realities that decide whether a model is shippable, not just accurate. And dimension ties back to index size and search speed. A serious comparison scores all of these together; optimizing only for benchmark accuracy is how teams end up with a model that's technically strong and practically wrong.

This code shows how concretely simple producing an embedding is: load a model, call encode, get a vector. The shape (384,) tells you the dimension, and the first few values show the raw, uninterpretable numbers that nonetheless carry meaning collectively.

The simplicity is the point. The hard part of 'comparing embedding models' is never the API call — every model exposes roughly this interface. The hard part is deciding which model's vectors place your queries nearest their true answers, which is what the rest of this day is about.

The pipeline diagram locates the embedding model inside the larger RAG flow so the stakes are visible. Chunking decides what text exists to be embedded; the embedding model turns that text into vectors; the store indexes them; search finds the nearest ones at query time.

The 'MODEL CHOICE' label on the embed stage is the whole message of this post. That one decision propagates through everything after it — and because the same model must embed both your documents and your queries, it's a decision that's expensive to change once you've indexed at scale.

Symmetric versus asymmetric is a distinction that quietly ruins comparisons when ignored. Symmetric models are trained to compare two things of the same kind — sentence to sentence — and shine at tasks like deduplication or clustering. Asymmetric models are trained for search: a short query against a longer passage, where the two sides look very different.

RAG retrieval is an asymmetric task. Using a sentence-similarity model for query→document search is a category error: you're applying a model to a job it wasn't trained for, then blaming the model when recall is mediocre. Always check what task a model was optimized for before you rank it.

The summary distills the post into four portable ideas. The embedding model is the operational definition of 'similar' for your system. Its output is a fixed vector whose geometry — not its individual numbers — carries the meaning. Dimension is a genuine tradeoff between quality and cost, not a 'bigger is better' dial.

Most importantly, the right model is the one matched to your task (asymmetric search) and your domain — which is exactly why the following posts move from 'what it is' to why it matters, how it works, how to test it, and how comparisons go wrong.

The CTA closes the concept post and points to the stakes. Now that you know what an embedding model is and what 'better' could mean, the next post makes the case for why this single choice quietly determines whether your whole RAG system works — long before the LLM is involved.