Qdrant in 8 Slides

This cover sets the tone for the most technical post of the day. Qdrant's millisecond responses can feel like magic, and the goal here is to dispel that by showing the actual machinery: a Hierarchical Navigable Small World graph you descend like a skip-list, background segment optimisation, a filter index woven into traversal, and quantization that compresses vectors to fit RAM.

The practical motive is debugging. When recall is bad or latency spikes, you can only fix it if you know which of these mechanisms is misbehaving. This post is the mental model that turns vague 'Qdrant is slow' complaints into specific, fixable hypotheses.

This slide re-states the division of labour because it's where many integration bugs originate. Qdrant does not produce embeddings; you do, with whatever model you chose. The non-negotiable rule is consistency: the model and vector size used when you store data must be identical to the model and size used at query time.

Two distinct failure modes follow. A size mismatch is loud — Qdrant rejects the request — so it's annoying but self-correcting. A model mismatch is silent: if you index with one model and query with another of the same dimension, the request succeeds and returns confidently wrong results. The silent failure is the dangerous one, and it's why this caveat leads the 'how it works' post.

HNSW is the heart of Qdrant's speed, so it's worth understanding the intuition even if you never touch the math. Picture a multi-layer graph. The top layer has few nodes connected by long-range links; each layer down adds more nodes and shorter links, until the base layer holds every point. A search starts at the top, greedily hops to whichever neighbour is closest to the query, and when it can't get closer it drops to the next layer and repeats.

The long top-layer links cover huge distances in a few hops, and the dense lower layers refine the result. The effect is roughly logarithmic search time instead of the linear cost of comparing against every point. Two parameters — m (links per node) and ef (search breadth) — trade memory and speed against recall, and they're the first knobs you reach for when tuning.

This flow diagram visualises the descent through HNSW. You enter at the top layer with its few, far-reaching nodes, hop toward the query, drop into the mid layer for more detail, and finally reach the base layer where every point lives. By the time you arrive at the base, you're already near the answer, so only a small neighbourhood needs careful scoring to produce the top-k.

Seeing it as a funnel — coarse at the top, fine at the bottom — explains why HNSW is fast without being exact. You skip the vast majority of points entirely; you only ever examine a thin path toward the query and its immediate surroundings.

Segments explain Qdrant's write behaviour, which surprises people who expect a monolithic index. Data is split into segments, each an independent shard with its own structures. New writes land first in a mutable segment so they are immediately searchable, then a background optimiser merges and builds the full HNSW index for them. Queries fan out across all segments and merge the per-segment results.

This architecture is why a freshly upserted point can be found right away, before the heavyweight indexing finishes — and also why heavy write bursts temporarily cost some query performance while optimisation catches up. Understanding segments helps you reason about both freshness and the transient latency you might see after a big load.

Filterable HNSW is the mechanism behind the previous post's 'filtering is first-class' claim, explained at the engine level. The two naive strategies both fail at scale: pre-filtering builds a candidate set first and then the graph structure no longer matches that subset, while post-filtering searches then throws away non-matching results, which can leave you short or force expensive over-fetching.

Qdrant instead builds a payload index and consults it during graph traversal, skipping points that don't satisfy the filter as it walks. This preserves the logarithmic search behaviour while honouring constraints, which is what lets you combine tight latency with correctly scoped results. It's also why indexing the fields you filter on is not optional — the traversal relies on that index.

Quantization is the answer to a brutal arithmetic problem: float32 vectors are big. A million 1536-dimensional vectors is roughly six gigabytes just for the raw values, before index overhead, and RAM is the expensive resource. If the index spills to disk, latency suffers badly.

Qdrant can compress vectors with scalar quantization (float32 to int8, a 4x reduction) or binary quantization (down to single bits, up to 32x). It searches on the compact representation to find candidates quickly, then optionally rescores the top candidates using the original full-precision vectors to recover most of the accuracy lost to compression. The result is a large memory saving with only a small, often negligible, hit to recall.

This snippet ties the theory to configuration. Creating a collection fixes the two immutable basics — size 768 and cosine distance — and then layers on scalar int8 quantization with always_ram=True, so the compressed vectors are pinned in memory for fast access. This is a realistic production setup, not a toy.

The code is worth typing because it forces the earlier abstractions to become concrete decisions. You must pick a size that matches your embedding model, a metric that matches how the model was trained, and a quantization strategy appropriate to your collection's scale. Each of those choices maps directly to a section of this post.

This pipeline diagram traces a single query end to end, which is the synthesis the whole post builds toward. Your model produces a query vector. Qdrant walks the HNSW graph, hopping toward that vector. During the walk it applies the payload filter, skipping non-matching points. Finally it scores the surviving candidates by the collection's metric and, if quantization is on, rescores the top ones with full vectors before returning top-k.

Holding this four-stage path in mind is the debugging superpower the cover promised. Bad recall? Suspect the metric, ef, or over-aggressive quantization. Slow filtered queries? Suspect a missing payload index. Each symptom maps to a specific stage.

This recap distils the engine into five memorable lines. You embed the data; Qdrant indexes the vectors. HNSW is a layered graph giving log-time search. Segments make writes searchable immediately. Filterable HNSW applies constraints during traversal rather than before or after. And quantization shrinks vectors so the index fits in RAM.

These five are deliberately the load-bearing concepts — the ones that explain both why Qdrant is fast and where it can go wrong. The mistakes post that follows is essentially a catalogue of what happens when you ignore one of them.

The CTA moves from theory to hands-on. You now have the conceptual map of the engine; the next post walks through real, runnable code — connecting, creating a collection, upserting, searching, and filtering — so the abstractions become muscle memory. Save this post as the diagram you'll glance at while reading the code.