Weaviate Essentials

This post is the engineering deep-dive. Once you can trace what physically happens when a query arrives, tuning stops being cargo-cult guesswork and becomes reasoning about data structures. The cover promises a tour of the moving parts: the query lifecycle, how HNSW achieves fast approximate nearest-neighbor search, what the inverted index contributes, how filtering interacts with the vector walk, and which parameters trade recall against latency and memory.

The theme throughout is that Weaviate is two cooperating indexes over one object store, and almost every performance question reduces to how you configure them.

The lifecycle is the backbone of the whole post. First, the query is turned into a vector — either by the vectorizer module from your text, or supplied directly if you pass a raw vector. Second, that vector traverses the HNSW graph to find the nearest stored objects. Third, any filters prune candidates, ideally before or during the walk. Fourth, if it's a hybrid query, BM25 runs in parallel and the scores are fused. Finally, the winning objects are fetched from storage and returned with whatever properties and metadata you requested.

Each stage maps to a tunable concern: vectorization is the model and its cost, the ANN walk is HNSW parameters, filtering is index design, and fusion is alpha. Knowing the stages tells you where a slow or low-quality query is actually being decided.

The pipeline diagram lays the four stages out linearly so the order is unambiguous: Vectorize, then ANN search, then Filter, then Fuse-and-fetch. The icons are mnemonic — numbers for embedding, a web for the graph walk, a broom for pruning, an outbox for the response.

The value of seeing it as a pipeline is that latency is additive across stages. If a query is slow, you can attribute the cost: a slow embedding model inflates the first stage, a high ef inflates the second, an unselective filter inflates the third. Diagnosing latency means knowing which stage owns it.

HNSW — Hierarchical Navigable Small World — is the algorithm that makes vector search fast. Brute force would compare the query to every stored vector, which is linear in the number of objects and far too slow at scale. HNSW instead builds a multi-layer graph of vectors connected to their neighbors.

Search enters at a sparse top layer where each hop covers a lot of distance, greedily moving toward the query vector, then drops into progressively denser layers to refine. The result is roughly logarithmic search time. It is approximate — it can occasionally miss the true nearest neighbor — which is the tradeoff you control with the ef parameter: higher ef explores more of the graph for better recall at the cost of speed.

This stack diagram visualizes HNSW's layered structure. The top layer (Layer 2) is sparse: few nodes, long-range links, used for big jumps that quickly get you to the right neighborhood. The middle layer is denser with medium hops. Layer 0 contains every node and the densest connections, where the fine-grained final search happens.

The layering is what delivers logarithmic behavior: coarse navigation up high, precise navigation down low, the same way you'd use a highway to get near a city and surface streets to find the exact address. Build-time parameters (max_connections, ef_construction) decide how richly these layers are wired, trading memory and build time for query quality.

The inverted index is the second pillar, handling everything keyword and filter related. It maps each token (and each filterable property value) to the list of objects that contain it — the same structure that powers traditional search engines. BM25 uses it to score documents by term frequency and rarity, giving the keyword branch of hybrid search.

Filters use the very same index: 'topic = ml' is answered by looking up the posting list for that value. Because both keyword scoring and filtering share this structure, properties you intend to filter or keyword-search on should be indexed accordingly in the schema — that decision directly affects which queries are fast.

This comparison crystallizes the two-index architecture. The HNSW vector index answers similarity questions approximately and fast. The inverted index answers keyword and exact-filter questions. Critically, both reference the same object IDs in the underlying store, so their results can be intersected (for filtering) or fused (for hybrid ranking).

Understanding that there are two distinct indexes explains a lot of behavior: why some queries are memory-bound (HNSW in RAM) while filters are more disk-and-CPU bound, and why making a property filterable has a different cost than vectorizing it. One store, two lenses.

Filtering and ANN interact in a subtle way that catches people out. The naive approach, post-filtering, would run ANN to get the top-k and then drop the ones failing the filter — but that can leave you with far fewer results than k if the filter is selective. Weaviate instead supports pre-filtering: it first builds an allow-list of objects matching the where-conditions from the inverted index, then constrains the HNSW search to those objects.

This keeps results correct and full even under heavy filtering, at some extra cost to build and apply the allow-list. The lesson is that filters aren't free riders on vector search; they're a first-class part of the query plan, and very selective filters change the performance profile of the ANN walk.

This configuration snippet exposes the dials that govern the vector index. ef_construction sets how thoroughly the graph is built — higher means a better-quality graph but slower import and more memory. max_connections (the M parameter) is the graph's degree: more connections improve recall but cost RAM. ef is the query-time search breadth: the single most impactful runtime knob for the recall-versus-latency tradeoff.

distance_metric must match your embedding model — cosine for most text models, dot product or L2 for others. Setting it correctly is not optional tuning; the wrong metric silently degrades every result. These parameters are where the abstract tradeoffs from earlier slides become concrete numbers you commit to.

These are the knobs worth knowing and roughly what each one buys you. ef at query time is your live recall/latency dial — raise it when results feel incomplete, lower it when latency hurts. ef_construction and M are build-time choices that set the ceiling on quality and the floor on memory; you pick them once per collection.

The distance metric must match the model, full stop. And quantization — PQ (product quantization) or BQ (binary quantization) — lets you trade a small amount of recall for a large reduction in RAM, which is often the difference between fitting on one node and not. Knowing which knob addresses which symptom is the whole point of understanding the internals.

This bar diagram makes the central tradeoff visceral. As ef rises from 16 to 64 to 256, recall climbs — more of the graph is explored, so the true nearest neighbors are found more often — but latency rises with it because each query does more work. The numbers are illustrative, but the monotonic shape is exactly what you observe in practice.

The practical method this implies: sweep ef across a range on a representative query set, measure recall against a brute-force ground truth and measure latency, and pick the smallest ef that meets your recall target. That gives you the fastest configuration that's still good enough, rather than guessing.

The teaser points to Day 88's Code Example post. Having seen the machinery, the next post puts hands on the keyboard: defining a schema, batch-importing data, and running semantic, hybrid, and filtered queries end to end — turning these internal concepts into working code.