Weaviate Essentials

This is the hands-on post: a complete, runnable Weaviate workflow rather than fragments. The cover sets expectations — we use the v4 Python client, the modern gRPC-backed library that replaced the older v3 client and has a different, cleaner API. The arc is the canonical one you'll repeat for every project: connect, create a typed collection with a vectorizer, batch-import data, then query three ways (semantic, hybrid-with-filter), inspect, and clean up.

Following it in order shows how the pieces compose, so afterward you can confidently bend the same skeleton to your own data and embedding model.

Step one does two things: open a connection and define the collection. connect_to_local() targets a Docker-hosted Weaviate and wires up the REST and gRPC ports for you. The create() call declares the schema — three text properties — and, importantly, attaches a vectorizer with Configure.Vectorizer.text2vec_openai().

That vectorizer choice is what makes near_text work later: Weaviate will call OpenAI's embedding API automatically on import and query. The Property/DataType objects give each field an explicit type, which controls how it's indexed for filtering and BM25. Run this once; creating a collection that already exists raises an error, so in real code you'd guard with collections.exists().

Step two imports data, and the use of batch.dynamic() is deliberate. The context manager streams objects in automatically-sized batches with parallel requests, which is how you import efficiently — both for indexing throughput and for amortizing the embedding API calls. Each add_object passes only the properties; because the collection has a vectorizer, Weaviate generates the embedding for you, so you never see or manage the vector here.

The final print of batch.failed_objects is not optional hygiene — it's essential. Batch imports can partially fail (a rate limit, a malformed object) without raising, so the failures are collected for you to inspect and retry. Ignoring this is how silent data loss happens during big imports.

Step three runs a semantic query with near_text. You pass natural-language text, not a vector; Weaviate embeds it with the same model configured on the collection and searches the HNSW index for the nearest objects. limit caps the results, and return_metadata=MetadataQuery(distance=True) asks Weaviate to include each result's distance so you can see how close the match is.

The payoff is visible in the query 'computer overheating' matching 'Why my laptop runs hot' despite sharing no words — the embeddings encode that these mean the same thing. near_text only works because a vectorizer is configured; with vectorizer none you would compute the query embedding yourself and call near_vector instead.

This trace shows the expected output and how to read it. The query 'computer overheating' returns the laptop-overheating article with a small distance (0.181) and the unrelated vector-search article with a larger distance (0.402). The comment makes the key point explicit: lower distance means a closer, more relevant match.

What distance actually means depends on the metric the collection uses — for cosine, it's effectively 1 minus cosine similarity, so 0 is identical direction and larger values are less similar. Always interpret distances in the context of your configured metric; comparing raw distances across collections with different metrics is meaningless.

Step four composes the two big ideas from the earlier posts in one call: hybrid search plus a metadata filter. hybrid() with alpha=0.5 fuses vector and BM25 results evenly, while filters=Filter.by_property('topic').equal('ml') restricts candidates to the machine-learning topic using the inverted index.

This is exactly the pre-filter-then-search behavior discussed in the How It Works post: Weaviate builds the allow-list of ml objects first, then performs hybrid retrieval within it. The Filter builder is expressive — you can chain and_/or_, use ranges, contains, and nested conditions — but even this one-liner demonstrates the headline capability: meaning, keywords, and structured constraints resolved together in a single request.

This slide explains why the batch pattern from step two matters enough to call out separately. batch.dynamic() tunes batch sizes on the fly and issues requests in parallel, so both the network round-trips and the embedding-model calls are amortized. The contrast is stark: inserting objects one at a time means one HTTP request and one serial embedding call per object, which turns a multi-minute import into hours and risks tripping API rate limits.

The recurring discipline is checking failed_objects after the batch completes. Because batching tolerates partial failure without raising, that list is your only signal that some objects didn't make it. Capture it, log it, and build a retry path for production imports.

Step five covers inspection and teardown. fetch_objects(limit=100) retrieves objects without any search ranking — useful for sanity-checking that your import landed and for browsing data. Here we just count them. collections.delete() removes the entire collection and its data, which is how you reset between experiments (and a reminder that it's destructive — never point it at production casually).

Finally, client.close() releases the gRPC connections. The v4 client holds persistent connections, and leaving them open leaks resources and triggers warnings. In scripts, close explicitly or use the client as a context manager; in services, keep one client alive for the process lifetime and close it on shutdown.

These practical notes catch the things that trip up first-time users. The OpenAI vectorizer needs credentials: set OPENAI_APIKEY in the environment Weaviate runs in, or pass it via request headers when connecting. Forgetting this produces auth errors at import, not at create time, which is confusing if you don't expect it.

The near_text-versus-near_vector distinction recurs: near_text requires a configured vectorizer, near_vector takes a raw vector and needs none. Distance values are only interpretable relative to the chosen metric. And connection hygiene — context manager or explicit close — prevents the resource leaks that otherwise accumulate quietly across runs.

This flow diagram summarizes the whole post as a four-step loop: connect, create, import, query. It mirrors the code exactly and is worth memorizing because it's the skeleton of essentially every Weaviate program you'll write. Swap the schema, swap the vectorizer, swap the query type, and the shape stays identical.

Seeing it as a flow also reinforces ordering dependencies: you can't import before the collection exists, and near_text won't work unless the create step attached a vectorizer. The diagram is the mental checklist you run before debugging — if something's broken, confirm each stage happened in order with the right configuration.

The teaser points to Day 88's Common Mistakes post. With a working pipeline in hand, the natural next step is learning the failure modes — the silent mismatches and resource traps that don't crash but quietly ruin recall, inflate cost, or exhaust memory — so you can run this same pipeline safely in production.