Embeddings, Visually

This is the hands-on post, and the framing promises something the previous three only described: code you can actually run. By the end you will have built a miniature semantic search end to end — load a model, embed sentences, rank a query, and even plot the result. The goal is to convert abstract understanding into muscle memory.

The cover sets that expectation. Everything here uses real, mainstream libraries and runs on a laptop, so there is no excuse not to try it. The fastest way to truly understand embeddings is to watch them respond to inputs you chose yourself.

Step zero is setup, and it matters that we use a real, widely-used model rather than toy vectors. sentence-transformers gives you pretrained encoders in a single import, and all-MiniLM-L6-v2 is the workhorse choice: small, fast, and good enough for most prototypes. Printing the embedding dimension confirms you get 384-dimensional vectors.

Choosing a specific named model is a deliberate teaching point that pays off in Post 5. Every vector you produce here lives in MiniLM's space; if you later compared them to vectors from a different model, the numbers would be meaningless. Pinning the model name now is the habit that prevents that mistake later.

This step turns three sentences into three points in 384-dimensional space. The encode call hides a lot of machinery — tokenization, a transformer forward pass, pooling — but the output is exactly the object from Post 1: an array of numbers per item. Printing the shape, (3, 384), reinforces that you now have three points in the same room.

Note that the three sentences were chosen so two are about account access and one is about weather. That setup lets the next step demonstrate that the geometry separates meaning, not just words. Picking your examples to make the result legible is itself a useful habit when exploring any embedding space.

Here the payoff arrives: a query that shares almost no words with the documents still ranks them correctly by meaning. 'forgot my login' has no token overlap with 'reset my password' or 'recover your account,' yet both score around 0.5 while the weather sentence scores near 0.05. Cosine similarity over the embeddings did what keyword matching never could.

The sorted print is the whole point of semantic search in five lines. util.cos_sim does the angle computation, and sorting turns scores into a ranking. Change the query to 'sunny day' and watch the ranking flip — the weather sentence jumps to the top. That responsiveness is the thing to feel.

This trace slide is a frozen snapshot of the previous step's output so you can study the numbers without running anything. The query is the input; the two account sentences come back with mid-range scores; the weather sentence comes back near zero. The comment underlines the lesson: meaning ranked above word overlap.

Reading the scores as values rather than truth is good practice. A 0.55 is higher than a 0.50, so 'recover account' edges out 'reset password,' but both are clearly relevant and the weather sentence clearly is not. The gap between roughly 0.5 and roughly 0.05 is the signal; the small difference between the two top results is noise you should not over-interpret.

Step three reduces the 384-dimensional vectors to two dimensions with PCA so you can plot them. PCA finds the two directions of greatest variance and projects onto them, which keeps the broad structure while discarding detail. The scatter plot then shows the two account sentences sitting close and the weather sentence off on its own.

This is the most visually satisfying step, but it carries a warning that Post 5 will hammer: the 2D picture is a lossy shadow of the real space. Use it to build intuition and to sanity-check that your clusters look sensible, never as the basis for a decision. The real distances live in 384 dimensions, not on the plot.

This diagram approximates what the PCA plot produces: 'reset password' and 'recover account' land close together while 'weather' sits far away. Seeing it as positioned points connects the code back to Post 1's core picture — meaning has a shape, and here you have actually drawn it from real model output.

The value of plotting, even imperfectly, is that it makes correctness visible at a glance. If you ran this and the weather sentence landed between the two account sentences, you would immediately suspect a bug — wrong model, swapped variables, or unnormalized inputs. The plot is a fast smell test, which is exactly the role projections should play.

This recap reads the result like an analyst would. The two account sentences cluster tightly, the weather sentence is an outlier, and crucially distance matched meaning rather than shared words. It also flags the honest caveat that PCA loses detail while preserving the gist, foreshadowing the failure-mode post.

Learning to read a result this way — what clustered, what separated, and whether that matches your expectation — is a transferable skill. Every time you explore embeddings, you will run a small probe like this and interpret the geometry. Doing it deliberately on a tiny example builds the instinct for when something looks wrong on a big one.

Step four packages everything into a reusable search function, which is how this would actually live in code. It encodes the query and corpus, scores them with cosine similarity, sorts to find the top k, and returns the matches with their scores. The query 'can't sign in' will surface the two account sentences despite, again, sharing no keywords.

Wrapping the logic in a function is the small engineering step that turns an experiment into a tool. Note the parameter k for how many results to return — a knob you will tune in real systems. In production you would also precompute and cache the corpus vectors instead of re-encoding them every call, but the logic shown is exactly the core of any vector search.

This pipeline diagram abstracts the function you just wrote into four stages: encode text into vectors, score with cosine similarity, rank by closeness, and return the top matches. It is intentionally the same shape as the RAG pipeline from Post 2, so you can see that what you built is a real, if minimal, instance of that pattern.

The abstraction matters because it scales. Swap the in-memory list for a vector database, swap MiniLM for a larger model, add a generation step on the end, and you have production RAG. The stages do not change — only their implementations do. Recognizing that continuity is what lets a toy example teach you a real architecture.

The final recap consolidates the hands-on lessons: a model turns text into 384-dimensional vectors, cos_sim ranks by meaning, PCA lets you plot the space, k controls how many results you return, and you should swap the sentences and re-run to build intuition. The last point is the most important — this code is meant to be played with, not just read.

Actively modifying the example is where the learning sticks. Add a sentence about billing, query it, and watch it cluster separately. Try a misspelled query and see that embeddings shrug it off. Each experiment converts a fact you read into an intuition you own, which is the entire purpose of a code-heavy post.

This hands off to Post 5. Having built something that works, you are now ready to learn how it quietly breaks. The teaser frames the final post as the traps — the places embeddings lie to you while the code keeps running and the numbers keep looking plausible.

That sequencing is intentional. Failure modes mean little until you have a working mental model to attach them to. Now that you have embedded, ranked, and plotted real text, the warnings about mixing models, misreading plots, and forgetting to normalize will land as concrete risks to your own code rather than abstract cautions.