The RAG Pipeline

The cover frames post 4 as the practical payoff: a complete, runnable RAG pipeline that takes raw text and returns a grounded, sourced answer. The promise is that reading it top to bottom is equivalent to building RAG yourself.

The deliberate choice here is to keep the vector store as plain NumPy so the mechanics of similarity search are visible rather than hidden behind a database. Once you've seen cosine similarity computed by hand, swapping in a production vector DB is a mechanical change, not a conceptual one.

Step zero installs the three dependencies and sets up the two objects the pipeline needs: an embedding model (all-MiniLM-L6-v2, a small, fast, widely used sentence-transformer) and an OpenAI client for generation. The embedder turns text into vectors; the client turns a prompt into an answer.

Keeping the dependency list short is intentional. Everything else — chunking, the store, retrieval — we'll write ourselves in a few lines, so there's no framework magic obscuring what's actually happening. In production you'd likely use a managed embedding endpoint and a real vector DB, but the roles of these two objects stay identical.

Step one implements chunking with overlap. The function splits text into words, then slides a window of `size` words forward by `step = size - overlap` each time, so consecutive chunks share `overlap` words. That overlap is what prevents an idea that straddles a boundary from being lost to both chunks.

Word-based splitting is the simplest approach and fine for learning; production systems often split on sentences or structural boundaries to avoid cutting mid-sentence, and measure size in tokens rather than words. But the principle — bounded, overlapping pieces — is exactly the same. This is the stage that, as post 3 stressed, most determines retrieval quality.

Step two embeds every chunk in one batch and stacks the results into a matrix of shape (n_chunks, dim). Crucially, normalize_embeddings=True makes each vector unit length, which is what lets us use a plain dot product as cosine similarity later.

This is the offline indexing step: it runs once over your corpus, and the resulting matrix is your entire searchable index in this toy version. In a real system this matrix lives in a vector database with an HNSW index so search stays fast at scale, but here the matrix itself plays the role of the index so you can see exactly what's being searched.

Step three is retrieval implemented from scratch. The query is embedded and normalized the same way the chunks were, then `matrix @ qv` computes the dot product between the query and every chunk at once. Because all vectors are unit length, that dot product equals cosine similarity. argsort with the reverse slice picks the indices of the k highest scores.

This is the entire 'magic' of vector search laid bare: no special data structure, just normalized vectors and a dot product. A production vector DB does the same comparison but with an approximate index so it doesn't have to score every vector — the result is the same top-k, found faster. Note both query and chunks use the same embedder and the same normalization, avoiding the mismatch bug from post 5.

This flow diagram traces what the ask() function does once everything is wired: take the user's question, retrieve the top chunks, build a prompt that combines context and question, and call the LLM to produce a grounded answer. It's the runtime path of the whole post in four boxes.

It mirrors the six-line skeleton from post 1, now backed by the concrete retrieve() and build_prompt() functions defined in the surrounding slides. Keeping this picture in mind while reading the code makes clear that each function maps to exactly one box in the flow.

Step four builds the grounded prompt. The SYSTEM string does the load-bearing work: it instructs the model to answer only from the context, to say 'I don't know' when the answer isn't present, and to quote the source line. build_prompt joins the retrieved chunks with clear separators and assembles the final text.

The separators between chunks (---) matter more than they look: they help the model see where one source ends and the next begins, reducing the chance it blends unrelated passages into one fabricated claim. The explicit 'I don't know' instruction is the escape hatch that post 5 treats as mandatory — without it, empty or irrelevant retrieval turns into confident guessing.

Step five wires everything into a single ask() call. It retrieves context, builds the prompt, sends it to the model with temperature 0, and returns the answer. Temperature 0 is deliberate: grounded question-answering wants the most likely, faithful reading of the context, not creative variation, so determinism is the right default here.

This function is the whole pipeline's public interface. Everything else — chunking, embedding, the matrix, retrieval, prompt building — is plumbing behind these few lines. The final print line shows it answering a real question ('What is our refund window?') entirely from your indexed documents.

This slide is the bridge from toy to production. The only real change is the store: replace the NumPy matrix with a managed vector database such as Qdrant, pgvector, or Pinecone. At index time you call store.add(vectors, payloads); at query time, store.search(qv, top_k). Critically, the signature of retrieve() doesn't change, so the rest of the pipeline is untouched.

The payoff of a real store is persistence (you don't re-embed on every restart), scale (approximate indexes handle millions of vectors), and metadata filters — which is how you implement the access control discussed in post 2, restricting search to chunks a given user is allowed to see.

These are the bugs most likely to bite when you run this code. Forgetting to normalize embeddings means the dot product is no longer cosine similarity, so rankings go subtly wrong. Embedding the query with a different model than the chunks puts them in incompatible spaces and retrieval returns noise. Too large a k overflows the model's context window. And omitting the 'I don't know' rule turns every retrieval miss into a confident fabrication.

Each of these is silent — the code runs and returns something — which is exactly what makes them dangerous. Post 5 is dedicated to this class of failure, where the pipeline 'works' but the answers are quietly wrong.

The tips slide lists the steps that turn this toy into something production-worthy. Persist embeddings so you don't pay to re-embed your whole corpus on every boot. Add a reranker before the final top-k cut for sharper relevance, as post 3 described. Return source ids with the answer so users can verify and you can debug. And cache identical queries to save both latency and cost.

None of these change the fundamental shape of the pipeline — they harden it. The conceptual model you built across slides 1 through 5 stays exactly the same; these are the operational layers you add once it's carrying real traffic.

The cta closes the build post by pointing at the failure modes. You now have a working pipeline, but 'working' and 'reliable' are different things — a RAG system can run cleanly and still return garbage.

The teaser sets up post 5 as the debugging counterpart to this build: the specific mistakes that make a functioning pipeline quietly produce bad answers, and the cheap fixes for each.