LlamaIndex Crash Course

Post 4 is the hands-on payoff of the crash course. The previous posts built understanding; this one builds a working application. Everything here is real, runnable code — paste it into a file, point it at a folder of your own documents, and you have a functioning RAG system that answers questions about your data with citations.

The deliberate choice to make this post code-heavy reflects how people actually learn a framework: by running the smallest thing that works, then expanding it. We start at six lines and grow toward a production shape, with each slide adding one real concern — persistence, citations, model choice.

Installation is one pip command, but the second line matters more than people expect. By default LlamaIndex uses OpenAI for both the LLM and the embedding model, so it needs OPENAI_API_KEY in the environment. If you skip this you'll hit an auth error on the first query, which trips up nearly every newcomer.

If you'd rather not depend on OpenAI, you can swap in local embeddings and a local or alternative LLM — shown in slide 4. But for a first run, setting the key and using the defaults is the fastest path to a working answer, so start there and optimize later.

This is the canonical LlamaIndex 'hello world,' and every line earns its place. SimpleDirectoryReader('./data').load_data() reads every supported file in the folder into Documents. VectorStoreIndex.from_documents(docs) silently chunks, embeds, and stores them. as_query_engine() bundles a retriever with the LLM. query() runs the full retrieve-then-generate cycle and returns a grounded answer.

Run this once with a few real files in ./data and you've exercised the entire pipeline from post 3. It's intentionally minimal — no persistence, no tuning — so the shape is unmistakable. The remaining slides add the concerns you need before this becomes more than a demo.

Persistence is the first production concern because embedding is the expensive part. This pattern checks whether a ./storage directory exists: if it does, it reloads the prebuilt index from disk in milliseconds; if not, it builds the index and persists it for next time. The result is that you pay the embedding cost once, not on every run.

The StorageContext / load_index_from_storage pair is how LlamaIndex serializes the whole index — vectors, Nodes, and metadata — to local files. It's the simplest durable option. For real deployments you'd swap local storage for a managed vector store, but the load-if-exists-else-build logic stays the same.

This slide reinforces why the persistence pattern matters rather than introducing new code. Re-embedding a corpus on every startup is slow and runs up API costs for zero benefit when the data hasn't changed — yet it's one of the most common beginner mistakes (it reappears in post 5). Persisting decouples the expensive ingestion path from the cheap query path.

The production note is important: local file storage is fine for development and small corpora, but at scale you point LlamaIndex at a dedicated vector database. The framework abstracts this, so moving from local storage to a managed store is a configuration change, not a rewrite.

Citing sources is what separates a trustworthy RAG system from a confident guesser. Setting similarity_top_k=3 retrieves three Nodes, and the response object carries both the synthesized text (resp.response) and the source_nodes that fed it. Iterating over source_nodes prints each one's similarity score, source filename, and a content preview.

Make reading source_nodes a habit, not an afterthought. During development it's how you verify the answer is actually grounded in the right passages. In production it's how you show users (or auditors) where an answer came from. An answer without inspectable sources is an answer you can't fully trust.

This flow diagram maps the four key calls in the script to the pipeline concepts from post 3, so the code and the theory line up explicitly. load_data() turns a folder into Documents; from_documents() chunks, embeds, and stores them; as_query_engine() assembles a retriever plus the LLM; query() returns an answer with its sources.

Seeing the function names sitting on the pipeline is reassuring: the abstractions you learned aren't hidden behind hundreds of lines. Four method calls cover the entire flow, and each one corresponds to a stage you can now name and customize.

The model-swapping slide demonstrates the 'high ceiling' promise from post 2 in concrete code. Settings is a global configuration object: set Settings.llm to choose the generation model and Settings.embed_model to choose the embedding model. Here we use a cheaper OpenAI model for generation and a local HuggingFace model for embeddings.

Local embeddings (like BAAI/bge-small-en-v1.5) are significant: they run on your machine, cost nothing per call, and keep your text from leaving your infrastructure — useful for privacy and for high-volume indexing where per-call embedding fees add up. The key caveat: changing the embedding model invalidates any previously built index, because the vectors live in a different space. Re-embed after switching.

The same index supports conversation, not just one-shot lookups. as_chat_engine() wraps retrieval in a chat loop that remembers prior turns, so a follow-up like 'and how do I start one?' resolves against the earlier question without you re-stating context. chat_mode='context' retrieves relevant Nodes each turn and keeps the running history in the prompt.

This is the bridge from a static Q&A tool to a usable assistant over your data. The retrieval mechanics are identical to the query engine — same Index, same Nodes — but the chat engine adds memory of the dialogue. For longer conversations, other modes like 'condense_question' rewrite the follow-up into a standalone query before retrieving, which keeps retrieval sharp as history grows.

The run-it-yourself checklist turns the post into action. Drop PDFs or text files into ./data; the first run embeds and persists, later runs reload from ./storage; raise similarity_top_k if answers feel like they're missing context; read resp.source_nodes to confirm grounding; and swap embed_model to a local model to go free and private.

Each item maps to a slide above, so this doubles as a quick-reference card. The throughline is that a working RAG app is genuinely close — a handful of lines plus a few deliberate choices about persistence, retrieval depth, and models.

You now have a complete, runnable LlamaIndex application. Post 5 closes the crash course by cataloging the mistakes that quietly wreck RAG quality — bad chunking, wrong top-k, ignored metadata, needless re-embedding, and blind trust in answers — so the system you just built actually performs. Save this code post; it's the template you'll come back to.