Fine-Tuning LLMs

This post sets the foundation for the whole day, so the cover deliberately frames fine-tuning as ownership: turning a general model into something that's specifically yours. The mental model to install early is that fine-tuning is an adaptation step, not a from-scratch process.

Everything else in the post — methods, trade-offs, when not to bother — hangs off getting this one framing right.

The definition matters because people conflate fine-tuning with training a model. They are different stages. Pretraining is where the model learns language, grammar, facts, and reasoning patterns from a vast corpus — that's the expensive, once-per-model phase done by the lab. Fine-tuning starts from those already-good weights and continues training on a far smaller, curated dataset.

The key word is 'shifts.' You are not building new capabilities from nothing; you are nudging billions of existing parameters toward a particular behavior. That's why a few hundred good examples can meaningfully change a model — the heavy lifting already happened.

This comparison gives the orders of magnitude that make the distinction concrete. Pretraining consumes trillions of tokens and runs for weeks on large GPU clusters; it's done once and you almost never do it yourself. Fine-tuning runs on hundreds to thousands of examples, finishes in hours on a handful of GPUs, and is something a small team iterates on repeatedly.

Understanding the scale difference also explains the economics later in the day: fine-tuning is cheap enough to be part of a normal product loop, whereas pretraining is a capital expense reserved for foundation labs.

Prompting versus fine-tuning is the decision most teams actually face, so it's worth being precise. Prompting steers a frozen model at inference time — you change the instructions, not the weights. It's instant to iterate and costs nothing to try, which makes it the correct first move almost always.

Fine-tuning changes the weights so the behavior is built in. The payoff is shorter prompts, tighter consistency, and behaviors that prompting can't reliably hold across many turns. The cost is data, compute, and an ongoing pipeline. The rule of thumb: exhaust prompting first, then fine-tune what prompting couldn't pin down.

This pipeline places fine-tuning in the larger lifecycle so it isn't seen in isolation. A model is pretrained, then optionally fine-tuned on task data, then often aligned with techniques like RLHF or DPO to match human preferences, and finally deployed.

Seeing the stages laid out clarifies that 'fine-tuning' in casual conversation can mean several different things. Supervised fine-tuning on input/output pairs is the focus of this day; alignment methods are a related but distinct topic we treat separately.

The methods slide is the practical spectrum, ordered cheapest to heaviest, and it doubles as a recommended order of attempt. Prompting and few-shot need no training at all. RAG adds knowledge by retrieving relevant text at query time without touching weights. PEFT methods like LoRA and QLoRA train roughly one percent of parameters. Full fine-tuning updates every weight.

The practical guidance is to climb this ladder only as far as you need. Most teams that think they need a full fine-tune actually get what they want from LoRA, and many that think they need fine-tuning at all really needed RAG or a better prompt.

Fine-tuning genuinely shines at a specific set of jobs, and naming them helps you recognize when it's the right tool. It's excellent at locking in an output format or schema, so the model reliably returns the JSON or structure you expect. It's strong at adopting a consistent tone or persona. It teaches narrow, repetitive skills well, and it lets you shrink long, brittle prompts by moving that behavior into the weights.

Notice the common thread: these are all about behavior and style, not about knowing new facts. That distinction is exactly what the next slide warns about.

The single most common misuse is treating fine-tuning as a way to teach the model new facts. It's poor at that. Facts injected via fine-tuning are absorbed lossily, can't be easily updated, and are prone to being hallucinated with confidence. When you need current or proprietary knowledge, retrieval (RAG) is the right tool — it keeps facts external, editable, and citable.

Two other limits: fine-tuning won't rescue a model that's simply too small for the reasoning the task demands, and it can't compensate for a messy dataset. Garbage examples produce a model that's reliably wrong.

This conceptual snippet shows the entire shape of fine-tuning in a handful of lines so the idea feels approachable before the heavy code in post 4. You load a pretrained model, hand a Trainer your dataset of input/output pairs, call train, and save the result.

The comment 'weights shift toward your task' is the whole point in one phrase. Behind that single train() call is the standard forward/loss/backward/update loop covered in post 3, but at the conceptual level it really is this simple: continue training, on your data, gently.

The decision diagram turns the day's advice into a quick triage you can run before spending any GPU time. First ask whether the problem is really about new facts — if so, route to RAG, not fine-tuning. If it's a behavior or format problem, fine-tuning is appropriate, and you should try LoRA before anything heavier.

If it's neither — the model just isn't following instructions well — the cheapest fix is usually a better prompt. Running this triage honestly prevents the most expensive mistake in the whole topic: training to solve a problem that training doesn't solve.

Closing the concept post with explicit 'when not to' guardrails is intentional, because the failure mode for fine-tuning is enthusiasm. If a better prompt solves it, stop there. If you have fewer than roughly fifty quality examples, you don't have enough signal to fine-tune well. If the need is fresh, changing facts, that's a retrieval problem.

And the discipline that underpins all of it: never fine-tune before you've measured a baseline. Without a number for the prompted version, you can't tell whether fine-tuning helped, and you'll end up shipping on faith.

The closing card teases the next post, which moves from 'what it is' to 'why it's worth doing' — the business and product case. Today established the concept and the guardrails; tomorrow makes the argument for when the investment pays off.

The through-line for the day is judgment: knowing not just how to fine-tune but when, and the concept post is where that judgment starts.