Chain-of-Thought Prompting

This cover frames the entire post: the difference between asking a model for an answer and asking it to reason toward one. The hook is deliberately concrete — a model that 'guesses' versus a model that 'reasons' — because that contrast is the whole intuition behind Chain-of-Thought.

The goal of post one is orientation. Before we argue why CoT matters or wire up code, you need a clear picture of what it actually is: a prompting technique that makes the model write out intermediate steps before the final answer.

Chain-of-Thought prompting is, at its core, a change in what you ask the model to output. A direct prompt maps a question to an answer in one move. A CoT prompt asks the model to first produce a sequence of intermediate reasoning steps in natural language, and only then state the answer.

The key insight is that those steps are not cosmetic. Because a language model generates text autoregressively, every step it writes becomes part of the context for the next token it predicts. So the act of writing the reasoning genuinely changes the computation that produces the final answer — it is not the same model doing the same work with extra narration bolted on.

This is why CoT can lift accuracy substantially on problems that require more than one step, without any change to the model's weights.

The comparison slide makes the two regimes side by side. A direct answer is a single shot: the model has one forward pass to land the conclusion, with no visible reasoning and no way to inspect where it went wrong. On a multi-step problem that single pass often isn't enough.

Chain-of-thought instead routes the question through written steps. Those steps are inspectable — when the answer is wrong, you can usually see exactly which step broke. And because each step constrains the next, the model is far more likely to hold the thread on multi-step logic and arithmetic.

The practical takeaway: CoT trades tokens for reliability and transparency on the class of problems where a single guess tends to fail.

This is the most important conceptual point in the post, so it gets stated plainly: the steps are computation, not commentary. People often imagine the model 'already knows' the answer and is just narrating. That's the wrong mental model.

A transformer does a fixed amount of work per forward pass, and it produces exactly one token before everything repeats. When the model writes a reasoning step, it gets to run additional forward passes whose inputs include that step. So generating intermediate text is literally how the model spends more computation on a hard problem.

Understanding this reframes prompting: 'think step by step' isn't a magic phrase, it's an instruction to allocate more compute to the answer by producing more intermediate tokens.

The flow diagram contrasts the two paths on a concrete example: 23 x 17. The direct path jumps straight to a number and is liable to be a confident guess. The CoT path decomposes the multiplication into 20x17 and 3x17, then sums the partial products.

The decomposition matters because each sub-result is something the model can get right with high confidence, and once it's written down it conditions the next step. The hard problem (a two-digit multiplication) becomes a chain of easy problems whose results compose into the answer.

This is the mechanism in miniature, and the rest of the series builds on exactly this picture.

Zero-shot and few-shot CoT are the two main ways to elicit the behavior, and knowing the difference saves a lot of confusion. Zero-shot CoT simply appends a cue — most famously 'Let's think step by step' — and trusts the model to generate its own reasoning structure. It's cheap, general, and requires no examples.

Few-shot CoT instead places two to four fully worked examples in the prompt, each showing a question, its reasoning, and its answer. The model imitates that pattern on the new question. This is usually more reliable, especially when the reasoning format matters, because you're demonstrating exactly the structure you want.

Which to use is a tradeoff: zero-shot for quick, broad coverage; few-shot when you can craft good exemplars and need consistency.

This code slide shows the absolute simplest form of CoT: a question followed by 'Let's think step by step.' The comment shows what the model then produces — a short chain (23 - 9 = 14, then 14 + 14 = 28) ending in the final answer.

The point of keeping it this minimal is to show there's no machinery required to get started. The trigger phrase alone shifts the model into producing structured reasoning, because in its training data that phrasing is overwhelmingly followed by step-by-step working.

Later posts add parsing, few-shot exemplars, and self-consistency, but everything builds on this one-line idea.

The decision diagram answers the practical question people actually have: when should I use CoT at all? The first branch asks whether the task needs multiple steps. If it's a single lookup or a one-step classification, CoT mostly wastes tokens and can even hurt.

If the task is multi-step, the second branch asks whether the individual steps are error-prone. If they are — arithmetic, chained logic, planning — CoT pays off strongly. If the steps are trivial, CoT still helps a little but the gain is modest.

The explicit message is that CoT is a tool with a cost, not a default. Matching it to genuinely multi-step, error-prone tasks is where it earns its keep.

The tips slide condenses the post into a portable mental model. CoT means writing reasoning before the answer; each step conditions the next token; zero-shot uses a trigger phrase while few-shot shows examples; and the technique shines on multi-step reasoning rather than simple lookups.

If a reader remembers only these five lines, they have the working intuition they need to start using CoT correctly and to follow the deeper mechanics in the next posts.

The closing mistake addresses the most seductive misconception about CoT: that the written steps are a faithful explanation of how the model decided. They are not. The chain is a generated artifact that, empirically, improves the answer — but it is not a transcript of the model's internal computation.

Research on faithfulness has shown models can produce reasoning that doesn't match the factors actually driving their output. So you should value CoT for what it reliably does — boost accuracy and give you something inspectable — without mistaking it for a literal window into the model's mind. This distinction matters enormously once you start relying on CoT in high-stakes settings.

The CTA points forward to post two, which shifts from 'what it is' to 'why it matters' — the accuracy jumps it produced, the fact that it's an emergent ability of scale, and the family of techniques it unlocked. The teaser sets up that this seemingly small prompting trick had outsized consequences.