Chain-of-Thought Prompting

This cover signals a shift in register: from concepts and mechanics to working code. The framing distinguishes knowing CoT from being able to extract a reliable answer from it in production — which is where most of the real engineering effort actually goes.

Post four is deliberately code-heavy. The plan is laid out: prompt the model to reason, generate the chain, parse the answer deterministically, and optionally sample multiple chains and vote. Everything here is meant to be copied and adapted directly.

The first code block is a clean zero-shot CoT function. It appends 'Let's think step by step' to the question, calls the chat completion endpoint with temperature 0 for determinism, and sets max_tokens high enough to fit the whole chain plus the answer.

The two parameter choices matter. Temperature 0 makes the single chain reproducible, which is what you want when you're not doing self-consistency. The 512-token budget is generous enough that the reasoning won't get truncated mid-thought — the failure mode the previous post warned about. This function is the minimal viable CoT call you can build everything else on.

The second block shows few-shot CoT. A constant string holds two fully worked exemplars — each with a question, explicit reasoning, and an 'Answer:' line — and the helper appends the new question with a trailing 'A:' so the model continues in the same format.

The design choices are intentional: the exemplars model exactly the reasoning style and the delimited answer format you want, so the model imitates both. Two examples is usually enough to lock in the pattern; adding more costs tokens with diminishing returns. The trailing 'A:' is a small but important cue that tells the model it's now its turn to produce a worked solution.

The third block is the unglamorous but essential part: parsing. The function first tries to find an explicit 'Answer: X' pattern with a regex, which is the reliable path when you've instructed the model to produce one. If that fails, it falls back to taking the last number in the text.

The ordering reflects a real priority: prefer the structured signal you asked for, and only guess as a last resort. The fallback is explicitly a fallback because, as a later slide shows, grabbing an arbitrary number is brittle — intermediate steps are full of numbers. Returning None when nothing parses lets the caller handle failures explicitly instead of silently propagating garbage.

The fourth block implements self-consistency, the highest-leverage upgrade to plain CoT. It samples k chains at a non-zero temperature so the reasoning paths differ, parses each one's answer, collects the valid answers, and returns the majority vote.

The two critical details: temperature 0.7 creates the diversity that makes voting meaningful — at temperature 0 every chain would be identical and voting would be pointless. And only successfully parsed answers are counted, so a chain that fails to produce a clean answer doesn't corrupt the tally. Majority voting works because correct reasoning tends to converge on the same answer while errors scatter, so the right answer is usually the modal one.

The pipeline diagram visualizes self-consistency end to end: one prompt fans out into k samples at temperature above zero, each sample is parsed into a candidate answer, and a majority vote selects the winner.

The shape captures why the method works and what it costs. The fan-out is where you spend k times the tokens; the vote is where scattered errors get outvoted by convergent correct reasoning. Seeing it as a pipeline also makes clear where you'd add logging or confidence thresholds — for instance, flagging cases where the vote is close rather than decisive.

The fifth block addresses machine-readability directly by asking the model to output valid JSON on its last line, containing both the reasoning and the answer. The parser then reads only that last line and pulls the answer field.

This is the most robust pattern for pipelines that consume the answer programmatically. JSON gives you a typed answer field and keeps the reasoning available for logging without it bleeding into your value extraction. The caveat — covered in the mistakes — is that you still need to handle the occasional malformed JSON, so production code wraps the json.loads in error handling.

The tips slide gives the production-tuning rules. Use temperature 0 for a single deterministic chain and a higher temperature (0.5 to 0.8) when sampling for self-consistency. Always demand a parseable 'Answer:' line. Cap k by cost — k=5 is a common sweet spot between accuracy and spend. And log the full chain so that when an answer is wrong you can see which step failed.

These are the settings that separate a demo from something you'd actually run at volume, and they connect every code block in the post to real operational concerns.

The cost-versus-accuracy bar chart makes the self-consistency tradeoff explicit with illustrative numbers: a single chain around forty percent, k=3 voting around fifty, k=5 around fifty-five — each step up multiplying token spend proportionally.

The shape shows diminishing returns: the jump from one chain to three is large, from three to five smaller. That's the practical reason k=5 is a common stopping point — beyond it you pay linearly more tokens for shrinking accuracy gains. The chart is the quantitative justification for treating k as a tunable cost dial, not a free knob.

The closing mistake targets the most common production bug: trusting the model's whole text as the answer. The chain is prose, and downstream code expects a value, so you must force a delimited answer and extract it deterministically.

It also flags a subtler self-consistency issue: when votes split evenly, silently picking one hides a low-confidence situation. Surfacing the tie — for instance, returning the distribution or flagging for review — is the honest engineering choice. Both points reinforce the post's theme that the parsing and aggregation layer is where CoT actually succeeds or fails in real systems.

The CTA hands off to the final post, which catalogs the common mistakes that quietly wreck CoT prompts in the wild — wrong tasks, misplaced trust, truncation, bad parsing, tiny models, and cost. The teaser frames it as the failure manual that completes the toolkit.