Decision Trees

This cover signals a deliberate gear change: the previous post explained the machinery, this one puts it under the reader's fingers. The promise is a complete, copy-paste-able pipeline on a real dataset, which is what the 'code example' angle demands.

Using the iris dataset is a conscious choice — it's small, built into scikit-learn, multi-class, and produces a tree shallow enough to read in full. That lets every later slide show real, reproducible output rather than hand-waved placeholders.

The install-and-import slide front-loads every dependency so the rest of the post runs without surprises. Listing the exact imports — the classifier, the splitter, export_text, and the metrics — also previews the structure of the post: load, fit, read, score.

Keeping imports in their own block is a deliberate good habit. It mirrors how a real notebook is organized and means a reader copying slide by slide never hits a missing-name error halfway through.

The load-and-split slide establishes the train/test discipline that the entire 'don't trust training accuracy' theme depends on. The fixed random_state=42 makes the split reproducible, so a reader running the code sees the same shapes and numbers shown here.

Printing X_tr.shape as (120, 4) anchors the scale: 120 training rows, 4 features, 3 species. Concrete numbers help readers verify their environment matches before moving on to fitting.

This is the heart of the build: three lines to configure, fit, and predict. The max_depth=3 is intentional — it keeps the resulting tree small enough that the export_text and diagram slides can show it in full, while still scoring well on iris.

Showing the first five predictions as actual class indices grounds the abstract 'predict' call in visible output. It also quietly reinforces that prediction is just walking each test row down the tree to its leaf, the mechanic from post 1.

Exporting the rules as text is the payoff slide that connects back to post 2's interpretability promise — here it is, the literal learned logic. The shown output reveals the tree's real strategy: split on petal width first, isolate setosa immediately, then separate versicolor from virginica at width 1.75.

Readers should notice the model rediscovered a genuine botanical fact: petal dimensions separate iris species cleanly. That a few greedy splits recover real structure is a satisfying demonstration that the algorithm finds signal, not arbitrary cuts.

The tree diagram visualizes exactly the rules the previous slide printed, giving readers two representations of the same model — textual and graphical. Seeing them side by side cements that the picture and the if/else text are literally the same object.

The structure is worth reading aloud: petal width <= 0.8 splits off setosa entirely in one question, which is why that branch is a leaf immediately. The other branch needs a second question to separate the two harder species. This asymmetry echoes the 'trees aren't balanced' point from post 1.

Feature importances are introduced here as a free, built-in explanation tool. The output shows petal width dominating at 0.92 while the sepal features score zero — the tree found it never needed them. That's a concrete, memorable demonstration of what importances communicate.

This slide also quietly sets up a trap defused in post 5: default impurity-based importances are convenient but biased toward high-cardinality features. Here, with clean low-cardinality data, they're trustworthy — but the reader should not over-generalize from this tidy example.

The scoring slide enforces the post's central discipline: judge on held-out data. accuracy_score gives a single headline number (~0.97 here), but classification_report breaks it into per-class precision, recall, and F1 — essential when classes differ in difficulty.

Introducing the full report rather than just accuracy is deliberate preparation for post 5's imbalance warning. Even on balanced iris, modeling the habit of looking past raw accuracy builds the instinct that matters more on skewed real-world data.

This explanatory slide unpacks what feature_importances_ actually measures: the total impurity reduction each feature contributed across all its splits, normalized to sum to one. Knowing the definition stops readers from treating importances as a black-box ranking.

The takeaway that petals do all the work and sepals do none is a clean, intuitive result. It also reinforces that a tree will simply ignore features it finds useless — there's no penalty for handing it irrelevant columns, which is part of why trees need so little feature selection up front.

The depth-sweep slide is the post's most important practical lesson: it shows overfitting happening in real numbers. Looping over depths and printing train and test scores together makes the divergence visible — training accuracy climbing to 1.0 while test accuracy peaks and then declines.

This is the empirical version of post 1's 'depth is the dial' and post 3's stopping rules. Letting readers run the loop themselves turns the abstract bias-variance trade-off into something they've personally observed, which is far stickier than being told about it.

The bars diagram distills the depth sweep into a memorable shape: test accuracy is low when underfit, peaks at a moderate depth, and falls again when the tree overfits. The inverted-U is the single most important picture in tree modeling.

Labeling the regimes — underfit, sweet spot, overfit — gives readers the vocabulary to diagnose their own models. When their test accuracy drops as they add depth, they'll recognize they've walked past the peak and need to prune back.

The recap orders the six pipeline steps so a reader can reproduce the entire workflow from memory: split with a seed, fit with a depth cap, read the rules, check importances, score with a report, and sweep depth. It's a portable template, not just iris-specific code.

Each bullet maps to a slide, so the recap doubles as an index. The emphasis on a fixed seed and a depth cap deliberately bakes in the two habits that prevent the most common beginner mistakes addressed in the final post.

The CTA pivots from the happy path to the cautionary one. Having built a working tree, the reader is ready to learn how the same code goes wrong — overfitting, instability, biased importances, leakage — which is exactly post 5.

Naming the specific traps in the teaser creates anticipation and signals that the series doesn't stop at 'it works in a notebook.' Real competence is knowing the failure modes, and that's the promise the final post delivers on.