What is Machine Learning?

This cover sets up the mistakes post with the most important warning about machine learning: the dangerous failures don't throw errors. Your code runs cleanly, your notebook reports a stellar score, and only in production — or never — do you discover the model is broken. A crash is easy to fix; a confidently wrong model can ship undetected and cause real harm.

The post is a field guide. For each common trap it names the symptom, explains the underlying cause, and gives the habit that prevents it. These are the errors that catch beginners and experienced practitioners alike, precisely because the tooling will happily let you fool yourself without any complaint.

The first and most insidious trap is data leakage: when information that shouldn't be available at prediction time leaks into training, letting the model effectively cheat. The textbook example is fitting a scaler or imputer on the entire dataset before splitting, so statistics computed from the test rows quietly influence the training pipeline. The reported score looks spectacular, then collapses in production where that future information isn't available.

Leakage is dangerous because it's invisible in the metrics — the model genuinely scores well on your contaminated evaluation. The cure is a strict ordering: split first, then learn every transformation, threshold, and parameter from the training set alone, applying them unchanged to the test set. Treating the test set as truly unseen at every step is the only reliable defense.

This code slide gives the concrete fix for leakage during preprocessing. The wrong approach fits the StandardScaler on the full X before splitting, which lets the scaler's mean and standard deviation absorb information from the test rows. The right approach fits the scaler only on X_train, then reuses that same fitted scaler to transform both the training and test sets.

The principle generalizes far beyond scaling. Any step that learns something from data — imputation values, encoding categories, feature selection, even choosing a threshold — must be fit on training data only and then applied to test data. In practice, wrapping these steps in a scikit-learn Pipeline enforces the ordering automatically, which is why pipelines are the recommended way to make leakage structurally hard to commit.

The second trap is trusting accuracy blindly, especially on imbalanced problems. When one class dominates — 99% legitimate transactions, say — a degenerate model that always predicts the majority class achieves 99% accuracy while being completely useless for the rare class you actually care about. The mistake is reporting that single number and stopping there, mistaking a high figure for a good model.

The fix is to always pair accuracy with metrics that reveal per-class performance: precision, recall, F1, or AUC, plus the confusion matrix to see exactly where errors fall. The metric you optimize should match what the problem cares about — catching every fraud case favors recall, never falsely accusing a customer favors precision. Choosing the right metric is as consequential as choosing the model.

This code slide shows the fix for the accuracy trap by reaching for balanced metrics. The macro-averaged F1 score combines precision and recall into one number and, by averaging across classes equally, refuses to let the majority class hide failure on minority classes. The ROC AUC score evaluates how well the model ranks positives above negatives across all thresholds, which is often the most informative single number on imbalanced data.

The broader lesson is metric literacy. Different metrics answer different questions, and reporting only accuracy is like judging a car by its top speed alone. Knowing when to reach for F1, when AUC is appropriate, and when you should simply read the confusion matrix directly is a core practical skill that separates trustworthy evaluations from misleading ones.

The third trap is overfitting: building a model so flexible that it memorizes the training data — including its noise and quirks — rather than learning the generalizable pattern. The unmistakable tell is a large gap between training and test performance, such as a model that scores 100% on training but 61% on test. It has effectively memorized the answers to the practice exam.

The remedies follow from the cause. More training data gives the model more genuine pattern and less room to memorize. A simpler model or explicit regularization limits its capacity to fit noise. Early stopping halts training at the point validation performance peaks. Above all, the test set is your truth serum: only performance on data the model never saw reveals whether you've overfit, which is why the held-out set is sacred.

This bar diagram contrasts the three outcomes by their test accuracy and the revealing train-versus-test pattern behind each. A good fit shows high test accuracy with training only slightly higher (94 train, 92 test) — the small gap signals healthy generalization. Overfitting shows a perfect or near-perfect training score collapsing to mediocre test accuracy (100 train, 61 test) — the yawning gap is the diagnosis. Underfitting shows both scores low and close together (60 train, 58 test) — the model is too weak to learn the pattern at all.

The visual cements the most useful diagnostic habit in ML: always compare training and test scores, never look at one in isolation. The relationship between the two — not either number alone — tells you whether to add capacity, add data, or add regularization.

The fourth trap is evaluating a model on the very data it was trained on. The slide's analogy is exact: it's like grading students using the precise questions they studied beforehand. A high score under those conditions measures memorization, not understanding, and tells you nothing about how the model will handle new inputs.

The consequence is a falsely rosy picture that evaporates on deployment. A perfect training score is not an achievement to celebrate — it's the absence of a real test, and often a warning sign of overfitting. The discipline is simple and absolute: maintain a held-out set the model never sees during training and consult it only to estimate real-world performance. Honest evaluation requires honest separation of training and testing data.

The fifth trap is the oldest principle in computing applied to ML: garbage in, garbage out. No algorithm, however sophisticated, can extract a correct pattern from data that is wrong. Mislabeled examples teach the model the wrong answers. Irrelevant features bury the signal in noise. A sample that doesn't represent the deployment environment teaches the model about the wrong world entirely.

The practical implication reorders most people's priorities. Beginners obsess over choosing and tuning models, but in real projects the largest gains almost always come from better data and better features — fixing labels, gathering more representative samples, engineering more informative inputs. A modest model on excellent data routinely beats a sophisticated model on poor data, which is why data quality, not model selection, is where serious effort belongs.

The sixth trap is forgetting to scale features for models that are sensitive to magnitude. Distance-based methods like k-NN, margin-based methods like SVMs, and gradient-trained models like logistic regression and neural networks all implicitly assume features live on comparable scales. Leave a feature measured in the thousands (income) next to one measured in the tens (age) and the large-magnitude feature dominates the distance or gradient, drowning out the smaller one regardless of its true importance.

The fix is standardization — rescaling each feature to comparable units, typically zero mean and unit variance. The critical detail, echoing the leakage lesson, is to fit the scaler on the training data only and then apply it to both train and test. Note that tree-based models like random forests are immune to feature scale, which is one reason they're such forgiving defaults.

This decision tree is a quick triage for the most common reason to be suspicious of an ML result: a score that looks too good. If the score is suspiciously high, first ask whether anything saw the test set early — if so, you have leakage and must move all preprocessing after the split. If nothing leaked, check the gap between training and test scores to diagnose overfitting. If the score isn't suspiciously high, ask whether the classes are imbalanced; if they are, switch from accuracy to F1 or recall, and otherwise the result is probably trustworthy pending one more validation.

Keeping this short flow in mind turns most everyday ML doubts into a fast, deterministic check rather than an open-ended worry. The two failure modes it front-loads — leakage and overfitting — account for the large majority of models that look brilliant in a notebook and fail in reality.

These bullets collect the preventive habits in one place: split first and only then preprocess, never evaluate on training data, match the metric to the actual problem, watch the gap between training and test scores, and fix the data before reaching for a fancier model.

None of these habits is difficult, and adopting them as defaults is what separates practitioners who occasionally ship confidently wrong models from those who essentially never do. The mistakes are predictable and repetitive — leakage, bad metrics, overfitting, dirty data — which is exactly what makes them preventable once you've seen each one named and understood why it bites.

This closes Day 27 and the topic of what machine learning is. With the core definition, the motivation, the training mechanics, a full worked example, and the common mistakes all covered, you have a complete working understanding of what ML is and how to do it honestly. The CTA points to Day 28, which begins a new topic, so following keeps the series going.