What is Machine Learning?

This cover sets the tone for the mechanics post by promising to demystify training. To outsiders, 'the model learns' sounds almost mystical, but the reality is a tight, repetitive loop: guess, measure the error, adjust to reduce it, repeat. There's no magic, just optimization run at enormous scale.

The promise of the post is that you don't need heavy mathematics to understand what's happening. Once you see that learning is the search for parameters that minimize a loss, and that gradient descent is the method for that search, the behavior of essentially every model — from linear regression to deep networks — becomes a variation on one comprehensible theme.

At its core, training is optimization: a search for the parameter values that make the model's predictions as accurate as possible. The loop has four beats. The model makes predictions with its current parameters. A loss function measures how far those predictions are from the truth. An optimizer figures out how to change the parameters to reduce that loss. Then the parameters are updated and the loop repeats.

This framing is the single most useful thing to carry forward. Every supervised learning algorithm is some version of this loop; they differ in the shape of the model and the details of the optimizer, but the predict-measure-adjust-repeat rhythm is universal. Everything else in the post is detail layered on top of it.

This cycle diagram visualizes the four-beat training loop as a continuous circle, which is the right mental image because it runs over and over. Predict: the model produces an output from the current parameters. Measure loss: compare that output to the known answer and score the error. Compute gradient: determine which direction each parameter should move to lower the loss. Update params: take a step in that direction.

Drawing it as a cycle rather than a line emphasizes that no single pass teaches the model much; the power comes from repetition. Each lap shaves a little off the loss, and after many thousands of laps the parameters settle into values that map inputs to outputs well. Recognizing this loop lets you reason about training regardless of the specific algorithm.

This slide introduces the discipline that separates real ML from self-deception: splitting your data before you train. The training set is what the model fits its parameters on. The validation set is used to tune higher-level choices — model size, learning rate, which features to include — without contaminating your final judgment. The test set is held back and looked at exactly once, at the very end.

The slide calls this non-negotiable for good reason. A model evaluated on data it trained on will look far better than it really is, because it has effectively seen the answers. Only performance on data it never touched during training tells you how it will behave in the real world. Skipping the split is the root cause of countless models that dazzle in a notebook and fail in production.

This pipeline diagram makes the three-way split concrete as a left-to-right flow. The training portion is where the model fits its parameters. The validation portion is where you tune settings and compare candidate models. The test portion is locked away and used only once, to produce the final honest score.

Visualizing the three roles in sequence reinforces a strict hygiene rule: information must flow one way. Decisions are informed by training and validation, and the test set stays sealed until the end. The moment you start tweaking your approach based on the test score, that score stops being an honest estimate of real-world performance, and you've quietly slipped back into grading yourself on the answer key.

This slide defines the loss function, the quantity that makes 'learning' precise. A loss function collapses the question 'how wrong is the model right now?' into a single number that training works to minimize. For regression the common choice is mean squared error — the average of the squared differences between predictions and true values, which punishes large misses heavily. For classification the standard is cross-entropy, which scores how confident and correct the predicted probabilities are.

The key insight is that defining a loss turns a vague goal into a concrete optimization target. 'Be more accurate' is not something a computer can act on; 'make this number smaller' is. Every training run is literally a campaign to drive the loss down, which is why choosing an appropriate loss for your problem is a foundational decision.

This slide explains gradient descent, the workhorse algorithm behind most of modern ML, in plain terms. The gradient is the slope of the loss surface — for each parameter, it points in the direction that would increase the loss fastest. To reduce the loss you move the opposite way, downhill. Gradient descent takes a small step in that downhill direction, with the step size set by the learning rate, then recomputes the gradient and steps again.

The intuition of walking downhill on a foggy hillside is apt: you can't see the whole landscape, but you can feel the slope under your feet and step toward lower ground. Repeat enough times and you settle into a valley — a set of parameters with low loss. The learning rate matters a lot: too large and you overshoot the valley, too small and training crawls.

This code slide implements gradient descent by hand so the abstraction becomes mechanical and undeniable. It fits a single parameter w to data that follows y = 2x. Each iteration computes the model's predictions, derives the gradient of the squared-error loss with respect to w, and nudges w in the downhill direction scaled by the learning rate.

Watching w start at 0.0 and converge to roughly 2.0 over a thousand steps is the whole concept made visible: no library, no magic, just repeated small corrections driven by the gradient. The lesson is that the fit method in any ML library is doing a more elaborate version of exactly this loop. Seeing the bare mechanism once removes the mystery from every model.predict you'll ever call.

This slide explains epochs and why training iterates. One epoch is a single complete pass over the training data. Because a model almost never learns a good mapping in one pass, you run many epochs, each one applying the gradient-descent loop across all the data and nudging the parameters a little closer to a good fit.

The practical wrinkle is knowing when to stop. You monitor the loss on the validation set across epochs: while it keeps falling, the model is still learning useful structure; once it plateaus or starts rising even as training loss keeps dropping, the model has begun memorizing rather than generalizing. Stopping at that turning point — early stopping — is one of the simplest and most effective defenses against overfitting.

This comparison contrasts the two failure modes of fit, which sit at opposite ends of a spectrum. Underfitting means the model is too simple to capture the real pattern; it does poorly on both training and test data, and the cure is more capacity — a bigger model, richer features, or longer training. Overfitting means the model is too flexible and has memorized the training data, including its noise; it scores beautifully on training but poorly on test, and the cure is the opposite — more data, regularization, or a simpler model.

Seeing them side by side teaches the central tension of ML: you're steering between too little and too much flexibility, and the held-out test score is your steering signal. The gap between training and test performance is the single most informative number for diagnosing which way you've drifted.

This trace diagram shows the loss decreasing across epochs, making convergence concrete with numbers. At epoch 0 the loss is high because the parameters are essentially random. By epoch 100 it has fallen sharply, and by epoch 500 it's near zero — the model has found parameters that fit the data well. The comment labels this as gradient descent converging.

Reading the values top to bottom gives the felt sense of training: a steep early drop as the easy gains are captured, then a slow approach toward a floor. This curve is something you'll watch constantly in real work; its shape tells you whether training is progressing, stuck, or unstable, which is why recognizing a healthy descending loss curve is a core practical skill.

These bullets compress the entire mechanics post into a memorizable core. The loop is predict, measure, gradient, update, repeat. Loss is the single number capturing how wrong the model is. The gradient is the direction that lowers that loss. Always split your data and never evaluate on what you trained on. And stop training before the model tips from learning into memorizing.

If you hold only this slide, you understand how machine learning works at a level sufficient to reason about almost any model. The specific algorithms decorate this skeleton with different model shapes and optimizer tricks, but the skeleton — iterative loss minimization on properly split data — is what's really going on every time.

This closes the mechanics post and hands off to the hands-on one. With the engine understood — training as iterative loss minimization, gradient descent as the method, proper data splitting as the discipline, and overfitting as the danger — the next post walks a complete real example, training and honestly evaluating a working classifier with every line of code shown.