Linear Regression

This is the foundational post of the topic, so the goal is to install the right mental model before any math or code arrives. Linear regression is the oldest and most widely used predictive model in existence, and almost every more sophisticated technique can be understood as a relaxation of its assumptions. Getting the vocabulary and the core picture right here pays off across the entire machine learning curriculum.

The cover frames the post as a map rather than a deep dive. You don't need to understand optimization or evaluation yet — you need to know what the object is, what its parts are called, and what problem it solves. Everything that follows builds on this.

The single most important idea in this post is that linear regression is just a weighted sum. Each input feature is multiplied by its own learned weight, and those products are added together along with a bias term. The word 'weight' is literal: it's how much that feature counts toward the final number. A house's price is built by adding up the contribution of its size, its number of bedrooms, its age, and so on.

The phrase 'no curves, no layers' is deliberate. People coming from deep learning often expect hidden complexity. There is none here, and that simplicity is the whole point — it's what makes the model fast, interpretable, and hard to overfit. The flatness (a line in 2D, a plane in higher dimensions) is a feature, not a limitation.

The equation y = w1*x1 + w2*x2 + ... + b is worth memorizing because every term has a name you'll use constantly. The x values are features (your inputs), the w values are weights or coefficients (what the model learns), and b is the bias or intercept. The bias deserves special attention: it's the predicted value when every feature is zero, which anchors the line vertically.

'Fitting' is the verb for finding the weights and bias. When you call model.fit() later, this equation is exactly what's being solved — the computer searches for the specific numbers that make the line hug the data. Understanding that fit() just sets w and b removes a lot of mystery from the library calls.

The diagram traces the flow from raw inputs to a single output number. Features enter on the left, each gets scaled by its learned weight, the results are summed with the bias, and a prediction emerges. This left-to-right picture is the simplest possible 'forward pass' and it's the same shape, conceptually, as a single neuron in a neural network — which is why linear regression is such a useful stepping stone.

Keeping the example concrete (size, bedrooms, age, price) anchors the abstraction. Learners retain the structure better when they can attach it to a tangible prediction problem.

Residuals are the heart of how the model gets trained, so introducing them early matters. A residual is simply actual minus predicted — the vertical gap between a real data point and the line. Some points sit above the line (positive residual), some below (negative). No line can pass through every point, so there will always be residuals.

The key insight to plant here is that fitting the model is the act of minimizing those gaps collectively. We don't optimize one point; we find the single line that makes all the residuals as small as possible at once. This sets up the cost function in the 'How It Works' post without yet getting into the math.

This slide clarifies what 'linear' actually constrains, which is a subtle point beginners often miss. Linear doesn't just mean 'a straight line on a graph' — it means each feature has a constant effect. Adding 100 square feet adds the same number of dollars whether the house is tiny or enormous, and that effect doesn't depend on the other features' values.

That assumption is sometimes wrong — real relationships can bend or interact — and naming this now sets up the 'Common Mistakes' post later. But it's worth stressing that the assumption is often good enough. Many real-world relationships are approximately linear over the range you care about, and the simplicity buys you speed, stability, and interpretability.

Distinguishing regression from classification is essential context because the two are constantly confused, and the entire 100 Days series will return to both. Regression predicts a continuous quantity — a price, a temperature, an age — where being 'close' matters and error is measured by magnitude. Classification predicts a discrete category, where you're either right or wrong about the label.

The comparison diagram makes the split crisp. The same dataset can support either task depending on the question: predicting a house's exact price is regression; predicting whether it sells above or below median is classification. Tomorrow's lens (logistic regression) is exactly the bridge between the two, so this distinction is laying track for later.

Even in a concept post, showing the three-line scikit-learn version grounds the abstraction in something runnable. Import the class, call fit() to learn the weights and bias from training data, call predict() to apply the weighted sum to new inputs. That's the entire interface most people will ever touch.

The comment 'learn weights + bias' ties the code directly back to the equation from earlier slides, reinforcing that the library is doing exactly what the math described — nothing hidden. The full runnable build comes in post 4; this is just a teaser that the concept is one import away from reality.

This slide separates simple from multiple linear regression, a distinction that trips up beginners reading documentation. Simple linear regression has exactly one feature — it's the classic line through a scatter plot you saw in school. Multiple linear regression has several features at once and fits a flat plane or hyperplane through higher-dimensional space.

The reassuring point is that the math is identical; you just stack more weights into the sum. There's no conceptual leap from one feature to fifty. Almost every real model is the multiple kind, so it's worth normalizing that 'linear regression' usually means many inputs, not one.

The vector diagram offers a geometric intuition: data points spread out as a cloud, and the fitted line threads through them capturing the dominant trend. The model's job is to find the orientation of that line so it best represents the bulk of the points.

Geometry is a powerful complement to algebra for this topic. Some learners click with the equation, others with the picture of a line minimizing its distance to a scatter of points. Offering both increases the odds the concept lands.

The vocabulary recap exists because these five terms — feature, weight/coefficient, bias/intercept, target, residual — are used relentlessly in every regression discussion, and beginners constantly mix them up (especially 'coefficient' versus 'weight,' which are the same thing, and 'bias' versus 'intercept,' also the same thing).

Consolidating them on one slide gives a reference learners can screenshot and return to. Once these words are automatic, reading any regression tutorial, paper, or library doc becomes dramatically easier. This is the payoff of a concept post: shared language.

The closing card sets up the natural next question. Now that you know what linear regression is, the obvious follow-up is why anyone still uses such a simple, old model in an era of deep learning. That's exactly the 'Why It Matters' angle of the next post.

The teaser frames the surprise — a 200-year-old model still beating fancy ones — to create a small open loop that pulls the reader forward to Day 36's first post.