Logistic Regression

This is the hands-on post. Theory becomes muscle memory only when you run the code, so everything here is copy-paste runnable on the breast-cancer dataset bundled with scikit-learn. We walk the complete professional workflow: import, split, scale, fit, predict, evaluate beyond accuracy, and interpret.

The dataset is a good choice — real medical features, two classes, mild imbalance — so the steps mirror what you'd do on an actual problem rather than a toy. Type it out and change things: drop the scaler, flip a parameter, and watch how the metrics respond.

Step zero is the imports, and they preview the entire pipeline. load_breast_cancer gives data, train_test_split handles the holdout, StandardScaler does feature scaling, and LogisticRegression is the model. The pip comment is there because scikit-learn isn't in the standard library.

Grouping imports by their role in the workflow is a small habit worth keeping: it makes the script self-documenting. Anyone reading these four lines can already guess the shape of what follows — load, split, scale, fit — before reading a single line of logic.

Loading and splitting is the foundation, and the details matter. We pull features X and target y, then split off 20% as a test set. Two arguments are doing important work: stratify=y preserves the class proportions in both splits, which matters because the dataset is imbalanced; and random_state=42 makes the split reproducible so your numbers match across runs.

The printed shape, (455, 30), confirms 455 training rows and 30 features. Always eyeball shapes after a split — a surprising number here is the fastest way to catch a data-loading bug before it silently corrupts everything downstream.

Scaling is non-optional for logistic regression in scikit-learn, and the reason is concrete. The model sums weighted features, and the default L2 regularization penalizes large weights uniformly. If one feature ranges in the thousands and another in fractions, the penalty effectively punishes them unequally, and the optimizer struggles to converge.

Standardizing each feature to mean 0 and variance 1 puts them on equal footing, so the regularization is fair and convergence is fast. This isn't a cosmetic preprocessing step — skipping it on this dataset will trigger convergence warnings and can meaningfully change which coefficients dominate.

This snippet scales correctly, and the comment carries the critical lesson: fit the scaler on training data only, then transform both train and test with those fitted statistics. fit_transform on train learns the means and variances; transform on test merely applies them.

The reason is that the test set must simulate truly unseen data. If you let the scaler peek at test statistics, information from the future leaks into preprocessing, your scores inflate, and the model underperforms in production. This data-leakage trap is so common it gets its own dedicated mistake in post 5, and the clean fix — a Pipeline — appears there.

Fitting is anticlimactic by design: instantiate LogisticRegression and call fit. The only argument we set is max_iter=1000, raised from the default 100 because the solver sometimes needs more iterations to converge on this data. The resulting test accuracy lands around 0.974.

The comment is the conceptual payoff from post 3: because the problem is convex, this fit finds the one global optimum. There's no randomness to manage, no need for multiple restarts. Run it twice and you get the same model — the reproducibility we promised, now demonstrated in two lines.

This step shows the distinction at the heart of post 1: predict gives hard labels, predict_proba gives probabilities. We take column index 1 of predict_proba to get P(y=1). The labels come out as 0s and 1s; the probabilities reveal confidence — 0.991 is a near-certain positive, 0.004 a near-certain negative, 0.887 a confident-but-not-certain positive.

Keep the probabilities around in real work. They let you tune the threshold, rank cases by risk, compute AUC, and feed expected-value decisions. Collapsing straight to labels throws away exactly the information that makes logistic regression valuable in the first place.

Accuracy is a trap on imbalanced data, so this step insists on better metrics. The confusion matrix shows the four outcome types; precision and recall separate the two error types; ROC-AUC measures ranking quality independent of any threshold.

The medical framing makes the stakes vivid. A false negative here means telling someone with a malignant tumor that they're fine — catastrophically worse than a false positive that triggers an extra test. Your evaluation metric must encode that asymmetry, which usually means prioritizing recall on the malignant class rather than chasing a single accuracy number.

This snippet runs the real evaluation. classification_report prints per-class precision, recall, and F1, so you can see how the model treats each class separately rather than in aggregate. roc_auc_score on the probabilities returns about 0.997, indicating excellent ranking: the model almost always assigns higher probability to true positives than to true negatives.

Note that AUC takes the probabilities, not the labels — it's threshold-independent by construction. Pairing the per-class report (threshold-dependent) with AUC (threshold-independent) gives a complete picture: how good the ranking is, and how well it translates into decisions at your chosen cutoff.

Interpretation closes the loop back to post 2's promise. We sort the coefficients by absolute magnitude and print the top three feature names with their weights. The largest-magnitude features are the ones driving the model's decisions most strongly; their signs tell you direction.

Because we scaled the features first, the coefficients are directly comparable in magnitude — a crucial detail. On unscaled data, a large coefficient might just reflect a small-scale feature, not real importance. Scaling makes 'biggest coefficient = most influential' an honest statement, which is why interpretation comes after scaling in any rigorous workflow.

This pipeline diagram is the whole post in four stages: split, scale (fitting only on train), fit the model, and evaluate with AUC and recall. It's the canonical supervised-learning workflow, and logistic regression slots into the 'fit' box like any other estimator.

Internalizing this shape pays off far beyond logistic regression. Swap the model box for a random forest or a gradient booster and the surrounding stages are identical. The discipline — proper split, leak-free scaling, honest evaluation — is what separates a trustworthy result from a misleading notebook, regardless of which algorithm sits in the middle.

These pitfalls are the workflow's sharp edges, gathered in one place. Scaling test data with its own statistics leaks information. A convergence warning means bump max_iter or scale your features. Forgetting stratify on imbalanced labels can produce a test set that misrepresents the problem. And judging on accuracy alone hides per-class failures.

Every one of these turns a 'working' notebook into a model that disappoints in production. Internalize them now, because the next post is entirely about mistakes — and several of these reappear there with deeper treatment and concrete fixes.

You now have logistic regression working end to end and, thanks to the earlier posts, you understand every step rather than just running it. Split, scale without leaking, fit a convex model, predict probabilities, evaluate with metrics that match your costs, and interpret comparable coefficients.

The final post is the field guide to what goes wrong. We'll catalog the six mistakes that most often quietly wreck a logistic regression model — and the specific fix for each — so your reliable baseline stays reliable when it actually matters.