MLOps in 8 Slides

This is the build post, so the detail entries focus on why each piece of code is shaped the way it is, not just what it does. The throughline is that this is the shape of a real pipeline in miniature: validation, tracking, registration, a quality gate, serving, and a drift check, wired together so 'train a model' becomes a repeatable, gated, observable process. It's a paste-and-grow skeleton, not a throwaway demo.

This step establishes setup the practical way: install the core stack — MLflow for tracking and registry, scikit-learn for the model, pandas for data, scipy for the drift test, FastAPI for serving — and name the experiment and model up front as constants. Naming them once means the rest of the script refers to a single source of truth for where runs and model versions live.

The choice to centralize MODEL_NAME and the experiment name matters more than it looks. These strings tie together training, the registry, serving, and rollback later in the post; keeping them as constants is what lets the same model be referenced consistently across every stage. It's the small discipline that prevents the 'which model was that?' confusion the mistakes post warns about.

The validate function is the gate that runs before any training, implementing the data-validation stage from the how-post. It checks that the expected columns are present, that each has the expected dtype, and that no column exceeds a 5% null rate, raising immediately if any check fails. Only data that passes proceeds to training.

Centralizing these checks in one function is what makes them enforceable. Rather than scattering ad-hoc assertions through a notebook, every dataset entering the pipeline passes through this single gate, so bad data is rejected at the boundary in milliseconds instead of corrupting a model that trains for an hour. In a real system these rules would be richer — value ranges, category sets, row counts — but the pattern is exactly this: fail fast, fail loud, at the entrance.

This step trains the model and tracks the run, fusing the training and experiment-tracking stages. Inside an MLflow run it fits a random forest, computes validation AUC, logs that metric, and logs the model — crucially passing registered_model_name, which both stores the artifact and creates a new version in the registry in one call. The run id is captured for reference.

The important design choice is that tracking wraps ordinary training code with almost no friction: the fit and scoring lines are normal scikit-learn, and the MLflow calls just record what happened. Registering the model at log time means every trained model automatically becomes a versioned registry entry, so there's no separate manual step where a model could get lost or mislabeled — the three-artifacts discipline enforced by default.

The quality gate is the safety mechanism that separates this from a script that ships whatever it trains. After training, the code compares the run's AUC against a hard threshold; if the model fails to clear it, the pipeline exits with an error and nothing is promoted. Only if it passes does the code transition the new version to the Production stage in the registry.

This gate is the runnable version of 'the loop won't promote a regression.' It encodes a policy — no model below 0.85 reaches production — as code that runs automatically on every pipeline execution, so a bad training run can't silently ship. Promotion being a stage transition rather than a file copy is what makes this both safe and instant, and it's the hook that the rollback fix in the mistakes post reuses.

The decision tree shows the gating logic in full, going a step beyond the code: first, did the new model clear the metric gate? If not, block the deploy and alert the team. If it did, ask whether it actually beats the current Production model; promote only if it's genuinely better, otherwise keep the incumbent. This two-level check is the guardrail that lets retraining run unattended.

The tree captures a subtlety the simple threshold misses: clearing an absolute bar isn't enough — the new model also has to be better than what's already live, or there's no reason to take on the risk of a swap. Encoding both checks means automation can be trusted to retrain on a schedule without a human approving each release, because the pipeline itself refuses to promote anything that isn't a clear improvement.

This step serves the Production model, implementing the serving stage. A FastAPI app loads the model by its registry stage — models:/churn-clf/Production — rather than a file path, then exposes a /predict endpoint that turns an incoming JSON row into a DataFrame and returns the churn probability. The comment shows it runs under uvicorn as an ordinary web service.

Loading by stage rather than path is the load-bearing detail: when the gate promotes a new version, this exact serving code picks it up with no change, because it always asks the registry for whatever is currently Production. That's what lets model updates and rollbacks happen entirely in the registry while the serving layer stays untouched — the clean boundary between model lifecycle and application that the how-post described.

The final step implements monitoring as a drift check. The drift_check function runs a Kolmogorov–Smirnov test comparing a training feature against the same feature in live data, returns the p-value and a boolean, and if drift is detected, triggers the retraining pipeline. This is the arrow that closes the loop from serving back to training.

The reason to monitor inputs rather than wait for accuracy is that ground-truth labels arrive late or never, so input drift is the earliest actionable signal that the world has shifted. Wiring the drift result directly to trigger_retraining_pipeline() is what makes the system self-healing: degradation in the input distribution automatically kicks off a fresh run through everything above, which gets gated and promoted only if it's better.

The flow diagram shows how the five pieces wire together at runtime: validate() gates bad data, train-and-log records the run, the quality gate enforces the AUC bar, serve exposes /predict, and drift_check triggers retraining. Each box has one job, which is what makes the skeleton extensible rather than a tangle.

Seeing the composition clarifies where real-world additions slot in. Logging wraps the serve box; canary or shadow deploys sit between the gate and full promotion; scheduling wraps the whole chain; richer validation lives inside validate(). Because the stages are separated, each can be hardened independently — the simple pipeline grows into a production system by thickening individual boxes, not by rewriting the flow.

The checklist condenses the post into the practices that turn this skeleton into something production-grade. Pin library and data versions so runs are reproducible. Compare against the live model, not just an absolute gate, so you never swap in a worse model. Add canary or shadow deploys to catch regressions on real traffic before full rollout. Schedule the pipeline so it runs unattended. And log every prediction so you have the data to monitor later.

None of these are difficult individually; the value is in doing all of them, because each closes a gap that only shows up in production. The skeleton above is honest about being a skeleton — these are the specific places it needs reinforcement before real traffic, and each one maps to a failure mode the final post catalogs.

With a runnable, gated, observable pipeline in hand, the natural next question is what still goes wrong even with good tooling — the operational mistakes that sink ML projects despite a working model. That is exactly where the final post goes.

The transition is intentional: you've now seen the right way to validate, track, gate, serve, and monitor, so the mistakes post reads as a checklist of the specific ways teams deviate from these patterns and pay for it in unreproducible, undebuggable, silently-wrong systems.