Docker for ML

This cover signals the shift from concepts and mechanics to working code. The framing distinguishes understanding Docker from actually authoring the Dockerfiles, compose files, and commands that build a GPU training image, serve a model, and orchestrate both — which is where the real engineering time goes.

Post four is deliberately code-heavy. The plan is laid out: a CPU training image, a GPU image on the official CUDA base, a slim serving image exposing an API, a compose file that wires train and serve together, a .dockerignore, and the build-run-push commands. Everything here is meant to be copied and adapted directly into a real project.

The first block is a clean CPU training image. It starts from python:3.11-slim, sets a working directory, and installs the system packages many ML libraries need to build — build-essential and git — cleaning the apt lists afterward to avoid bloating the layer. Then it copies requirements and installs them before copying the code, preserving the cache-friendly ordering from the mechanics post.

Two deliberate choices stand out. The apt-get install uses --no-install-recommends and removes /var/lib/apt/lists to keep the layer lean. And it uses ENTRYPOINT rather than CMD for train.py, so the container is effectively a 'training command' you can pass arguments to — a good fit for a job-style image whose whole purpose is to run one thing.

The second block is the GPU variant, and the only fundamental change is the base image: nvidia/cuda:12.4.1-cudnn-runtime-ubuntu22.04. That official image ships the CUDA runtime and cuDNN, so the host only needs a compatible NVIDIA driver and the container toolkit. From there it installs Python and pip, then follows the same deps-before-code pattern.

The choice of the '-runtime-' rather than '-devel-' CUDA tag is intentional: runtime images carry what's needed to execute CUDA workloads without the larger development toolkit, keeping the image smaller. The comment flags the key contract — the driver is matched on the host, not in the image — which is the boundary the mechanics post established. This is the canonical starting point for any GPU training image.

The third block is the serving image, and it's deliberately separate from training. It starts slim, installs a smaller serving-only requirements file (FastAPI, uvicorn, joblib — not the full training stack), copies in the trained model directory and the server code, exposes port 8000, and launches uvicorn bound to all interfaces.

The separation matters: a serving image shouldn't carry training-only dependencies like data loaders or experiment trackers, which only bloat it and widen its attack surface. Binding to 0.0.0.0 is required so the server is reachable from outside the container, and EXPOSE documents the port for tooling and humans even though it's the published mapping at run time that actually opens it.

This block is the FastAPI server the serving image runs. It loads a serialized model once at import time with joblib, defines a typed request body with pydantic, and exposes a /predict endpoint that runs the model and returns a JSON prediction.

Loading the model at module load rather than per request is the important pattern — it happens once when the container starts, so each request is just a fast inference call, not a disk read and deserialize. The pydantic model gives you request validation for free: malformed input is rejected with a clear error before it ever reaches the model. Casting the prediction to float ensures the response is clean, JSON-serializable output rather than a numpy scalar that might not serialize.

The fourth block is the docker-compose file that orchestrates both images. It defines a trainer service built from the GPU Dockerfile, with data and model directories mounted as volumes so artifacts persist outside the container, and a GPU reservation under the deploy section so the container gets device access. The api service builds from the serving Dockerfile, publishes port 8000, and depends on the trainer.

Compose is the right tool here because it captures the multi-container topology — who builds from what, which volumes mount where, which ports publish, and the start ordering — in one declarative file. The GPU reservation block is the compose-native way to request devices, equivalent to --gpus on a raw docker run. One 'docker compose up' now reproduces the whole local stack.

The fifth block is a .dockerignore, which is easy to skip and quietly important. It excludes the git directory, Python bytecode caches, virtual environments, the data directory, notebook checkpoints, experiment-tracker output, and model checkpoints from the build context.

The reason this matters is that everything in the build context is sent to the Docker daemon before the build even starts, and anything COPYed in becomes part of the image. Without a .dockerignore, a multi-gigabyte data directory or a local .venv gets shipped into the context, making builds slow and images bloated — and secrets or credentials can leak in by accident. The comment underscores the rule: keep big artifacts and secrets out of the build context entirely.

The pipeline diagram shows the path an image takes from source to production: build and tag it, test it by running it in CI, push it to a registry, then deploy by pulling and running it. Each stage operates on the same versioned artifact.

The shape captures the discipline that separates a demo from a deployment. Testing happens on the exact image you'll ship, not a re-derived environment. The registry is the single source of truth that every environment pulls from. And deployment is just 'pull this tag and run it,' which is what makes rollbacks as simple as deploying a previous tag. This is the lifecycle the next code slide makes concrete.

This block gives the concrete build, run, and push commands. It builds the serving Dockerfile and tags the image with an explicit version (1.0). It runs the image locally with the port published, then exercises the endpoint with a curl POST sending example features and reading back a prediction. Finally it pushes the tagged image to a registry.

The details reflect good practice. The tag is a real version, not :latest, so the artifact is identifiable. The local run-and-curl is a smoke test you should always do before pushing — it catches a broken image in seconds rather than after deployment. And the push makes the image available for any environment to pull, completing the build-to-deploy loop the diagram described.

The tips slide gives the production rules that tie the post together. Keep training and serving as separate images so neither carries the other's baggage. In production, pin the base image by digest, not just a tag, so it can't change under you. Mount data and models as volumes rather than baking them into the image. Use .dockerignore to keep the build context small. And tag images with a meaningful model or build version, never relying on :latest.

These five rules are what make the code in this post safe to run at scale rather than just in a demo, and several of them are the positive form of the mistakes the final post catalogs.

The closing mistake targets a costly and security-relevant habit: baking data and secrets into the image. COPYing a multi-gigabyte dataset bloats the image and forces a rebuild every time the data changes. Worse, an API key or credential copied in during the build persists in the image layers — it remains recoverable from the layer history even if a later instruction appears to delete it.

The fix is a clean separation of concerns. Data belongs in a mounted volume so it lives outside the image and changes independently. Secrets belong in runtime environment variables or dedicated secret mounts, never in a build layer. The image should contain code and the trained model — nothing private, nothing huge. This keeps images small, rebuildable, and safe to push to a registry.

The CTA hands off to the final post, which catalogs the common mistakes that quietly bloat, break, and slow ML containers — oversized bases, cache-busting order, running as root, unpinned tags, missing GPU runtime, and ignored image security. The teaser frames it as the failure manual that completes the toolkit.