Docker for ML

This cover sets the agenda for the mechanics post and reframes the image as something concrete rather than opaque: a stack of read-only layers plus a recipe for building them. The promise is practical — once you understand the layers, fast and small builds come almost for free, because you'll order your Dockerfile to cooperate with the cache instead of fighting it.

Post three is the engine room. We've established what Docker is and why ML needs it; now we explain the actual machinery — layers, the build cache, instruction ordering, the build-to-container lifecycle, and GPU passthrough — so that working with Docker stops being trial-and-error.

The foundation is that every instruction is a layer. FROM creates the base layer; each RUN, COPY, and ADD stacks a new read-only layer on top of the previous one. A union filesystem merges all those layers into a single coherent view that processes inside the container see as one normal filesystem.

When you actually run a container, Docker adds one more layer on top — a thin writable layer unique to that container. Everything below it remains immutable and is shared across every container started from the image. This is why launching ten containers from one image doesn't copy the image ten times: they all share the read-only layers and differ only in their thin writable tops.

The flow diagram shows how the build cache decides what to do for each instruction. For a given instruction, Docker checks whether anything it depends on has changed since the cached layer was built. If nothing changed, it's a cache hit and the layer is reused instantly. If something changed, that layer is rebuilt.

This is the mechanism that makes iterative development fast — most rebuilds touch only your code, so most layers are cache hits. But the same mechanism is a trap if you order instructions badly, because of the positional rule the next slide explains. Understanding the cache is the difference between five-second and five-minute rebuilds.

This slide states the rule that trips up almost everyone: cache invalidation is positional. When a layer is rebuilt, every layer after it is rebuilt too, regardless of whether those later steps themselves changed. The cache can only be reused as an unbroken prefix from the bottom.

The practical consequence is concrete and painful. If you COPY your whole project before the pip install step, then any edit to any source file changes the COPY layer, which invalidates the install layer below it, which forces a full reinstall of every dependency. Ordering instructions from least-frequently-changing to most-frequently-changing keeps your slow, expensive steps cached across the edits you actually make.

This code slide shows the correct ordering and annotates the reasoning inline. Dependencies change rarely, so requirements.txt is copied and installed first, landing in a stable layer. Code changes constantly, so COPY . . comes last, in a cheap top layer.

With this ordering, editing train.py invalidates only the final COPY layer; the expensive pip install layer below it is a cache hit and is reused untouched. The --no-cache-dir flag is a small bonus that keeps pip's own download cache out of the image, shaving size. This single ordering convention is the highest-leverage habit in Dockerfile authoring.

The pipeline diagram traces the full lifecycle from recipe to running process. docker build executes the Dockerfile and produces an image made of read-only layers. docker run takes that image, adds a thin writable layer, and starts it as an isolated process — a container.

The diagram makes the immutability boundary visible: the image is fixed and reusable, while the writable layer is per-container and disposable. This is why you can run many containers from one image and why anything you write inside a container vanishes when it's removed unless it's on a mounted volume. Seeing build and run as distinct phases clears up a lot of beginner confusion about where state lives.

This slide explains GPU passthrough, which feels like magic until you see the parts. The host machine has the NVIDIA kernel driver installed. The NVIDIA Container Toolkit hooks into container startup and injects the driver's userspace libraries and the GPU device files into the container. The image itself ships only the CUDA runtime, not the driver.

When you start a container with --gpus all, the toolkit wires up that injection, so CUDA calls made inside the container reach the real hardware through the host's driver. The clean separation — driver on the host, runtime in the image — is exactly why an image built on a CUDA base is portable across GPU hosts as long as each host has a compatible driver.

This code slide turns the passthrough explanation into commands. The comment states the host prerequisites: the NVIDIA driver and the NVIDIA Container Toolkit. The docker run uses --gpus all to expose the GPUs, mounts a local data directory as a volume so the dataset lives outside the image, and runs training.

The second command is the verification habit every GPU user should adopt: run nvidia-smi inside the container before a long job. If it lists your GPUs, passthrough is working; if it errors or shows nothing, you'd otherwise discover the problem hours into a silent CPU-bound run. Verifying first is cheap insurance against the most common GPU-in-Docker failure.

This slide introduces multi-stage builds, the standard technique for shrinking ML images. You define a heavy 'builder' stage that contains compilers and build-time tooling to produce artifacts — compiled wheels, for example. Then a slim final stage copies only those finished artifacts and discards everything the builder needed.

The payoff is that the image you actually ship carries no compilers, no build caches, and no intermediate junk — just the runtime and your model. For ML images, which often start gigabytes large because of build dependencies, this routinely cuts the final size to a fraction. Smaller images pull faster, deploy faster, and expose a smaller attack surface, so multi-stage is close to free value.

This code slide implements a concrete two-stage build. Stage one uses the full python:3.11 image to build wheels for all requirements into a /wheels directory with pip wheel. Stage two starts from the slim base, copies only the prebuilt wheels from the builder, installs them, and adds the code.

The critical line is COPY --from=builder, which reaches into the previous stage and pulls only what's named — the wheels — leaving the builder's bulk behind. The result is a lean runtime image that never contained the compilers or download caches used to produce those wheels. This pattern generalizes to any case where building artifacts needs heavy tooling that serving doesn't.

The tips slide condenses the under-the-hood mechanics into a checklist. Layers are read-only and shared across images and containers. The cache rebuilds from the first changed layer downward, so positional ordering matters. Copy dependencies before code to protect the expensive install layer. GPU access needs both the host driver and the container toolkit plus --gpus. And multi-stage builds keep final images small.

These five points are the operational residue of the mechanics: enough to diagnose a slow build, reason about image size, and set up GPU access correctly.

The closing mistake names the most common build-performance bug: putting COPY . . near the top, before the pip install. Because cache invalidation is positional, that single misordering means every one-character code change busts the dependency layer and reinstalls the entire heavy stack — torch, CUDA, everything — turning a five-second rebuild into a five-minute one.

The fix is the convention from earlier: copy requirements first, install, then copy code. It's worth emphasizing here because ordering is the cheapest optimization available — it costs nothing, requires no new tooling, and routinely delivers the biggest single improvement to iteration speed. Most painfully slow ML builds are slow for exactly this reason.

The CTA hands off to post four, the code-heavy one: a full training Dockerfile, a GPU image on the CUDA base, a slim FastAPI serving image, and a docker-compose file wiring them together. The teaser promises concrete, runnable patterns that turn the mechanics just explained into working infrastructure.