Docker for ML

This cover sets the tone for the failure manual: broken ML containers almost never mean Docker itself failed — they mean a handful of avoidable mistakes bloated the image, busted the cache, or broke the GPU. That reframing is encouraging: the fixes are all within reach and mostly cost a line or two.

Post five completes the series by cataloging six mistakes that most often degrade ML containers in practice: bloated base images, cache-busting layer order, running as root, unpinned :latest tags, a missing GPU runtime, and ignored image size and security. Each is common, each is fixable, and each quietly costs build time, image size, or a production incident.

The first mistake is starting from a bloated base. Building on the full python:3.11 — or hand-assembling from a generic ubuntu — ships compilers, documentation, and package caches the running service will never touch. For a serving image this is pure waste: the result is several gigabytes when it could be hundreds of megabytes.

The fix is to choose the right base for the job: a slim or distroless image for CPU serving, or an official CUDA runtime image for GPU work, combined with multi-stage builds so build-time tooling never reaches the final image. Smaller images pull faster, start faster, cost less to store and transfer, and present a smaller attack surface — every dimension improves at once.

The comparison puts bloated and lean images side by side. The bloated column: the full python image, build tools left in the final image, a dataset baked in, and multi-gigabyte pulls on every deploy. The lean column: a slim or distroless base, a multi-stage build that keeps only the runtime, data mounted as a volume, and images measured in hundreds of megabytes.

Keeping this contrast in mind is the fastest way to audit your own Dockerfile. Each lean-column item is a concrete, independent fix, and you rarely need all of them at once — picking off even one or two typically cuts image size dramatically and speeds up your whole build-and-deploy loop.

The second mistake is cache-busting layer order, the single most common reason ML builds are painfully slow. Putting COPY . . before the dependency install means every code edit — even a one-character change — invalidates the layer holding your code, which forces the heavy install layer beneath it to rebuild, reinstalling the entire dependency stack including torch and CUDA.

The fix is the ordering convention from the mechanics post: copy requirements.txt and install first, then copy the code. More generally, order instructions from least-frequently-changing to most-frequently-changing. This costs nothing and is the highest-leverage Dockerfile change you can make, routinely turning multi-minute rebuilds back into a few seconds.

This code slide shows the wrong and right ordering directly. The wrong version copies everything and then installs, so any change to any file busts the install layer. The right version copies just requirements.txt, installs into a stable cached layer, and only then copies the rest of the code.

The contrast is intentionally minimal because the fix is so small relative to its impact — three lines reordered. With the right ordering, editing your training script invalidates only the final code layer and the expensive install is reused from cache. Seeing the two side by side makes it obvious why the convention exists and why it's worth making automatic.

The third mistake is running the container as root, which is the default if you do nothing. A process running as root inside the container has root in that container, and combined with any misconfiguration or a container-escape vulnerability, that's a wider path toward compromising the host. For an internet-facing ML service this is needless risk.

The fix is cheap: create an unprivileged user in the Dockerfile and switch to it with USER before the run command. The application almost never needs root to serve predictions, so dropping privileges removes an entire class of escalation for the cost of two lines. It's one of the easiest security wins available and should be a default habit for any image that runs a service.

This code slide implements the non-root fix. After installing dependencies and copying code as root (which is fine at build time), it creates an unprivileged user with useradd --create-home and switches to it with USER appuser before the CMD. From that point the service runs without root privileges.

The ordering is deliberate: privileged operations like installing packages happen before the USER switch, and only the long-running service process runs unprivileged. If the application needs to write files at runtime, you ensure the relevant directories are owned by or writable to appuser. This pattern adds negligible size and turns a root-by-default service into a properly de-privileged one.

The fourth mistake is relying on :latest. FROM python:latest means the base can silently change between builds, and deploying myorg/model:latest means the running image can change out from under you — a base bump might pull a new CUDA version that breaks compatibility or a new library release that changes behavior overnight.

The fix is to pin. Pin base images by an explicit version, and in production ideally by digest, which nails the exact bytes regardless of how a tag is later moved. Tag your own images with the model version or git commit so every deployed artifact is identifiable and reproducible. Reproducibility — the entire reason to use Docker for ML — dies the moment 'latest' enters a Dockerfile.

The fifth mistake is forgetting the GPU runtime. Build a beautiful CUDA-based training image, then run it with a plain docker run and the container sees no GPU at all. The especially nasty part is that PyTorch and most frameworks silently fall back to CPU rather than erroring, so training simply runs many times slower with no obvious cause.

The fix has two parts: ensure the host has the NVIDIA Container Toolkit installed, and pass --gpus all (or a specific GPU set) on docker run. Critically, verify before committing to a long job. Running nvidia-smi or checking torch.cuda.is_available() inside the container takes seconds and catches the silent-fallback trap before it wastes hours of compute.

This code slide contrasts the wrong and right way to launch a GPU job. The wrong command omits --gpus, so the container has no GPU access and training quietly falls back to CPU. The right commands request GPUs with --gpus all and then verify access two ways: by running nvidia-smi to list the visible GPUs, and by printing torch.cuda.is_available() to confirm the framework actually sees them.

The lesson generalizes into a habit: never assume the GPU is wired up — prove it with a quick check before a long run. The verification commands cost seconds and turn the most common and most expensive GPU-in-Docker failure into an immediate, obvious signal rather than a slow, mysterious one.

The tips slide condenses the whole failure manual into a five-line checklist: use a slim base with a multi-stage build, copy dependencies before code, run as a non-root user, pin bases and tag images by version, and request --gpus while verifying with nvidia-smi.

These five lines are the practical residue of the entire series. A reader who internalizes them will avoid the great majority of ML container failures and will build images that are small, fast, reproducible, secure, and actually use the GPU they were built for.

The sixth mistake is ignoring image size and security as ongoing concerns. Images that are never rebuilt accumulate known vulnerabilities in their old system packages, and oversized images slow every pull, every deploy, and every autoscaling event. An ML image left unattended drifts from 'shipped once' to 'liability.'

The fix is to treat the image as production software with a supply chain. Scan images in CI with a tool like Trivy or Docker Scout, rebuild on a regular cadence so base-image security patches actually land, and keep size and layer count down. Reasoning, accuracy, and uptime all depend on the substrate the model runs on; neglecting that substrate is how a working model becomes an incident.

The CTA closes the series and hands off to the next day's topic, framing the five posts as a contribution to the reader's growing MLOps toolkit. It signals continuity — the series keeps building practical, in-depth technique — without naming the next subject, since each day brings something new.