Docker for Machine Learning: A Practical Guide

"It works on my machine" is a bigger problem in machine learning than in most software, because an ML project depends on more moving parts: the Python version, dozens of numerical libraries, sometimes system libraries and GPU drivers, and a saved model file that only loads under the library versions it was saved with. A Docker image packages all of that into one artefact that runs the same on a laptop, a CI runner and a server. This guide covers how to build that image well.

Why ML projects need exact versions

The strongest reason to containerise a model is not deployment convenience. It is the saved model file. scikit-learn states that there is no supported way to load a model trained with a different version of scikit-learn, and possibly of NumPy and SciPy too; loading one raises an InconsistentVersionWarning (scikit-learn: model persistence). A model that loads with a warning may still predict differently. So the image must reproduce the training environment exactly, down to the library versions, and pip install scikit-learn without a version does not do that.

# requirements.txt: exact versions, matching the environment the model was trained in
scikit-learn==1.5.2
pandas==2.2.3
joblib==1.4.2

Generate the list from the training environment (for example pip freeze in that environment, trimmed to what the code imports), and store it next to the model artefact it belongs to.

A Dockerfile ordered for fast rebuilds

Docker builds an image as a stack of layers and reuses cached layers when nothing they depend on has changed. When a layer is invalidated, every layer after it is rebuilt too (Docker: build cache). Installing ML libraries is the slowest step, so it goes before the files that change most often, your code and your model:

# syntax=docker/dockerfile:1
FROM python:3.12-slim

ENV PYTHONDONTWRITEBYTECODE=1 \
    PYTHONUNBUFFERED=1 \
    PIP_NO_CACHE_DIR=1

WORKDIR /app

# 1. Dependencies: rebuilt only when requirements.txt changes
COPY requirements.txt .
RUN pip install -r requirements.txt

# 2. Code, then the model artefact: these change most often
COPY src/ ./src/
COPY models/churn-2026-09.joblib ./models/model.joblib

# 3. Run as an unprivileged user that owns the output directory
RUN useradd --create-home --no-log-init appuser \
    && mkdir -p /app/output && chown appuser /app/output
USER appuser

CMD ["python", "-m", "src.predict_batch", "--model", "models/model.joblib"]

Edit a Python file and rebuild: the dependency layer comes from cache and only the last few layers run. Put COPY . . at the top instead, and every code change reinstalls every library.

• Pin the base image. python:3.12-slim is a moving tag. For fully reproducible builds, Docker recommends pinning to a digest (python:3.12-slim@sha256:…), which always resolves to the same image, even if the publisher replaces the tag (Docker: building best practices).
• Run as a non-root user. Docker’s guidance is to switch with USER when a service does not need privileges. A model server never does. Create every directory the process writes to, and give it to that user, before the USER line. The batch run below mounts a volume at /app/output; without the mkdir and chown, the unprivileged process cannot write there and the run fails with PermissionError.
• One job per image or container. Training and serving usually need different dependencies and resources. Separate images keep the serving image small.

Keep data out of the image

Everything in the directory you build from, the build context, is available to the build unless .dockerignore excludes it (Docker: build context). ML repositories accumulate large things that do not belong in an image: raw data, notebook checkpoints, experiment-tracking folders and virtual environments. A .dockerignore keeps them out:

# .dockerignore
.git
.venv/
__pycache__/
.ipynb_checkpoints/
*.ipynb
data/
mlruns/
*.csv
*.parquet

Data a container reads or writes at run time belongs in a volume or a bind mount, not in the image. A container’s own writable layer is destroyed with the container; a volume’s contents live outside that lifecycle (Docker: volumes). For a batch scoring run:

docker build -t churn-scorer:2026-09 .

docker run --rm \
  --mount type=bind,source="$(pwd)/data",target=/app/data,readonly \
  --mount type=volume,source=churn-output,target=/app/output \
  churn-scorer:2026-09

Small model files can go in the image, as above. The image tag then identifies the exact model, and rolling back means running the previous tag. Large models (several gigabytes) are usually downloaded from a model registry or object storage when the container starts, and the registry version is recorded instead. Model versioning with MLflow and DVC covers the registry side.

Multi-stage builds for compiled dependencies

Some Python packages need a compiler to install from source. Installing build-essential into the final image adds hundreds of megabytes you never use at run time. A multi-stage build compiles in one stage and copies only the result into a clean final stage; everything else is left behind (Docker: multi-stage builds):

# syntax=docker/dockerfile:1
FROM python:3.12-slim AS build
RUN apt-get update && apt-get install -y --no-install-recommends build-essential \
    && rm -rf /var/lib/apt/lists/*
COPY requirements.txt .
RUN pip wheel --wheel-dir /wheels -r requirements.txt

FROM python:3.12-slim
# Mount the wheels from the build stage for this one step only,
# so they never become a layer of the final image
RUN --mount=type=bind,from=build,source=/wheels,target=/wheels \
    pip install --no-cache-dir /wheels/*
WORKDIR /app
COPY src/ ./src/

The final stage borrows the wheels from the build stage only while pip install runs, through a bind mount (Dockerfile reference: RUN --mount). The obvious alternative, COPY --from=build /wheels /wheels followed by rm -rf /wheels, does not shrink the image: the files are deleted in a later layer but stay in the layer that copied them. Note also apt-get update and apt-get install in the same RUN. Docker’s best-practice guide explains that splitting them lets the build reuse a stale cached package index.

GPUs in containers

Containers do not see the host’s GPUs by default. On a Linux host with NVIDIA GPUs, install the NVIDIA Container Toolkit, then pass --gpus when you start the container (Docker: GPU access):

# Check that the container can see every GPU
docker run -it --rm --gpus all ubuntu nvidia-smi

# Give a training container only the first GPU
docker run --rm --gpus device=0 my-trainer:latest

GPU images are large. Use a GPU base image only for the stages that need it, typically training, and keep CPU-only inference on a slim image.

A checklist before you ship an image

1. Library versions pinned to the environment the model was trained in.
2. Base image pinned, by digest if builds must be byte-for-byte reproducible.
3. Dependencies installed before code is copied.
4. .dockerignore excludes data, notebooks, virtual environments and experiment folders.
5. Runs as a non-root user.
6. Input and output data mounted, not baked in.
7. Image tagged with something meaningful (model version or date), never only latest.

Where to go next

Once you have a good image, Kubernetes for ML covers running it at scale, and CI/CD for machine learning covers building and testing it automatically. Containerisation, Kubernetes and CI/CD for ML are modules of our MLOps course.

Sources

• Building best practices — Docker documentation
• Docker build cache — Docker documentation
• Multi-stage builds — Docker documentation
• Dockerfile reference: RUN --mount — Docker documentation
• Build context — Docker documentation
• Volumes — Docker documentation
• GPU access — Docker documentation
• Model persistence — scikit-learn user guide