Containerizing a Machine Learning Model Step by Step
Most container tutorials stop when the container starts. This walkthrough containerizes a real scikit-learn model step by step, then measures what matters: build and rebuild times, image size, the user it runs as, time to healthy, memory, and whether the container's predictions match training exactly.
GloryTecks Team
Training & Placement Faculty
Last updated: 10 October 2026
Containerizing a machine learning model is often described in three commands: write a Dockerfile, build, run. The container starts, the tutorial ends. But "it starts" says little about whether the container is fit to deploy: whether it predicts exactly like your notebook, how big it is, what user it runs as, how long it takes to become ready and how quickly you can rebuild it after a change.
This walkthrough containerizes a real model step by step, then measures each of those properties. The principles behind the Dockerfile are covered in the Docker for machine learning guide; this is the worked example, with real numbers from one run.
Step 1: train, and record golden predictions
The model is a random forest that predicts whether a white wine scores 7 or more for quality, from the UCI Wine Quality data (UCI: Wine Quality). Besides the model file, the training script saves 25 held-out rows and the model's predictions for them. Those golden predictions are how we will prove the container behaves exactly like the training environment.
import json, joblib, pandas as pd
from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import train_test_split
# UCI Wine Quality, white wines (CC BY 4.0): is the wine "good" (quality 7 or more)?
wine = pd.read_csv("data/winequality-white.csv", sep=";")
wine.columns = [c.replace(" ", "_") for c in wine.columns]
features = [c for c in wine.columns if c != "quality"]
X_train, X_test, y_train, y_test = train_test_split(
wine[features], (wine.quality >= 7).astype(int), test_size=0.2, random_state=0, stratify=wine.quality >= 7)
model = RandomForestClassifier(n_estimators=200, random_state=0, n_jobs=1).fit(X_train, y_train)
joblib.dump(model, "model.joblib")
print(f"held-out accuracy {model.score(X_test, y_test):.3f} on {len(X_test)} wines")
# Golden predictions: inputs and outputs recorded now, checked against the container later
golden = X_test.head(25)
json.dump({"rows": golden.to_dict(orient="records"),
"p_good": model.predict_proba(golden)[:, 1].tolist()}, open("golden.json", "w"))
print(f"saved model.joblib and {len(golden)} golden predictions")held-out accuracy 0.890 on 980 wines
saved model.joblib and 25 golden predictionsStep 2: pin the training environment
scikit-learn does not support loading a model with a different version from the one that trained it, and the same can apply to numpy and scipy. Its documentation recommends the same dependencies and versions in training and production (scikit-learn: model persistence). So requirements.txt pins the exact versions from the training environment, not ranges:
scikit-learn==1.9.1
numpy==2.4.6
scipy==1.17.1
joblib==1.6.0
pandas==3.0.6
fastapi==0.142.2
pydantic==2.13.5
uvicorn==0.54.0Step 3: the service
The app loads the model once at start-up, exposes a health endpoint, and orders the request's fields by the model's own stored feature names, so the column order cannot drift. For the tests a prediction API needs, see building a prediction API.
import joblib, pandas as pd
from fastapi import FastAPI
from pydantic import BaseModel, ConfigDict
model = joblib.load("model.joblib") # loaded once, when the server starts
FEATURES = list(model.feature_names_in_) # the training column order, stored in the model
class Wine(BaseModel):
model_config = ConfigDict(extra="forbid")
fixed_acidity: float
volatile_acidity: float
citric_acid: float
residual_sugar: float
chlorides: float
free_sulfur_dioxide: float
total_sulfur_dioxide: float
density: float
pH: float
sulphates: float
alcohol: float
app = FastAPI()
@app.get("/health")
def health():
return {"status": "ok"}
@app.post("/predict")
def predict(wine: Wine):
row = pd.DataFrame([wine.model_dump()])[FEATURES]
return {"p_good": float(model.predict_proba(row)[0, 1])}Step 4: the Dockerfile and .dockerignore
The Dockerfile follows Docker's advice to put expensive steps first: dependency files are copied and installed before the code, so a code change does not reinstall the libraries (Docker: optimize cache usage). It creates an ordinary user, and declares a health check that calls the service.
FROM python:3.12-slim
ENV PYTHONDONTWRITEBYTECODE=1 PYTHONUNBUFFERED=1
WORKDIR /app
# Dependencies first: this layer is reused until requirements.txt changes
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
# Then the model and code, which change more often
COPY model.joblib app.py ./
# Run as an ordinary user, not root
RUN useradd --create-home appuser
USER appuser
EXPOSE 8000
HEALTHCHECK --interval=5s --timeout=3s --start-period=5s --retries=3 \
CMD python -c "import urllib.request; urllib.request.urlopen('http://127.0.0.1:8000/health')"
CMD ["uvicorn", "app:app", "--host", "0.0.0.0", "--port", "8000"]The .dockerignore file keeps the training data, scripts and golden file out of the build context and the image:
data/
golden.json
train.py
check.py
__pycache__/
*.pyc
.venv/Step 5: build, inspect, run and compare
The check script drives Docker: it builds from scratch and after a one-line code change, inspects the image, looks inside it, starts a container with a 512 MB memory limit, waits for it to report healthy, and sends the golden rows to it.
import json, subprocess, time, urllib.request
IMAGE, PORT = "wine-api:1.0", 18000
def sh(*args):
return subprocess.run(args, capture_output=True, text=True, encoding="utf-8", check=True).stdout.strip()
def timed_build(*extra):
start = time.perf_counter(); sh("docker", "build", "-q", *extra, "-t", IMAGE, "."); return time.perf_counter() - start
print(f"C1 build from scratch: {timed_build('--no-cache'):.0f} seconds")
original = open("app.py").read()
open("app.py", "a").write("# a one-line code change\n")
try:
print(f" rebuild after a code change: {timed_build():.0f} seconds (dependency layer reused)")
finally:
open("app.py", "w").write(original)
timed_build() # back to the original code
compressed_mb = int(sh("docker", "image", "inspect", IMAGE, "--format", "{{.Size}}")) / 1e6
on_disk = sh("docker", "image", "ls", IMAGE, "--format", "{{.Size}}")
print(f"\nC2 image: {on_disk} on disk unpacked, {compressed_mb:.0f} MB compressed to push or pull")
print(" layers added on top of python:3.12-slim:")
layers = sh("docker", "history", IMAGE, "--no-trunc", "--format", "{{.Size}}\t{{.CreatedBy}}").splitlines()
for line in [l for l in layers if not l.startswith("0B")][:3]:
size, created = line.split("\t", 1)
print(f" {size:>8} {created.replace('/bin/sh -c ', '').replace(' # buildkit', '')[:60]}")
print("\nC3 inside the image")
print(" user:", sh("docker", "run", "--rm", "--entrypoint", "whoami", IMAGE))
print(" versions:", sh("docker", "run", "--rm", "--entrypoint", "python", IMAGE, "-c",
"import sys, sklearn; print(sys.version.split()[0], 'scikit-learn', sklearn.__version__)"))
print(" training data in image:", sh("docker", "run", "--rm", "--entrypoint", "ls", IMAGE, "/app").split())
container = sh("docker", "run", "-d", "--memory", "512m", "-p", f"{PORT}:8000", IMAGE)
try:
start = time.perf_counter()
while sh("docker", "inspect", "--format", "{{.State.Health.Status}}", container) != "healthy":
time.sleep(0.5)
print(f"\nC4 healthy {time.perf_counter() - start:.0f} seconds after start")
golden = json.load(open("golden.json"))
served = []
for row in golden["rows"]:
request = urllib.request.Request(f"http://127.0.0.1:{PORT}/predict", data=json.dumps(row).encode(),
headers={"content-type": "application/json"})
served.append(json.load(urllib.request.urlopen(request))["p_good"])
gap = max(abs(a - b) for a, b in zip(served, golden["p_good"]))
print(f" {len(served)} golden predictions from the container, largest difference from training: {gap}")
print(" memory in use:", sh("docker", "stats", "--no-stream", "--format", "{{.MemUsage}}", container))
finally:
sh("docker", "rm", "-f", container)C1 build from scratch: 51 seconds
rebuild after a code change: 4 seconds (dependency layer reused)
C2 image: 673MB on disk unpacked, 153 MB compressed to push or pull
layers added on top of python:3.12-slim:
69.6kB RUN useradd --create-home appuser
13.6MB COPY model.joblib app.py ./
373MB RUN pip install --no-cache-dir -r requirements.txt
C3 inside the image
user: appuser
versions: 3.12.14 scikit-learn 1.9.1
training data in image: ['app.py', 'model.joblib', 'requirements.txt']
C4 healthy 6 seconds after start
25 golden predictions from the container, largest difference from training: 0.0
memory in use: 148.6MiB / 512MiBWhat the measurements say
Rebuilds are fast because of layer order. Building from scratch took about a minute, almost all of it installing packages. After a code change, the rebuild took a few seconds, because the dependency layer was reused. If COPY . . came before pip install, every code change would pay the full minute.
The libraries are the image. The image takes 673 MB on disk unpacked and 153 MB compressed, which is what a registry stores and a server downloads. Of the layers added to the base image, the package install was 373 MB, the model and code 13.6 MB and the user setup 70 kB. To shrink the image, look at the dependencies first: they are where the size is.
It runs as an ordinary user. whoami printed appuser. Docker's USER instruction sets the user for later RUN instructions and for the command the container runs (Docker: Dockerfile reference). If someone finds a flaw in your service, they land in an unprivileged account, not root.
The versions are what we pinned, except Python. scikit-learn is 1.9.1 as in training, but the base image has Python 3.12 while training used 3.11. The golden check is what makes this acceptable: if a version difference mattered, it would show up there.
No data leaked into the image. The app directory holds only the app, the model and the requirements file. The training data stayed out thanks to .dockerignore.
Healthy in seconds, and identical predictions. Docker reported the container healthy a few seconds after start. Health status starts as starting and becomes healthy when a check passes (Docker: Dockerfile reference); orchestrators use it to decide when to send traffic. All 25 golden predictions from the container matched training with a largest difference of 0.0. That one line is the real proof the container is the model you trained.
Memory use was about 150 MB, well under the 512 MB limit. Measure it under load before setting limits in production.
What to automate
Run the same checks in CI on every image build: build, inspect the user, run the golden comparison against the container, and fail the build on any difference. Store the golden file with the model version, as described in model packaging. Load model files only from sources you trust, because joblib and pickle files can run code when loaded. For running many replicas of the container, see Kubernetes for ML.
How do I containerize a machine learning model?
How do I know the container predicts the same as my notebook?
Why is my machine learning Docker image so large?
Should a model container run as root?
Packaging, serving and deploying models are part of the MLOps course.
Sources
- Model persistence (scikit-learn documentation) — scikit-learn
- Optimize cache usage in builds (Docker documentation) — Docker
- Dockerfile reference (Docker documentation) — Docker
- Wine Quality (UCI Machine Learning Repository) — UCI Machine Learning Repository
GloryTecks Team
Training & Placement Faculty
The GloryTecks faculty team mentors learners across Data Science, AI and Data Engineering, with a focus on real-time projects and placement readiness in Hyderabad.
Want to master MLOps?
Join GloryTecks’ expert-led MLOps training in Hyderabad with real-time projects and 100% placement support.
Model Explainability in Production with SHAP
NextShadow Deployment and Canary Releases for ML
Related articles
Docker for Machine Learning: A Practical Guide
How to containerise a machine-learning project so it runs the same everywhere: pinning the exact library versions a saved model depends on, a Dockerfile ordered for fast rebuilds, keeping data out of images, multi-stage builds, non-root users and GPU access.
CI/CD for Machine Learning: Pipelines That Work
CI/CD for machine learning has to test three things that change: code, data and models. This guide runs five commits through a staged pipeline on real wine data. It stops a broken delivery, a feature bug and a weaker model at different stages, and shows duplicates had inflated the AUC from 0.822 to 0.896.
What Is MLOps? DevOps for Machine Learning Explained
MLOps is the practice of building machine learning systems that keep working after launch. This guide defines it, explains why ML needs more than DevOps, and shows the problem it solves on real data: a model built in 2011 that was 38% wrong through 2012, and how much the way you retrain it matters.