Docker and Kubernetes Interview Questions for ML

Docker and Kubernetes interview questions for ML engineers are less about commands than about the mistakes that make images slow to build, oversized, insecure, or killed in production. The four Docker answers below come from real builds and runs of a small scikit-learn service on a local Docker engine. The Kubernetes questions that follow build on them, and their real-cluster behaviour is shown in Kubernetes for ML.

import json
import os
import re
import shutil
import subprocess
import time

run = lambda *cmd, check=True: subprocess.run(cmd, capture_output=True, text=True, encoding="utf-8", errors="replace", check=check)

app = "import sklearn\nprint('model service', sklearn.__version__)\n"
slow = "FROM python:3.12-slim\nWORKDIR /app\nCOPY . .\nRUN pip install --no-cache-dir -r requirements.txt\nCMD [\"python\", \"app.py\"]\n"
fast = ("FROM python:3.12-slim\nWORKDIR /app\nCOPY requirements.txt .\nRUN pip install --no-cache-dir -r requirements.txt\n"
        "COPY app.py .\nRUN useradd --uid 1000 app\nUSER app\nCMD [\"python\", \"app.py\"]\n")


def build(folder, tag):
    out = run("docker", "build", "--progress=plain", "-t", tag, folder).stderr
    context = re.findall(r"load build context\n#\d+ transferring context: ([\d.]+[kMG]?B)", out)
    return out, context[-1] if context else "?"


def pip_cached(out):
    steps = {m.group(1): m.group(2) for m in re.finditer(r"(#\d+) \[\d+/\d+\] (RUN pip install[^\n]*)", out)}
    return all(f"{step} CACHED" in out for step in steps)


print("DK1 layer caching: change one line of app.py and rebuild")
for name, dockerfile in (("COPY . before pip install", slow), ("requirements first, code last", fast)):
    folder = f"img_{len(dockerfile)}"
    shutil.rmtree(folder, ignore_errors=True)
    os.makedirs(folder)
    open(f"{folder}/requirements.txt", "w").write("scikit-learn==1.9.1\n")
    open(f"{folder}/app.py", "w").write(app)
    open(f"{folder}/Dockerfile", "w").write(dockerfile)
    build(folder, f"dk-demo:{len(dockerfile)}")
    open(f"{folder}/app.py", "a").write(f"# a one-line change {time.time_ns()}\n")   # unique, so no old cache matches
    out, _ = build(folder, f"dk-demo:{len(dockerfile)}")
    print(f"   {name:<31} pip install step reused from cache: {pip_cached(out)}")

print("\nDK2 a 127 MB data file in the build folder, with COPY . .")
folder = f"img_{len(slow)}"
shutil.copy("household_power_consumption.txt", folder)
app_size = lambda tag: run("docker", "run", "--rm", tag, "du", "-sm", "/app").stdout.split()[0]   # MB inside the image
_, without = build(folder, "dk-demo:ctx")
before = app_size("dk-demo:ctx")
open(f"{folder}/.dockerignore", "w").write("*.txt\n!requirements.txt\n")
_, with_ignore = build(folder, "dk-demo:ctx")
print(f"   /app in the image: {before} MB without .dockerignore -> {app_size('dk-demo:ctx')} MB with it")
print(f"   build context sent: {without} -> {with_ignore} (transfer size varies)")

print("\nDK3 which user does the container run as?")
print(f"   COPY-first image: uid {run('docker', 'run', '--rm', f'dk-demo:{len(slow)}', 'id', '-u').stdout.strip()}"
      f" | image with USER app: uid {run('docker', 'run', '--rm', f'dk-demo:{len(fast)}', 'id', '-u').stdout.strip()}")

print("\nDK4 a 64 MB memory limit and a process that wants 300 MB")
run("docker", "rm", "-f", "hungry", check=False)
code = run("docker", "run", "--name", "hungry", "--memory", "64m", "--memory-swap", "64m", "python:3.12-slim",
           "python", "-c", "x = bytearray(300 * 1024 * 1024)", check=False).returncode
state = json.loads(run("docker", "inspect", "hungry").stdout)[0]["State"]
print(f"   exit code {code} | OOMKilled {state['OOMKilled']}")
run("docker", "rm", "-f", "hungry", check=False)

DK1 layer caching: change one line of app.py and rebuild
   COPY . before pip install       pip install step reused from cache: False
   requirements first, code last   pip install step reused from cache: True

DK2 a 127 MB data file in the build folder, with COPY . .
   /app in the image: 127 MB without .dockerignore -> 1 MB with it
   build context sent: 132.99MB -> 155B (transfer size varies)

DK3 which user does the container run as?
   COPY-first image: uid 0 | image with USER app: uid 1000

DK4 a 64 MB memory limit and a process that wants 300 MB
   exit code 137 | OOMKilled True

DK1. Why does the order of lines in a Dockerfile matter?

Because of layer caching. Each instruction produces a layer, and when one changes, every layer after it is rebuilt (Docker: build cache). With COPY . . before pip install, a one-line change to app.py invalidated the copy layer, and the dependency installation ran again. Copying requirements.txt first, installing, and copying the code last kept the installation cached after the same change. For ML images, where installing libraries can take minutes, this is the difference between a fast and a painful build loop.

DK2. What is .dockerignore for?

Keeping files out of the build context and, with COPY . ., out of the image. A 127 MB data file sitting in the build folder was sent to the builder and copied into the image: /app held 127 MB. Excluding it in .dockerignore, which removes matching files from the context before it is sent (Docker: build context), left 1 MB. Datasets, model checkpoints, virtual environments, .git and secrets such as .env files should not be in an image. Data belongs in storage or a mounted volume; models in a registry.

DK3. Should a container run as root?

No, if it does not need to. The image built without a USER instruction ran as uid 0, root; the one that created a user and switched to it ran as uid 1000. Docker's own guidance is to use USER when a service can run without privileges (Docker: best practices). If an attacker breaks out of a model service, running as root makes everything worse.

DK4. What does exit code 137 mean?

The process was killed with signal 9, most often by the out-of-memory killer. A container limited to 64 MB that tried to allocate 300 MB exited with 137, and Docker recorded OOMKilled: true (Docker: resource constraints). ML services hit this when a model is larger than expected, or when a batch of requests loads more data than planned. The fix is to measure memory under realistic load, then set limits with headroom, not to remove limits.

DK5. What is a multi-stage build, and when does it help?

A Dockerfile with several FROM stages: one stage installs compilers and builds wheels or artifacts, and the final stage copies only the results into a slim base image. The build tools never reach the image you ship, which makes it smaller and leaves less to patch. It helps most for libraries that compile native code; for pure-Python dependencies that install as ready-made wheels, a single slim stage, as above, is often enough.

DK6. How do containers use a GPU?

The image needs the CUDA libraries the framework expects, usually by starting from a CUDA or framework base image, and the host needs the NVIDIA driver and container toolkit; the container is then started with GPU access, for example --gpus all. Interviewers mainly check that you know the driver lives on the host while the CUDA runtime lives in the image, and that their versions must be compatible.

Kubernetes questions that follow

Kubernetes turns the same ideas into requests and limits:

resources:
  requests:
    memory: "512Mi"      # what the scheduler reserves when placing the pod
    cpu: "250m"
  limits:
    memory: "1Gi"        # exceeding this gets the container OOM-killed, exit code 137

What is the difference between a request and a limit? The scheduler uses requests to choose a node with enough room; limits cap what the running container may use, and memory limits are enforced by out-of-memory kills (Kubernetes: resource management). A request larger than any node can offer leaves the pod Pending with a FailedScheduling event.

Why do pods restart with OOMKilled? The same reason as DK4: the container went over its memory limit. Check kubectl describe pod for the last state, then raise the limit or reduce the memory the model needs.

How do you update a model without downtime? A Deployment with a readiness probe and a rolling update that never drops below the desired number of ready pods. The Kubernetes for ML article ran exactly this on a local cluster: no failed requests out of hundreds during an update, then a one-command rollback.

Docker or Kubernetes? Docker packages and runs a container; Kubernetes runs many containers across machines and handles scheduling, restarts, scaling and rollouts. Small services may not need Kubernetes at all.

How to answer container questions

• Order Dockerfiles for caching: dependencies first, code last.
• Keep images lean: no data, no secrets, a slim base.
• Run as non-root, with memory measured and limits set.

The packaging steps are taught in Docker for machine learning and serving ML models with FastAPI and Docker. The wider round is covered in MLOps interview questions and answers.

Containers and deployment are part of the MLOps course.

Sources

• Docker build cache (Docker documentation) — Docker
• Build context (Docker documentation) — Docker
• Building best practices (Docker documentation) — Docker
• Resource constraints (Docker documentation) — Docker
• Resource Management for Pods and Containers (Kubernetes documentation) — The Kubernetes Authors