MLOps Interview Questions and Answers

MLOps interview questions test whether you can keep a model useful after it ships: package it, release it safely, watch it, and decide what to do when it gets worse. This guide maps the rounds with model answers, then works a real degradation case, because "the model got worse, what do you do?" is the question every MLOps interview gets to eventually.

The rounds and model answers

Packaging and serving. "How would you deploy this model?" Containerise it with pinned versions, load it from a registry by alias rather than a file path, add a health check, and roll out gradually. Containers and orchestration questions are covered in Docker and Kubernetes interview questions for ML, and versioning code in Git interview questions.

Pipelines and CI/CD. "What runs before a model reaches production?" Data checks, training with logged parameters and data versions, evaluation against the current model on the same held-out data, and a gate that blocks worse models. See CI/CD for machine learning and the model registry guide.

Monitoring. "What would you monitor?" Service health, input data quality, input and prediction drift, and above all performance once true outcomes arrive. See monitoring ML models in production.

Retraining. "When do you retrain?" On a schedule, on a trigger, or both, but always through the same evaluation gate. See continuous training.

Behavioural. "Tell me about a model incident." Explain how it was detected, who was affected, how you rolled back or mitigated, and what monitoring or gate you added afterwards.

System design. "Design an ML platform for a team of ten data scientists." Start from their workflow and constraints, not from a list of tools; a complete loop is walked through in building an end-to-end MLOps pipeline.

A degradation case, worked on real data

The model, a histogram-based gradient boosting regressor (scikit-learn: HistGradientBoostingRegressor), predicts a New York yellow taxi trip's total fare from distance, pickup and drop-off zone, hour and weekday. It is trained on January 2024 trips and then faces January 2025, the month New York's congestion pricing began. The TLC added a cbd_congestion_fee column to its 2025 data for the new charge (NYC TLC: trip record data).

import numpy as np
import pandas as pd
from sklearn.ensemble import HistGradientBoostingRegressor

cols = ["tpep_pickup_datetime", "trip_distance", "PULocationID", "DOLocationID", "total_amount"]


def load(path, year):                                           # NYC TLC yellow taxi trips for one January
    t = pd.read_parquet(path, columns=cols)
    t = t[(t.tpep_pickup_datetime.dt.year == year) & (t.tpep_pickup_datetime.dt.month == 1)
          & t.trip_distance.between(0.1, 50) & t.total_amount.between(3, 300)]
    return t.assign(hour=t.tpep_pickup_datetime.dt.hour, weekday=t.tpep_pickup_datetime.dt.dayofweek)


old, new = load("yellow_tripdata_2024-01.parquet", 2024), load("yellow_tripdata_2025-01.parquet", 2025)
features = ["trip_distance", "PULocationID", "DOLocationID", "hour", "weekday"]
train = old.sample(400_000, random_state=0)
holdout = old.drop(train.index).sample(200_000, random_state=1)
later = new.sample(200_000, random_state=2)
model = HistGradientBoostingRegressor(random_state=0).fit(train[features], train.total_amount)

mae = lambda d: np.abs(model.predict(d[features]) - d.total_amount).mean()
bias = lambda d: (d.total_amount - model.predict(d[features])).mean()
print(f"1. total fare model, trained on January 2024: MAE {mae(holdout):.2f} USD on held-out 2024 trips, {mae(later):.2f} on January 2025")
print(f"   average error (actual minus predicted): 2024 {bias(holdout):+.2f} USD | 2025 {bias(later):+.2f} USD")


def psi(expected, actual, bins=10):                             # population stability index on deciles of the old data
    edges = np.unique(np.quantile(expected, np.linspace(0, 1, bins + 1)))
    e = np.histogram(np.clip(expected, edges[0], edges[-1]), edges)[0] / len(expected)
    a = np.histogram(np.clip(actual, edges[0], edges[-1]), edges)[0] / len(actual)
    e, a = np.clip(e, 1e-6, None), np.clip(a, 1e-6, None)
    return float(np.sum((a - e) * np.log(a / e)))


print("\n2. drift, January 2024 -> January 2025 (PSI; under 0.1 is usually read as stable, over 0.25 as a major shift)")
for c in ("trip_distance", "hour", "total_amount"):
    print(f"   {c:<14} PSI {psi(old[c].to_numpy(), new[c].to_numpy()):.3f} | median {old[c].median():.2f} -> {new[c].median():.2f}")

added = sorted(set(pd.read_parquet("yellow_tripdata_2025-01.parquet").columns) - set(pd.read_parquet("yellow_tripdata_2024-01.parquet").columns))
print(f"\n3. schema check: columns new in 2025: {added}")
fee = pd.read_parquet("yellow_tripdata_2025-01.parquet", columns=["cbd_congestion_fee"]).cbd_congestion_fee
print(f"   share of 2025 trips charged the fee: {(fee > 0).mean():.1%}, mean fee on those trips {fee[fee > 0].mean():.2f} USD")

fresh = new.drop(later.index).sample(400_000, random_state=3)
retrained = HistGradientBoostingRegressor(random_state=0).fit(fresh[features], fresh.total_amount)
print(f"\n4. retrained on other January 2025 trips: MAE on the same 2025 sample {np.abs(retrained.predict(later[features]) - later.total_amount).mean():.2f} USD")

1. total fare model, trained on January 2024: MAE 3.08 USD on held-out 2024 trips, 3.53 on January 2025
   average error (actual minus predicted): 2024 +0.01 USD | 2025 -0.21 USD

2. drift, January 2024 -> January 2025 (PSI; under 0.1 is usually read as stable, over 0.25 as a major shift)
   trip_distance  PSI 0.002 | median 1.70 -> 1.71
   hour           PSI 0.003 | median 15.00 -> 15.00
   total_amount   PSI 0.005 | median 20.16 -> 20.22

3. schema check: columns new in 2025: ['cbd_congestion_fee']
   share of 2025 trips charged the fee: 64.6%, mean fee on those trips 0.75 USD

4. retrained on other January 2025 trips: MAE on the same 2025 sample 3.50 USD

How a strong candidate reads this

Performance got worse. Mean absolute error rose from 3.08 USD on held-out 2024 trips to 3.53 on January 2025, about 15%. That is the signal that matters, and it is only visible because true fares arrive with each trip. In many systems the truth comes weeks later, which is why the drift checks exist.

Drift metrics stayed calm. The population stability index was 0.002 for trip distance, 0.003 for hour and 0.005 for the fare itself, all far below the common 0.10 threshold for "little shift" (Yurdakul 2018). Medians barely moved. A team that watched only these numbers would have seen nothing. The same source notes that the usual PSI thresholds are rules of thumb without known error rates, which is a good point to make in an interview.

The schema check did catch something. A new column appeared, and 64.6% of 2025 trips were charged the fee, 0.75 USD on average. A schema change in a source is exactly the kind of event that should trigger a closer look, even when the distributions look stable.

Retraining was not the fix. A model trained on other January 2025 trips scored 3.50, barely better than the old model's 3.53. The fee is small and partly predictable from zones; most of the extra error is not something more recent training data removes. Automatic retraining would have spent compute, produced a new version, and left the problem in place. The right next step is analysis: break the error down by zone, hour and fee status, look at what else changed in 2025, and decide whether the model needs new features rather than new data.

The general lesson: drift metrics are early warnings, not proof of harm or of health. Performance against ground truth is the measure, and retraining is one possible response among several.

How to prepare

Build one small project that goes the whole way: a model in a container, loaded from a registry, tested in CI, monitored with real outcomes and retrained behind a gate. Then break it on purpose, change the input data or ship a worse model, and practise explaining how your system noticed. That story answers half the questions in any MLOps round.

Mistakes that cost offers

• Monitoring only drift and never actual performance.
• Retraining automatically without a gate against the current model.
• Ignoring schema changes in upstream data.
• Listing tools instead of describing the decisions they support.

These practices are taught hands-on in the MLOps course.

Sources

• TLC Trip Record Data (NYC Taxi and Limousine Commission) — NYC Taxi and Limousine Commission
• Statistical Properties of Population Stability Index (Yurdakul, Western Michigan University, 2018) — Western Michigan University
• HistGradientBoostingRegressor (scikit-learn documentation) — scikit-learn