Model Deployment: From Notebook to Production

A model that scores well in a notebook has proved that a problem can be solved. It has not yet solved it for anyone. Between the two lie several steps that have little to do with accuracy: making training repeatable, making sure the model sees data in production exactly as it did in training, checking inputs, and noticing when the world changes. This guide walks through those steps with a small credit risk model built on the German credit dataset, 1,000 loan applicants labelled good or bad risks (OpenML: credit-g).

First, decide how predictions will be used

The right deployment depends on when predictions are needed:

Start with batch if the decision can wait. It is simpler to build, test and monitor, and the scores can go straight into dashboards people already use. Batch vs real-time inference compares the two in depth.

Step 1: turn the notebook into a training script

Notebooks encourage running cells out of order and keeping state in memory, and a result that depends on that cannot be reproduced. Before anything else:

• Restart and run all. If the notebook fails or gives different numbers, fix that first.
• Move the steps into functions and a script: load data, build features, train, evaluate, save. The script should run from start to finish with one command.
• Fix random seeds, pin library versions in a requirements file, and record which data snapshot was used.

Step 2: package the whole pipeline, not just the model

The most common deployment bug is preprocessing that exists only in the notebook. Put every transformation inside one pipeline, so scaling and encoding are fitted with the model and saved with it (scikit-learn: ColumnTransformer):

import json
import joblib
import pandas as pd
import sklearn
from sklearn.compose import make_column_transformer
from sklearn.datasets import fetch_openml
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import cross_val_score
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import OneHotEncoder, StandardScaler

credit = fetch_openml(data_id=31, as_frame=True).frame
features = ["duration", "credit_amount", "age", "purpose", "housing"]
X, y = credit[features], (credit["class"] == "bad").astype(int)

pipeline = make_pipeline(
    make_column_transformer(
        (StandardScaler(), ["duration", "credit_amount", "age"]),
        (OneHotEncoder(handle_unknown="ignore"), ["purpose", "housing"])),
    LogisticRegression(max_iter=1000))
cv_auc = cross_val_score(pipeline, X, y, cv=5, scoring="roc_auc").mean()
pipeline.fit(X, y)

joblib.dump(pipeline, "credit_risk.joblib")
with open("credit_risk.json", "w") as f:
    json.dump({"features": features, "scikit_learn": sklearn.__version__, "cv_roc_auc": round(cv_auc, 3)}, f)
print("CV ROC AUC:", round(cv_auc, 3), "| saved with scikit-learn", sklearn.__version__)

CV ROC AUC: 0.689 | saved with scikit-learn 1.8.0

The model is deliberately simple, using five of the dataset's twenty attributes; the point here is the packaging. Two details matter. The metadata file records the features, the library version and the validation score, because a saved model should be loaded with the same scikit-learn version that saved it, and only from sources you trust, since loading such files can execute code (scikit-learn: model persistence). And handle_unknown="ignore" means a category never seen in training does not crash the service.

A caution specific to this example: it uses applicant age as a feature. In real credit decisions, using attributes such as age can be restricted by law and raises fairness questions, so any real model needs that review before deployment.

Step 3: avoid training-serving skew

Google's Rules of Machine Learning define training-serving skew as a difference between a model's performance in training and in serving, and one of its main causes is handling data differently in the two places. Their advice: re-use code between the training pipeline and the serving pipeline whenever possible (Google: rules of machine learning). Here is why, using the common notebook habit of one-hot encoding with pd.get_dummies:

train_columns = pd.get_dummies(X).columns
one_request = pd.DataFrame([{"duration": 24, "credit_amount": 4000, "age": 35, "purpose": "used car", "housing": "rent"}])
print("columns at training:", len(train_columns), "| columns for one request:", len(pd.get_dummies(one_request).columns))
print("pipeline prediction:", pipeline.predict_proba(one_request)[0, 1].round(3))

columns at training: 16 | columns for one request: 5
pipeline prediction: 0.193

Encoded in the notebook, the training data had 16 columns: the three numbers plus one column per purpose and housing category. A single incoming application, encoded the same way, has only 5, because it contains one purpose and one housing value. A model trained on 16 columns either rejects the request or, worse, if someone pads the columns by hand in the wrong order, returns a confident wrong answer. The saved pipeline carries its fitted encoder, so it produces the same 16 inputs for one row as for a thousand, and returns a risk of 0.193.

Step 4: serve it behind a validated API

For online use, wrap the saved pipeline in a small web service. FastAPI with Pydantic lets you declare what a valid request looks like, so bad inputs are rejected before they reach the model. FastAPI's TestClient sends requests to the app directly, which makes the service testable without starting a server (FastAPI: testing):

from typing import Literal
from fastapi import FastAPI
from fastapi.testclient import TestClient
from pydantic import BaseModel, Field

model = joblib.load("credit_risk.joblib")
app = FastAPI()

class Application(BaseModel):
    duration: int = Field(gt=0, le=120)
    credit_amount: float = Field(gt=0)
    age: int = Field(ge=18, le=100)
    purpose: str
    housing: Literal["own", "rent", "for free"]

@app.post("/score")
def score(application: Application):
    row = pd.DataFrame([application.model_dump()])
    return {"risk": round(float(model.predict_proba(row)[0, 1]), 3), "model_version": "credit_risk-2026-09"}

client = TestClient(app)
ok = client.post("/score", json={"duration": 24, "credit_amount": 4000, "age": 35, "purpose": "used car", "housing": "rent"})
print(ok.status_code, ok.json())
bad = client.post("/score", json={"duration": 24, "credit_amount": 4000, "age": 15, "purpose": "used car", "housing": "castle"})
print(bad.status_code, [(e["loc"][-1], e["msg"]) for e in bad.json()["detail"]])

200 {'risk': 0.193, 'model_version': 'credit_risk-2026-09'}
422 [('age', 'Input should be greater than or equal to 18'), ('housing', "Input should be 'own', 'rent' or 'for free'")]

The valid request returns the same risk as the pipeline did directly, with the model version, so every prediction can be traced to the model that made it. The invalid one is refused with a 422 status and a message per problem field. Tests like these belong in the project's test suite and run on every change. Packaging this service in a container and running it is the subject of serving ML models with FastAPI and Docker.

Step 5: release carefully

• Version everything: model files, training code, data snapshot and metadata, so any prediction can be reproduced.
• Release gradually. Run the new model alongside the old one without acting on its outputs, or send it a small share of traffic first, and compare. Shadow deployment and canary releases covers both.
• Keep the previous model ready, so rolling back is one configuration change.

Step 6: monitor what the model sees and does

A deployed model degrades silently when the world changes. Watch four things:

• Inputs: distributions of incoming features compared with training data, plus missing values and unseen categories.
• Outputs: the distribution of predicted risks; a sudden shift is often the first sign of trouble.
• Outcomes: once real results arrive, such as whether loans were repaid, recompute the validation metric on recent predictions.
• Service health: latency, error rates and rejected requests.

Rule 29 in Google's guide suggests logging the features used at serving time so they can be used for training, which also makes skew easy to detect. Decide in advance which changes trigger an investigation or retraining.

A deployment checklist

Model serialisation with joblib, REST APIs with FastAPI, batch versus real-time inference, model versioning, and monitoring for drift with retraining are all part of the MLOps course.

Sources

• Rules of Machine Learning (Google) — Google for Developers
• ColumnTransformer for heterogeneous data (scikit-learn) — scikit-learn documentation
• Model persistence (scikit-learn) — scikit-learn documentation
• Testing (FastAPI documentation) — FastAPI documentation
• credit-g (OpenML dataset 31) — www.openml.org