Building a Prediction API: A Complete Walkthrough
A prediction API can return confident, well-formed and completely wrong answers. This walkthrough builds a FastAPI service for a real model and the four tests that catch what usual API tests miss: a feature-order bug, inputs that should be refused, a NaN that crashes the error handler, and batch results that drift.
GloryTecks Team
Training & Placement Faculty
Last updated: 10 October 2026
Building a prediction API looks like the easy part of machine learning: load the model, accept JSON, return a number. The usual API tests then check that the endpoint answers with status 200 and a number between 0 and 1. The trouble is that a broken prediction API also answers with status 200 and a number between 0 and 1. Model APIs fail silently, so they need tests that know what the right answer is.
This walkthrough builds a small FastAPI service around a real model and runs four checks that ordinary API tests miss. The FastAPI and Docker serving guide covers the production design of such a service, and the FastAPI guide covers FastAPI itself. This article is about proving the API gives the right answers.
The model predicts whether a red wine scores 7 or more for quality, from 11 laboratory measurements in the UCI Wine Quality dataset (UCI: Wine Quality). The request schema declares the fields in alphabetical order, as people often do. Version 1 of the API has a realistic bug: it builds the feature vector in the schema's order. Version 2 builds it in the order the model was trained on.
import json, time, warnings
import numpy as np, pandas as pd
from fastapi import FastAPI
from fastapi.exceptions import RequestValidationError
from fastapi.responses import JSONResponse
from fastapi.testclient import TestClient
from pydantic import BaseModel, ConfigDict, Field, create_model
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import train_test_split
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler
warnings.filterwarnings("ignore")
# Train: is a red wine "good" (quality 7 or more)? UCI Wine Quality, CC BY 4.0
wine = pd.read_csv("winequality-red.csv", sep=";")
wine.columns = [c.replace(" ", "_") for c in wine.columns]
TRAINING_COLUMNS = [c for c in wine.columns if c != "quality"]
X, y = wine[TRAINING_COLUMNS].to_numpy(), (wine.quality >= 7).astype(int)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=0, stratify=y)
model = make_pipeline(StandardScaler(), LogisticRegression(max_iter=1000)).fit(X_train, y_train)
# The request schema, with fields declared alphabetically (as people often do)
class Wine(BaseModel):
model_config = ConfigDict(extra="forbid")
alcohol: float = Field(ge=0, allow_inf_nan=False)
chlorides: float = Field(ge=0, allow_inf_nan=False)
citric_acid: float = Field(ge=0, allow_inf_nan=False)
density: float = Field(gt=0, allow_inf_nan=False)
fixed_acidity: float = Field(ge=0, allow_inf_nan=False)
free_sulfur_dioxide: float = Field(ge=0, allow_inf_nan=False)
pH: float = Field(ge=0, le=14, allow_inf_nan=False)
residual_sugar: float = Field(ge=0, allow_inf_nan=False)
sulphates: float = Field(ge=0, allow_inf_nan=False)
total_sulfur_dioxide: float = Field(ge=0, allow_inf_nan=False)
volatile_acidity: float = Field(ge=0, allow_inf_nan=False)
# A lenient schema for comparison: plain floats, no limits
LenientWine = create_model("LenientWine", **{c: (float, ...) for c in TRAINING_COLUMNS})
def make_app(column_order_fixed, schema=Wine, safe_errors=False):
app = FastAPI()
if safe_errors: # report what was wrong, without echoing the submitted value
@app.exception_handler(RequestValidationError)
async def validation_error(request, exc):
detail = [{"loc": e["loc"], "msg": e["msg"], "type": e["type"]} for e in exc.errors()]
return JSONResponse(status_code=422, content={"detail": detail})
def features(w):
d = w.model_dump()
return [d[c] for c in TRAINING_COLUMNS] if column_order_fixed else list(d.values()) # v1 bug: schema order
@app.post("/predict")
def predict(w: schema):
return {"p_good": float(model.predict_proba([features(w)])[0, 1])}
@app.post("/predict/batch")
def predict_batch(ws: list[schema]):
return {"p_good": model.predict_proba([features(w) for w in ws])[:, 1].tolist()}
return app
def payload(row):
return dict(zip(TRAINING_COLUMNS, map(float, row)))
golden_rows = X_test[:20]
golden_expected = model.predict_proba(golden_rows)[:, 1] # recorded offline, at training time
for name, fixed in [("v1 (schema order)", False), ("v2 (training order)", True)]:
client = TestClient(make_app(fixed))
got = np.array([client.post("/predict", json=payload(r)).json()["p_good"] for r in golden_rows])
wrong = np.abs(got - golden_expected) > 1e-9
print(f"T1 golden predictions, {name}: {wrong.sum()} of {len(golden_rows)} differ, largest gap {np.abs(got - golden_expected).max():.3f}")
def post_raw(client, body): # raw JSON text, so NaN can be sent
return client.post("/predict", content=json.dumps(body), headers={"content-type": "application/json"})
lenient = TestClient(make_app(True, LenientWine), raise_server_exceptions=False)
strict = TestClient(make_app(True), raise_server_exceptions=False)
client = TestClient(make_app(True, safe_errors=True), raise_server_exceptions=False)
good = payload(golden_rows[0])
cases = {
"valid request": good,
"missing field": {k: v for k, v in good.items() if k != "alcohol"},
"negative alcohol": {**good, "alcohol": -1.0},
"pH of 20": {**good, "pH": 20.0},
"alcohol is NaN": {**good, "alcohol": float("nan")},
"alcohol as text": {**good, "alcohol": "twelve"},
"unknown extra field": {**good, "colour": "red"},
}
print("\nT2 input validation (status codes) lenient strict strict + error handler")
for label, body in cases.items():
codes = [post_raw(c, body).status_code for c in (lenient, strict, client)]
print(f" {label:<33}{codes[0]:>8}{codes[1]:>9}{codes[2]:>11}")
batch = [payload(r) for r in X_test[:50]]
batch_result = client.post("/predict/batch", json=batch).json()["p_good"]
single = [client.post("/predict", json=b).json()["p_good"] for b in batch]
print(f"\nT3 batch and single endpoints agree on all 50 rows: {np.allclose(batch_result, single, rtol=0, atol=1e-12)}")
times = []
for r in X_test[:300]:
start = time.perf_counter(); client.post("/predict", json=payload(r)); times.append((time.perf_counter() - start) * 1000)
print(f"T4 in-process p95 latency over 300 calls within a 50 ms budget: {np.percentile(times, 95) < 50}")T1 golden predictions, v1 (schema order): 20 of 20 differ, largest gap 0.996
T1 golden predictions, v2 (training order): 0 of 20 differ, largest gap 0.000
T2 input validation (status codes) lenient strict strict + error handler
valid request 200 200 200
missing field 422 422 422
negative alcohol 200 422 422
pH of 20 200 422 422
alcohol is NaN 500 500 422
alcohol as text 422 422 422
unknown extra field 200 422 422
T3 batch and single endpoints agree on all 50 rows: True
T4 in-process p95 latency over 300 calls within a 50 ms budget: TrueCheck 1: golden predictions
Version 1 returned a valid probability for every request, and every one was wrong, by up to 0.996. The model was trained on columns in the dataset's order, while the API passed them alphabetically, so alcohol went where fixed acidity belonged. No error, no warning: the model accepts any 11 numbers.
The test that catches this is simple. When the model is trained, record its predictions for a few dozen held-out rows. In the API's test suite, send the same rows through the endpoint and require the same outputs. Version 2 matched all 20 exactly. This golden-prediction test catches any difference between training and serving: column order, a missing scaling step, a different preprocessing version, or a model file swapped by mistake. It is the most valuable test a prediction API can have.
Check 2: refuse what the model cannot handle
A lenient schema, with plain floats, accepted a negative alcohol value, a pH of 20 and an unknown extra field, and the model produced predictions for all of them. Pydantic ignores extra fields by default and accepts NaN and infinity in float fields by default (Pydantic: configuration). An extra field is often a client's typo or an outdated contract, and silently ignoring it hides the mismatch.
The strict schema sets ranges on each field, forbids extra fields and rejects NaN. It returned 422 for every bad input. Base the ranges on what the model saw in training and what is physically possible, not on guesses.
Check 3: the NaN that crashed the error handler
One result is surprising. With the strict schema, a NaN input still returned 500, not 422. Pydantic rejected the value as intended, but in this run, FastAPI's default validation response included the submitted value, and NaN cannot be written as standard JSON. The error response itself failed. A client sending one bad number would see a server error, and your monitoring would count a bug.
FastAPI lets you replace the default validation handler (FastAPI: handling errors). The version here returns each error's location, message and type, without echoing the input. That turned the 500 into a proper 422, and it has a second benefit: submitted values, which may be personal data, no longer appear in error responses and the logs that capture them. Add NaN, infinity and very large numbers to your test cases, sent as raw JSON text, because standard JSON encoders refuse to produce them.
Check 4: batch and single agree, within a latency budget
Many prediction APIs offer a batch endpoint for efficiency. The batch and single paths often share less code than people think, and a difference in either can go unnoticed for months. A test that sends the same 50 rows both ways and requires identical results costs a second to run. Here they agreed.
A latency budget turns performance into a test too. Over 300 in-process calls, the 95th percentile stayed within the illustrative 50 ms budget. In-process timing excludes the network, so treat it as a regression guard that catches a slow change in your code, not as the latency users will see. Measure that against the deployed service. For when real-time serving is the right choice at all, see batch vs real-time inference.
The test suite to keep
- Golden predictions: recorded at training time, versioned with the model, checked on every build.
- Validation cases: missing fields, wrong types, out-of-range values, NaN and infinity, extra fields.
- Error responses: every bad input returns 422 with a useful message and no echoed data.
- Path consistency: batch and single endpoints agree.
- Latency budget: a percentile limit that catches regressions.
These complement the model and data tests in testing ML code and data: those prove the model is right; these prove the API delivers the model's answers.
How do you test a machine learning prediction API?
What is a golden prediction test?
Why does my FastAPI endpoint return 500 for NaN input?
Should a prediction API reject extra fields?
Serving, testing and monitoring models are part of the MLOps course.
Sources
- Wine Quality (UCI Machine Learning Repository) — UCI Machine Learning Repository
- Configuration (Pydantic documentation) — Pydantic
- Handling errors (FastAPI documentation) — FastAPI
GloryTecks Team
Training & Placement Faculty
The GloryTecks faculty team mentors learners across Data Science, AI and Data Engineering, with a focus on real-time projects and placement readiness in Hyderabad.
Want to master MLOps?
Join GloryTecks’ expert-led MLOps training in Hyderabad with real-time projects and 100% placement support.
Related articles
MLOps Roadmap 2026: Skills, Tools and Career Path
MLOps sits between data science, software engineering and operations, which makes it hard to know where to start. This roadmap orders the skills from foundations to production operations, gives each stage a project that proves the skill, and explains what the MLOps engineer role actually involves.
MLflow Tutorial: Tracking Experiments and Models
MLflow records what produced each model so you never have to remember. This tutorial trains three models on real bike-sharing data, logs every run, compares them by querying the tracking store, registers the best one, points a champion alias at it and loads it back by that alias, all on a laptop.
Docker for Machine Learning: A Practical Guide
How to containerise a machine-learning project so it runs the same everywhere: pinning the exact library versions a saved model depends on, a Dockerfile ordered for fast rebuilds, keeping data out of images, multi-stage builds, non-root users and GPU access.