Blue-Green Deployments for ML Services
Blue-green deployment keeps the old model serving while the new one waits beside it, so switching either way is quick. This guide runs it with two real spam filters, shows a pre-switch check catching a score-scale change that would have made a review queue five times longer, and switches under live traffic.
GloryTecks Team
Training & Placement Faculty
Last updated: 8 October 2026
Blue-green deployment runs two production environments side by side. Blue serves users while green receives the new version. Once green has been checked, the router sends traffic to green, and if anything goes wrong it sends traffic back to blue. Both switches take a moment, because both environments are already running.
For ML services the switch is the easy part. The hard part is knowing green is ready, and for models "ready" includes something a schema check cannot see: what the scores mean.
How it works
Martin Fowler's description is the standard one: two production environments, as identical as possible. You put the new release on the idle one, switch the router to it once it is checked, and switch back if anything goes wrong. He also warns that databases are often the hard part, particularly when a schema has to change (martinfowler.com: blue-green deployment).
- Deploy the new version to the idle environment.
- Warm it up and check it against production-like traffic.
- Switch the router.
- Keep the old environment running, ready for an instant rollback.
- Retire it, or keep it as the next idle environment.
What is different for models
- An environment is more than a model file. Ship the artefact, the feature code, the dependencies and the decision thresholds together.
- Load models before the switch, so the first users do not pay the start-up cost.
- Shared state must suit both versions. Feature tables, schemas and prediction logs have to work for blue and green at once.
- Scores carry meaning. Two models can return the same JSON while their scores sit on different scales. scikit-learn's guide notes that logistic regression is more likely to be well calibrated by itself, while naive Bayes tends to push probabilities towards 0 or 1 (scikit-learn: probability calibration).
The demo
The data are the SMS Spam Collection: 5,574 text messages, 747 of them spam, collected by Tiago Almeida and José María Gómez Hidalgo, under CC BY 4.0 (UCI: SMS Spam Collection). Blue is naive Bayes on word counts. Green is logistic regression on character n-grams. Both train on 60% of the messages; 10% is kept for validation and 30% is replayed as recent traffic.
Downstream, a message is blocked above a threshold, sent to manual review between 0.5 and that threshold, and delivered otherwise. Three local servers stand in for production: blue, green and a router. The same service code runs in both environments:
import json
import os
import joblib
from fastapi import FastAPI
from pydantic import BaseModel
COLOR = os.environ["MODEL_COLOR"] # blue or green: one image, two environments
model = joblib.load(f"models/{COLOR}.joblib") # loaded at start-up, before any traffic arrives
config = json.load(open(f"models/{COLOR}.json")) # thresholds ship with the model they belong to
class Message(BaseModel):
text: str
app = FastAPI()
@app.get("/health")
def health():
return {"status": "ok", "model_version": config["version"]}
@app.post("/predict")
def predict(message: Message):
score = float(model.predict_proba([message.text])[0, 1])
action = "block" if score >= config["block_at"] else "review" if score >= config["review_at"] else "deliver"
return {"spam_score": round(score, 4), "action": action, "model_version": config["version"]}The router forwards each request to whichever environment is live:
import httpx
from fastapi import FastAPI, Request, Response
ENVIRONMENTS = {"blue": "http://127.0.0.1:8001", "green": "http://127.0.0.1:8002"}
live = {"color": "blue"} # the only thing a switch changes
client = httpx.AsyncClient()
app = FastAPI()
@app.post("/admin/switch")
async def switch(target: dict):
live["color"] = target["to"] # in-flight requests finish where they started
return live
@app.post("/predict")
async def predict(request: Request):
upstream = await client.post(f"{ENVIRONMENTS[live['color']]}/predict", content=await request.body(),
headers={"content-type": "application/json"})
return Response(upstream.content, upstream.status_code, media_type="application/json")The release script trains both models, checks green before switching, and then switches back and forth while traffic flows:
import json
import os
import subprocess
import sys
import threading
import time
import joblib
import numpy as np
import pandas as pd
import requests
from sklearn.feature_extraction.text import CountVectorizer, TfidfVectorizer
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import train_test_split
from sklearn.naive_bayes import MultinomialNB
from sklearn.pipeline import make_pipeline
sms = pd.read_csv("SMSSpamCollection", sep="\t", names=["label", "text"], quoting=3) # UCI SMS Spam Collection
sms["spam"] = sms.label == "spam"
train, rest = train_test_split(sms, test_size=0.4, random_state=0, stratify=sms.spam)
validation, recent = train_test_split(rest, test_size=0.75, random_state=0, stratify=rest.spam)
blue = make_pipeline(CountVectorizer(), MultinomialNB()).fit(train.text, train.spam)
green = make_pipeline(TfidfVectorizer(analyzer="char_wb", ngram_range=(2, 5), sublinear_tf=True),
LogisticRegression(C=10, max_iter=2000)).fit(train.text, train.spam)
os.makedirs("models", exist_ok=True)
def release(color, model, block_at): # an environment's artefacts: model plus its thresholds
joblib.dump(model, f"models/{color}.joblib")
json.dump({"version": f"spam-{color}", "block_at": block_at, "review_at": 0.5}, open(f"models/{color}.json", "w"))
def start(port, app, color=""):
return subprocess.Popen([sys.executable, "-m", "uvicorn", app, "--port", str(port), "--log-level", "warning"],
env=dict(os.environ, MODEL_COLOR=color))
def stop(process):
if os.name == "nt": # Windows: stop the whole process tree, not just the launcher
subprocess.run(["taskkill", "/T", "/F", "/PID", str(process.pid)], capture_output=True)
else:
process.terminate()
process.wait()
def ready(url):
for _ in range(60):
try:
if requests.get(url, timeout=1).ok:
return
except requests.ConnectionError:
time.sleep(0.5)
raise RuntimeError(f"{url} did not start")
BLUE, GREEN, ROUTER = "http://127.0.0.1:8001", "http://127.0.0.1:8002", "http://127.0.0.1:8000"
def outcomes(base_url): # replay recent traffic directly against one environment
with requests.Session() as session:
answers = pd.DataFrame([session.post(f"{base_url}/predict", json={"text": t}).json() for t in recent.text])
action = answers.action.to_numpy()
spam = recent.spam.to_numpy()
return {"block": (action == "block").mean(), "review": (action == "review").mean(),
"spam blocked": (action == "block")[spam].mean(), "ham blocked": int((action == "block")[~spam].sum()),
"fields": sorted(answers.columns)}
def check(label):
b, g = outcomes(BLUE), outcomes(GREEN)
print(f"{label}\n same response fields as blue: {b['fields'] == g['fields']}")
for name, o in (("blue (live)", b), ("green (idle)", g)):
print(f" {name:<13} blocks {o['block']:5.1%}, reviews {o['review']:4.1%}; "
f"spam blocked {o['spam blocked']:.0%}, legitimate messages blocked {o['ham blocked']}")
return g["review"] <= 2 * b["review"] + 0.01 # release rule: the review queue may not more than double
release("blue", blue, 0.9)
release("green", green, 0.9)
servers = {"blue": start(8001, "service:app", "blue"), "green": start(8002, "service:app", "green"),
"router": start(8000, "router:app")}
try:
for url in (f"{BLUE}/health", f"{GREEN}/health"):
ready(url)
if not check("Check 1: green with blue's thresholds"):
print(" -> hold the switch: green's scores sit on a different scale")
target = np.quantile(green.predict_proba(validation.text)[:, 1], 1 - (blue.predict_proba(validation.text)[:, 1] >= 0.9).mean())
release("green", green, round(float(target), 3)) # block the share blue blocked on validation messages
stop(servers["green"])
servers["green"] = start(8002, "service:app", "green") # redeploy green; blue keeps serving throughout
ready(f"{GREEN}/health")
print(f" green redeployed with block_at = {round(float(target), 3)}")
print(" -> switch" if check("Check 2: green as redeployed") else " -> hold")
seen, failures, finished = [], [], threading.Event()
def traffic():
with requests.Session() as session:
while not finished.is_set():
for text in recent.text:
if finished.is_set():
break
reply = session.post(f"{ROUTER}/predict", json={"text": text})
if reply.status_code != 200:
failures.append(reply.status_code)
elif not seen or seen[-1] != reply.json()["model_version"]:
seen.append(reply.json()["model_version"])
ready(f"{ROUTER}/docs")
worker = threading.Thread(target=traffic)
worker.start()
for target in ("green", "blue", "green"): # switch, roll back as a drill, switch again
time.sleep(1)
requests.post(f"{ROUTER}/admin/switch", json={"to": target})
time.sleep(1)
finished.set()
worker.join()
print("\nSwitching under continuous traffic")
print(" versions that answered, in order:", " -> ".join(seen))
print(" failed requests:", len(failures))
finally:
for server in servers.values():
stop(server)Check 1: green with blue's thresholds
same response fields as blue: True
blue (live) blocks 11.7%, reviews 0.7%; spam blocked 87%, legitimate messages blocked 1
green (idle) blocks 8.8%, reviews 3.6%; spam blocked 66%, legitimate messages blocked 0
-> hold the switch: green's scores sit on a different scale
green redeployed with block_at = 0.721
Check 2: green as redeployed
same response fields as blue: True
blue (live) blocks 11.7%, reviews 0.7%; spam blocked 87%, legitimate messages blocked 1
green (idle) blocks 11.2%, reviews 1.2%; spam blocked 84%, legitimate messages blocked 0
-> switch
Switching under continuous traffic
versions that answered, in order: spam-blue -> spam-green -> spam-blue -> spam-green
failed requests: 0What the run shows
- A schema check passed and proved little. Green returned exactly the same fields as blue.
- The outcome check caught the real problem. With blue's thresholds, green would have blocked 66% of spam instead of 87%, and sent 3.6% of all messages to manual review instead of 0.7%, about five times the queue. The release rule held the switch, and no user noticed, because green was idle.
- The fix was a release, not a hot patch. Green was redeployed with its own block threshold, 0.721, chosen on validation messages so that it blocks the same share blue did. It then blocked 84% of spam, sent 1.2% to review and blocked no legitimate message, where blue had blocked one.
- Switching under load cost nothing. Traffic moved from blue to green, back to blue as a rollback drill, and to green again, with zero failed requests. Requests already in flight finished where they started.
The same switch on Kubernetes
On Kubernetes, a Service picks the Pods it sends traffic to with a label selector (Kubernetes: Service). Run blue and green as two Deployments with different labels, and the switch becomes a change to the selector:
# Two Deployments, spam-filter-blue and spam-filter-green, carry the labels app: spam-filter and color: blue|green.
# The Service sends traffic to whichever color its selector names.
apiVersion: v1
kind: Service
metadata:
name: spam-filter
spec:
selector:
app: spam-filter
color: blue
ports:
- port: 80
targetPort: 8000
# Switch: kubectl patch service spam-filter -p '{"spec":{"selector":{"app":"spam-filter","color":"green"}}}'
# Roll back: the same command with "blue"For running models on a cluster in more depth, see Kubernetes for ML.
Costs and limits
- Double capacity while both environments run.
- Everyone moves at once. When the risk is high, use shadow mode or a canary first; see shadow deployment and canary releases.
- Schema changes need backward-compatible steps, so blue and green can share a database during the switch.
- A switch does not measure improvement. For that, see A/B testing ML models.
For the service pattern itself, see serving ML models with FastAPI and Docker, and for keeping versions ready to restore, see the model registry.
Common mistakes
- Treating a matching schema as compatibility.
- Shipping a model without its thresholds.
- Deleting blue straight after the switch.
- Never practising the rollback.
- Switching to an environment that has not loaded its model.
What is blue-green deployment in machine learning?
How is blue-green deployment different from a canary release?
How do you roll back a blue-green deployment?
What should be checked before switching a model?
Deployment strategies are part of the Model Deployment module of the MLOps course, and Kubernetes services are covered in its Kubernetes for MLOps module.
Sources
- Blue Green Deployment (martinfowler.com) — Martin Fowler
- Service (Kubernetes documentation) — Kubernetes
- Probability calibration (scikit-learn documentation) — scikit-learn
- SMS Spam Collection (UCI Machine Learning Repository) — UCI Machine Learning Repository
GloryTecks Team
Training & Placement Faculty
The GloryTecks faculty team mentors learners across Data Science, AI and Data Engineering, with a focus on real-time projects and placement readiness in Hyderabad.
Want to master MLOps?
Join GloryTecks’ expert-led MLOps training in Hyderabad with real-time projects and 100% placement support.
Related articles
MLOps Roadmap 2026: Skills, Tools and Career Path
MLOps sits between data science, software engineering and operations, which makes it hard to know where to start. This roadmap orders the skills from foundations to production operations, gives each stage a project that proves the skill, and explains what the MLOps engineer role actually involves.
MLflow Tutorial: Tracking Experiments and Models
MLflow records what produced each model so you never have to remember. This tutorial trains three models on real bike-sharing data, logs every run, compares them by querying the tracking store, registers the best one, points a champion alias at it and loads it back by that alias, all on a laptop.
Docker for Machine Learning: A Practical Guide
How to containerise a machine-learning project so it runs the same everywhere: pinning the exact library versions a saved model depends on, a Dockerfile ordered for fast rebuilds, keeping data out of images, multi-stage builds, non-root users and GPU access.