Prompt Templates and Versioning: Managing Prompts at Scale

The first prompt in an application is usually a string in the code. The tenth is often a string in a database that anyone can edit, and nobody can say which wording was live last Tuesday when the complaints started.

Prompts change behaviour as much as code does, so they need the same care: a place to live, a version, a review, a test and a way back. OpenAI's own guidance has moved firmly in this direction. It now says to treat prompts as application code, kept in versioned modules and reviewed in pull requests. It is retiring its hosted reusable prompt objects, with the API for them scheduled to shut down on 30 November 2026 (OpenAI: prompting).

This guide builds the pieces of prompt versioning in a few dozen lines of Python, then covers the process around them.

What a prompt version includes

A prompt is not just its wording. The same text behaves differently on another model or at another temperature, and a change of output schema or tools changes it again. A version therefore pins:

• The instructions, with their placeholders.
• The model, exactly as named in the API.
• Generation settings such as temperature and output limits.
• The output schema and tools, if any.

Change any of them and you have a new version, to be evaluated before it ships. See evaluating LLM outputs.

A registry in code

The script keeps two versions of a ticket-summary prompt and does what a prompt store should: lists them, shows what changed, builds a request, refuses bad input, and checks for edits that skipped a version number.

import difflib
import hashlib
import json
from dataclasses import dataclass, replace
from string import Template


@dataclass(frozen=True)
class PromptVersion:
    name: str
    version: str
    model: str            # a prompt is tuned for one model, so the model is part of the version
    temperature: float
    instructions: str     # trusted text; $placeholders take trusted values only

    def fingerprint(self):
        # a hash of everything that changes behaviour, not just the wording
        payload = json.dumps([self.instructions, self.model, self.temperature])
        return hashlib.sha256(payload.encode()).hexdigest()[:12]

    def variables(self):
        return Template(self.instructions).get_identifiers()

    def build(self, ticket, **values):
        # the customer's text never goes through the template: it travels as separate input
        return {
            "instructions": Template(self.instructions).substitute(values),
            "input": ticket,
            "meta": {"prompt": self.name, "version": self.version, "fingerprint": self.fingerprint()},
        }


V1 = PromptVersion("ticket_summary", "1.0.0", "example-model-a", 0.0,
                   "Summarise the support ticket in one sentence.")
V2 = PromptVersion("ticket_summary", "1.1.0", "example-model-a", 0.0,
                   "You summarise support tickets for the $team on-call engineer.\n"
                   "Write one sentence of at most $max_words words: the product and what the customer needs.\n"
                   "If the ticket describes no problem, reply exactly: NO ISSUE")
REGISTRY = [V1, V2]

# written at release time and committed with the code
LOCK = {(p.name, p.version): p.fingerprint() for p in REGISTRY}

print("registry")
print(f"{'name':<15} {'version':<8} {'model':<16} {'fingerprint':<13} variables")
for p in REGISTRY:
    print(f"{p.name:<15} {p.version:<8} {p.model:<16} {p.fingerprint():<13} {', '.join(p.variables()) or '-'}")

print("\nwhat changed from 1.0.0 to 1.1.0")
for line in difflib.unified_diff(V1.instructions.splitlines(), V2.instructions.splitlines(), lineterm="", n=0):
    if not line.startswith(("---", "+++", "@@")):
        print(line)

ticket = "Checkout shows {total} instead of the price, and I was charged $40 twice for order 7731."

print("\na request built from 1.1.0")
print(json.dumps(V2.build(ticket, team="payments", max_words=25), indent=2))

print("\na missing value")
try:
    V2.build(ticket, team="payments")
except KeyError as err:
    print("refused to build, no value for", err)

print("\nthe trap: pasting the ticket into the template text")
for label, render in [("str.format", lambda text: text.format()),
                      ("Template", lambda text: Template(text).substitute())]:
    try:
        render("Summarise this ticket:\n" + ticket)
    except (KeyError, ValueError) as err:
        print(f"{label:<11} {type(err).__name__}: {err}")

print("\nrelease check against the lock file")
candidates = [
    ("unchanged", V2),
    ("wording edited", replace(V2, instructions=V2.instructions.replace("at most", "no more than"))),
    ("model switched", replace(V2, model="example-model-b")),
    ("new version", replace(V2, version="1.2.0", model="example-model-b")),
]
for label, p in candidates:
    locked = LOCK.get((p.name, p.version))
    if locked is None:
        verdict = "new version, needs evaluation before release"
    elif locked == p.fingerprint():
        verdict = "ok"
    else:
        verdict = "FAIL: changed without a new version number"
    print(f"{label:<15} {p.version:<6} {p.fingerprint()}  {verdict}")

registry
name            version  model            fingerprint   variables
ticket_summary  1.0.0    example-model-a  d137461f7cf2  -
ticket_summary  1.1.0    example-model-a  b9c3a39cdbab  team, max_words

what changed from 1.0.0 to 1.1.0
-Summarise the support ticket in one sentence.
+You summarise support tickets for the $team on-call engineer.
+Write one sentence of at most $max_words words: the product and what the customer needs.
+If the ticket describes no problem, reply exactly: NO ISSUE

a request built from 1.1.0
{
  "instructions": "You summarise support tickets for the payments on-call engineer.\nWrite one sentence of at most 25 words: the product and what the customer needs.\nIf the ticket describes no problem, reply exactly: NO ISSUE",
  "input": "Checkout shows {total} instead of the price, and I was charged $40 twice for order 7731.",
  "meta": {
    "prompt": "ticket_summary",
    "version": "1.1.0",
    "fingerprint": "b9c3a39cdbab"
  }
}

a missing value
refused to build, no value for 'max_words'

the trap: pasting the ticket into the template text
str.format  KeyError: 'total'
Template    ValueError: Invalid placeholder in string: line 2, col 64

release check against the lock file
unchanged       1.1.0  b9c3a39cdbab  ok
wording edited  1.1.0  cf35922eeb1e  FAIL: changed without a new version number
model switched  1.1.0  8d4b16183e69  FAIL: changed without a new version number
new version     1.2.0  8d4b16183e69  new version, needs evaluation before release

What the output shows

• Each version has a fingerprint: a short hash of its instructions, model and settings. Version 1.1.0 is b9c3a39cdbab. Logging it with every request records exactly which prompt produced an answer.
• The diff is readable, so a reviewer sees the change in wording the way they would see a change in code.
• The customer's text stays out of the template. The request carries instructions and input separately, and the braces and dollar sign in the ticket pass through untouched.
• A missing value stops the build. Python's template class raises an error rather than sending a half-filled prompt (Python: Template strings). Its safe_substitute method would have quietly sent the placeholder to the model instead. Prefer the error.
• Pasting user text into a template breaks it. With str.format, the {total} in the ticket was read as a placeholder. With the template class, the $40 was rejected as an invalid one. In production, both errors depend on what a customer happens to type.
• The lock file catches silent changes. Editing the wording or switching the model without a new version number fails the check. A real version bump passes, flagged for evaluation.

Keep user content out of instructions

Separating instructions from input is not only about formatting errors. Text from customers, documents and web pages can contain instructions of its own, which is how indirect prompt injection works. OWASP lists keeping untrusted content separate and clearly marked as one of the mitigations (OWASP: prompt injection). Template variables should hold values your code controls, such as a team name or a word limit. Everything else travels as input.

Templating engines raise the same question. A template your developers write is code. A template your users write is untrusted input. Jinja, for example, offers a sandbox for rendering untrusted templates, and its documentation warns that the sandbox alone is not perfect security (Jinja: sandbox).

Version numbers that mean something

A simple convention, borrowed from software releases:

Even a patch is re-evaluated. Models are sensitive to small wording changes, which is the reason for versioning in the first place.

Releasing a prompt change

1. Change it in a branch, with the new version number. The diff goes through review like any code change.
2. Run the evaluation set against old and new versions. A tool such as promptfoo, an open-source command-line tool for evaluating and red-teaming LLM applications, compares outputs across prompts and can run in CI (promptfoo: intro).
3. Block the merge if the lock-file check or the evaluation fails.
4. Roll out gradually behind a feature flag, comparing the new version with the old on live traffic.
5. Log the version and fingerprint with every call, so problems can be traced and rolled back.

Rolling back is then a release of the previous version, not an edit made in a hurry.

Where prompts should live

OpenAI's guidance now favours the first: production prompts in code-managed modules, variables as typed parameters, and changes shipped with git history, review, release tags and feature flags.

Common mistakes

• Versioning the wording but not the model.
• Building prompts by pasting user text into a format string.
• Using a forgiving substitution that sends unfilled placeholders to the model.
• Not logging which version answered, so incidents cannot be traced.
• Editing a live prompt to fix one complaint without running the evaluation set.
• Keeping old versions nowhere, so there is nothing to roll back to.

For techniques that make the prompts themselves better, see prompt engineering techniques. For the controls around prompts in production, see guardrails for LLMs.

Prompt optimization and prompt evaluation are part of the Generative AI course.

Sources

• Prompting (OpenAI API documentation) — OpenAI
• string: Template strings (Python documentation) — Python Software Foundation
• Sandbox (Jinja documentation) — Pallets
• LLM01:2025 Prompt Injection (OWASP Gen AI Security Project) — OWASP
• Intro (promptfoo documentation) — promptfoo