← Research library

Prompt libraries are not strategy: versioning agent instructions

AI-assisted research. This note was prepared with the assistance of AI tools. It may contain errors and is not professional advice. See our Legal Disclosures.

As agent programs grow, the instructions that steer them become some of the most consequential text a marketing team owns. Yet they are often managed like scratch notes. Many teams start with a prompt library: a shared document of instructions that worked well once.

Over time the library grows, copies drift apart, and nobody remembers which version is live or why a particular sentence was added. When quality changes, there is no way to tell whether an edit caused it.

Agent instructions are operating logic for systems that reach customers, and they deserve the same care as any other part of those systems. This post covers what to version, a lightweight process, how to test changes, and how to catch the changes nobody made on purpose.

Why a library is not enough

A prompt library answers the question "what has worked?" A versioned set of instructions answers different questions: what is live now, what changed, why, and what happened after. Those are the questions you need when quality shifts.

Libraries also encourage copying. A team member takes a good prompt, adapts it for a new agent, and the two versions start to diverge. Months later, a fix applied to one never reaches the other, and nobody can say which is canonical.

The library has a place as a source of ideas. It should not be where live agents get their instructions.

What to version

An agent's behavior depends on more than its prompt. For each agent, keep track of four things together:

  • Instructions: the role, goal, rules, and format guidance.
  • Reference material: voice specifications, approved facts, and examples.
  • Tool configuration: which tools are available and their descriptions.
  • Model and settings: which model, and any parameters that affect output.

A change to any of these can change behavior, so record them as one unit. A version number should identify the whole set. That way, "version 14" means the same thing to everyone, and you can compare any two versions in full.

Reference material is the item most often left out. A brand voice specification edited by another team changes every agent that loads it, even though no one touched those agents' prompts.

A lightweight process

You do not need complex tooling to start. A simple process covers most of the value:

  1. Store instructions in one place, with a version number or date.
  2. Write a short change note for every edit, stating what changed and why.
  3. Test before releasing by running the new version against a small test set and comparing it with the current one.
  4. Record which version produced each run, so outputs can be traced to the instructions behind them.
  5. Keep the ability to roll back to the previous version quickly.

Teams with engineering support often store instructions in the same version control system as code. Teams without it can use a shared document with strict change notes and a single owner per agent. The discipline matters more than the tool.

Step four is the one teams skip, and it is the one that makes the others useful. Without a version stamp on each run, you cannot connect a quality change to an instruction change. Run histories that record versions are covered in observability for marketing agents.

Why "it worked once" is not enough

A prompt that produced a great result on one example may perform poorly across a hundred. Single examples hide trade-offs: a change that fixes tone on one input can break format on another.

Before promoting any instruction change, test it on a representative set of cases, including difficult ones. We describe how to build these sets in golden datasets.

Consider a hypothetical team that adds a sentence to its email agent: "Always open with the customer's first name." On the three examples the editor tried, the drafts improve. On a 40-case test set, the change also fires when the name field is empty or holds a company name, producing awkward openers in several cases. The test catches it before release, and the change note records why the sentence was revised to apply only when a verified first name is present.

Watch for silent changes

Some changes happen without anyone editing instructions. Each of these can shift behavior with no change note:

  • Model updates: a provider updates or retires a model the agent uses.
  • Reference edits: another team changes a document the agent loads.
  • Tool changes: a tool's output format or description changes.
  • Input shifts: the data the agent receives starts to look different.

Monitoring quality over time, and recording the versions of everything an agent depends on, helps catch these. When quality moves and no instruction changed, check the other three items first. A test set can also go stale as inputs shift, which is the problem described in evaluation drift.

Instructions as institutional knowledge

Well-maintained instructions, with their change history, become a record of what your team has learned about doing the work well. Every rule exists because something went wrong or something worked. That history is valuable, and it disappears if instructions live only in scattered copies.

Change notes are what preserve the reasoning. A rule without a note will eventually look arbitrary to someone, and they will delete it. A rule with a note saying which failure it prevents is much more likely to survive, or to be replaced deliberately.

Instruction versions also belong in governance. A periodic review of what changed, who approved it, and what the tests showed fits naturally into a quarterly policy review.

The question we are studying next is how to version instructions shared across many agents. A single voice rule may feed ten agents, and a change to it needs testing against all of them. We expect shared instruction modules to need their own owners, test sets, and release notes.

How to apply this: pick your most important agent and put its instructions, reference material, tool list, and model settings in one versioned place. From now on, every change gets a note and a test before it goes live, and every run records the version that produced it.

Related research

Put this research to work

Marketix turns these patterns into software: agents, automated workflows, and a hub that keeps people in control. Sign up for early access.