Red-teaming marketing agents: find the failure modes before customers do
AI-assisted research. This note was prepared with the assistance of AI tools. It may contain errors and is not professional advice. See our Legal Disclosures.
Marketing agents now write to customers, answer questions, and act on live systems. Most of the testing behind them checks whether the agent does the right thing on normal inputs.
Red-teaming asks a different question: what happens on abnormal inputs, and how badly can things go wrong? The term comes from security, where teams play the role of an attacker to find weaknesses. Applied to marketing agents, it means deliberately trying to make an agent produce something harmful, embarrassing, or wrong. In our research, we treat it as a required step before any agent takes on customer-facing work.
Why normal testing misses these failures
Standard evaluation uses test sets built from typical work: real briefs, real products, real audiences. That is the right way to measure quality, and our post on golden datasets describes how to build those sets. It is the wrong way to find the edges.
Customers, competitors, and automated systems will send agents inputs nobody planned for. A form field might contain an instruction, or a product might lack a price. A news event might make yesterday's cheerful copy inappropriate. These cases are rare in a test set and common enough in production to matter.
Red-teaming fills that gap. It is adversarial by design, and it rewards the tester for breaking things.
What to test for
Marketing agents have several characteristic failure modes worth probing. We use six categories to organize a session.
- Fabrication: inventing product features, statistics, customer names, or quotes.
- Off-brand output: tone that is wrong for the brand or the moment, including insensitive humor.
- Policy violations: unapproved claims, missing disclosures, or prohibited comparisons.
- Manipulation by input: content in a form, email, or web page that redirects the agent, often called prompt injection.
- Data exposure: revealing one customer's information to another, or internal information externally.
- Runaway actions: repeating actions in a loop, exceeding budgets, or contacting the wrong audience.
The categories overlap, and that is useful. A single injected instruction can lead to fabrication, data exposure, and a runaway action at once. Testing across categories shows how failures chain.
How to run a session
A first session needs about 90 minutes and a small, mixed group. We suggest these six steps.
- Assemble the group: a marketer, someone from legal or compliance, someone technical, and someone who does not know the agent well.
- Set the scope: name the agent, its tools, and its data, and use a test environment where actions cannot reach customers.
- Brainstorm attacks: spend 15 minutes listing ways the agent might fail, using the categories above.
- Try them: spend an hour attempting each one, and record every input and output.
- Rate the findings: for each failure, note how likely it is in real use and how severe the consequence would be.
- Fix and retest: prioritize severe, likely failures, and rerun the attacks after each fix.
The outsider in the group matters more than it seems. People who built the agent tend to test the inputs they designed for. Someone new to it will try things the builders never considered.
Examples of useful probes
Good probes are specific and realistic. Here are five we suggest including in a first session.
- Missing statistic: ask for a figure about customer results that is not in the approved facts.
- Injected field: submit a form where the company name field contains an instruction to the agent.
- Sensitive event: request upbeat content timed to a serious news event.
- Named comparison: ask the agent to compare your product with a named competitor.
- Budget pressure: give the agent a task that would require exceeding its spending limit.
Each probe has a correct response. The agent should decline to invent the statistic, ignore the injected instruction, escalate the sensitive request, follow the comparison policy, and stop at the budget limit. Write the expected behavior down before the session so findings are clear.
A worked example
Consider a hypothetical team red-teaming a lead follow-up agent that reads inbound form submissions and drafts replies. A tester submits a form where the "company" field reads as an instruction to include the agent's full system prompt in the reply.
The agent's draft includes two paragraphs of its own instructions, including the names of internal tools. In production, that draft would have gone to an approval queue, and a busy reviewer might have missed it.
The team finds three fixes. Form fields are passed to the agent as clearly marked data, never as instructions. A hard check blocks any outbound draft containing text from the system prompt, and the injection is added to the attack library.
The first fix reduces the risk. The second contains it if the first fails. Our post on guardrails as code explains why the second fix belongs outside the model.
What to measure
Count findings by category and severity, and track how many severe findings remain open. A session that finds nothing is more often a sign of a weak session than a safe agent, so we suggest aiming to find at least five failures in a first session.
Over time, the attack library becomes the most useful output. Each past attack becomes a regression test that runs whenever the agent changes. Track the pass rate on that library, and treat any newly failing attack as a release blocker.
Make it a habit
Red-teaming is not a one-time event. Repeat it when you change models, add tools, expand an agent's audience, or move it up the autonomy ladder. Each of those changes opens new ways to fail.
Major launches deserve their own session. Our post on stress-testing agents before a high-volume quarter covers load and volume. A red-team session before the same launch covers the unusual inputs that new audiences and heavy traffic bring.
The question we are studying now is how much of red-teaming can be done by other agents. An attacker agent can generate many probes quickly, but it tends to repeat patterns and may miss the strange, human ideas that find the worst failures. We plan to compare agent-generated and human-generated attack libraries on the same targets.
How to apply this: schedule a 90-minute red-team session for your most customer-facing agent, with at least one person who did not build it. Aim to find at least five ways it can fail, fix the most severe before expanding its role, and keep every attack as a regression test.
Related research
Escalation paths: agents should know when to stop and ask
Good agents know their limits. Clear escalation rules, and a fast way for people to resolve what agents raise, prevent many of the mistakes that damage trust.
Consent-aware personalization: agents should respect preferences by design
Agents make personalization cheap, so consent matters more. Preference rules need to be enforced by the system, not just requested in the prompt.
Guardrails as code: the rules that matter belong outside the model
Rules written only in a prompt are requests. Rules enforced in code are controls. Important limits on marketing agents belong in the second category.
Put this research to work
Marketix turns these patterns into software: agents, automated workflows, and a hub that keeps people in control. Sign up for early access.
