Synthetic audiences are a thinking tool, not an evidence tool
AI-assisted research. This note was prepared with the assistance of AI tools. It may contain errors and is not professional advice. See our Legal Disclosures.
Language models can now play the part of a buyer convincingly, and that has opened a new kind of marketing research. Synthetic audiences are AI-generated personas that react to messaging, rate concepts, or answer survey questions as if they belonged to a target segment.
The appeal is plain. Real research is slow and expensive, while synthetic research takes minutes and costs little. The question this post addresses is what synthetic audiences are actually good for, and where they quietly mislead.
Our position
Synthetic audiences are a useful thinking tool and a poor evidence tool. They belong at the start of a research process, never at the end.
One rule follows from that position, and we treat it as non-negotiable. Synthetic data is always labeled as synthetic, and it is never presented as real customer research. That applies in internal decks, in reports to leadership, in sales enablement, and in anything published. A simulated persona's opinion is a hypothesis about customers, not a finding about them.
How synthetic audiences work
A synthetic audience starts with a persona description, such as a mid-market IT director evaluating security software. A model is asked to respond as that person would: react to a headline, rank three value propositions, or answer an interview guide.
The output reads like research. It has the tone of a real buyer and often names plausible concerns. That fluency is the source of both the value and the risk.
The model is producing a plausible answer drawn from patterns in its training data. It has no memory of buying anything, no budget, and no knowledge of your product beyond what you give it.
Where synthetic audiences can help
Used early and labeled clearly, synthetic personas can save time in four ways.
- Stress-testing early ideas: asking personas to challenge a message can surface objections you had not considered.
- Preparing real research: synthetic responses can help draft sharper survey questions, interview guides, and hypotheses before you spend money on fieldwork.
- Checking clarity: a simulated reader can flag jargon, confusing structure, or missing information in a draft.
- Widening perspective: asking how a finance lead, an engineer, and a procurement manager might each view a proposal can push a team past its own assumptions.
Each of these uses treats the output as input to thinking. Checking clarity, in particular, is closer to editing than to research. None of these uses requires the persona to be right about real buyers.
Where they mislead
The same properties that make synthetic personas fast also make them unreliable as evidence. We see four main failure modes.
They reflect patterns, not people. Synthetic personas generate responses based on what text about similar people tends to say. They do not have your customers' actual experiences, constraints, or context, such as a contract renewal date or a failed rollout with a competitor.
They tend toward the expected. Simulated responses often echo conventional views of a role. Real research is valuable partly because customers surprise you, and a persona built from averages rarely does.
They can confirm bias. The way you describe a persona strongly shapes its answers. It is easy, often unintentionally, to write a persona that tells you what you hoped to hear. Rewording the description and rerunning the prompt is a useful check.
They have no stakes. A persona saying it would request a demo tells you nothing about whether a real buyer will spend real time or money. Stated intent is weak evidence even from real people, and weaker still from simulated ones.
A worked example
Consider a hypothetical B2B analytics company preparing to reposition a product for operations leaders. The team writes five personas and asks each to react to three messaging options.
The personas favor option B, which leads with cost savings. The team does not take that as a finding. It labels the output "Synthetic: for hypothesis generation only" and writes a hypothesis: operations leaders respond more to cost savings than to speed.
Next, the team uses the persona objections to write a short interview guide. It runs interviews with eight real operations leaders. Suppose those interviews show that speed matters more than expected, because several buyers are under pressure to close reporting gaps quickly.
The synthetic step was still useful. It produced the hypothesis and the interview guide in an afternoon. The evidence came from the interviews, and only the interview results go into the positioning document.
Guidelines for responsible use
We suggest writing these rules into your team's research process, so they apply to every project rather than depending on individual judgment.
- Label everywhere: mark every synthetic output as simulated, in the file name, the document, and every slide that cites it.
- Keep it out of evidence: never present synthetic results as customer research to leadership, sales, or in published material.
- Pair with a test: every synthetic insight that could influence a decision gets a plan to test it with real customers.
- Calibrate over time: when real results arrive, compare them with what the personas predicted, and record how far apart they were.
- Take extra care with unfamiliar groups: be cautious with personas for segments the team does not know well, where bias is hardest to spot.
Labeling also matters when synthetic material reaches the outside world. Our post on disclosure practices for AI-generated content covers how to handle that. Confirm any public disclosure requirements with counsel.
What to measure
If your team uses synthetic audiences regularly, we suggest tracking how well they predict real outcomes. For each project, record the main synthetic prediction and the real result. Over several projects, that log tells you which uses the method serves and which it does not.
Track how often synthetic hypotheses are tested at all. A rising share of untested synthetic insights is a warning sign. It suggests the method is drifting from thinking tool toward substitute evidence.
Where a synthetic hypothesis leads to a real campaign change, measure the change with a proper comparison group. Our post on incrementality testing covers how. Persona feedback can also help red-team marketing agents, as long as the results are treated as test cases rather than research.
What we are studying next
We are now studying where synthetic audiences fit inside a structured research loop, and whether a persona's disagreement is more informative than its agreement. Our working hypothesis is that objections from synthetic personas are a better guide to interview questions than their preferences. We expect that to hold for some segments and fail for others, and we are testing it against real interviews.
How to apply this: if your team uses synthetic personas, add a written rule that every synthetic output is labeled as simulated and never presented as customer research. Pair each synthetic insight with a plan to test it with real customers before it influences a decision.
Related research
Shadow mode: run agents alongside humans before going live
The safest way to learn how an agent will perform on real work is to let it do the work in parallel, without its output reaching anyone. That is shadow mode.
Golden datasets: building test sets for marketing agent evaluation
A small set of test cases with agreed good answers is the most useful evaluation tool a marketing team can build. It can be built in a week.
Cost per task: budgeting model spend for marketing agents
Model costs are easy to ignore during a pilot and hard to ignore at scale. Measuring cost per completed task gives finance and marketing a shared language.
Put this research to work
Marketix turns these patterns into software: agents, automated workflows, and a hub that keeps people in control. Sign up for early access.
