Retrieval for marketing agents: getting the right facts at the right time
AI-assisted research. This note was prepared with the assistance of AI tools. It may contain errors and is not professional advice. See our Legal Disclosures.
Marketing agents now write product pages, answer pricing questions, and draft sales follow-ups, and each of those tasks depends on facts the model was never trained on. How those facts reach the agent decides whether the output is accurate.
The simplest approach is to paste everything into the agent's instructions. It works while the knowledge is small. As product lines, policies, and regional rules grow, the instructions balloon, cost rises with each call, and the agent loses track of which facts matter for the task in front of it.
Retrieval is the alternative. The agent looks up what it needs at the moment it needs it, and works only from what it found.
How retrieval works
The basic loop has four steps, and each one is a place where quality can be lost.
- The agent, or the system around it, forms a query from the task, such as "integration capabilities of Product X."
- A search runs against one or more knowledge sources, with filters applied.
- The top results, usually a handful of short passages, are passed into the agent's context.
- The agent writes from those results and cites the entry behind each factual claim.
Search can match on keywords, on meaning (often called semantic or vector search), or on both. For marketing content we recommend starting with a hybrid.
Product names, SKUs, and plan tiers need exact matching, and embeddings can blur two products with similar descriptions. General questions, such as how a feature helps a small team, benefit from meaning-based matching. When the candidate set is large, a reranking step on the combined results is usually worth its cost.
A worked example
Consider a hypothetical software company with three plans and a knowledge base of a few hundred entries. A content agent is asked to write a comparison page and needs to know which plans include single sign-on.
Without metadata filters, the search returns four passages: the current plan matrix, last year's matrix, a support article about a beta, and a sales deck note written before a pricing change. All four are semantically relevant. Two are wrong today. The agent, seeing conflicting evidence, picks one, and the page goes to review with a claim that a person has to trace by hand.
With status and date filters, the search returns only the current matrix and the current support article. With a citation requirement, the reviewer sees exactly which entry supports the claim and can check it quickly. Nothing about the model changed. The fix was entirely in the sources and the query.
What makes retrieval work well
Five properties separate retrieval that helps from retrieval that adds noise.
Well-organized sources. Retrieval is only as good as what it searches. Short, focused, current entries retrieve far better than long documents that mix topics, because a passage that covers one fact can be matched and cited cleanly. See building a marketing knowledge base agents can trust.
Metadata and filters. Tag entries with product, audience, region, status, and effective date. A filter such as "status is current and region includes Europe" removes whole classes of error before ranking begins.
Specific queries. An agent left to itself may search with the requester's words verbatim, or with something vaguer. Instructions that show how to form queries, with examples, help. So does letting the system expand one task into two or three targeted queries instead of one broad one.
Citations. Requiring an entry ID for each claim makes verification possible, whether by a checker agent or a person. It also makes unsupported claims visible at a glance.
An honest fallback. When retrieval finds nothing relevant, the agent should say so and stop rather than fill the gap from general knowledge. For product facts, a flagged gap is far cheaper than a confident guess.
Failure modes and how to spot them
Retrieval failures are quiet. The agent produces fluent text either way, so you find them by inspecting what was retrieved alongside what was written. These are the failures we look for first:
- Stale entries: outdated content ranks alongside current content. Log the effective date of each retrieved entry and alert when a retired entry appears.
- Conflicting near duplicates: two entries state the same fact differently. Flag retrievals where top results disagree on a value, then consolidate to one source of truth.
- Context crowding: too many results dilute the useful ones and raise cost. Return fewer, better passages, as we argue in context windows are a budget.
- Chunking errors: a fact split across two passages is retrieved in halves. Watch for reviews where the cited passage supports only part of the claim.
- Instruction injection: searchable content contains text that reads as an instruction to the agent. Keep agent-facing sources curated, treat retrieved text as data, and keep unreviewed third-party content out of the same index.
Measuring retrieval separately
We test retrieval on its own before judging the agent that uses it. Mixing the two hides the cause, since a wrong claim might come from a bad search, a good search that found a stale entry, or a model that ignored a correct passage.
Build a small labeled set of queries, each paired with the entry or entries that should come back. Then measure four things:
- Recall at k: how often the correct entry appears in the top k results, with k set to the number of passages you pass the agent.
- Context precision: how much of what you pass is relevant, since irrelevant passages cost tokens and invite errors.
- Staleness rate: how often a retired or superseded entry appears in the top results.
- Citation faithfulness: whether each cited entry in the agent's output supports the claim attached to it.
The labeled queries belong with your other test cases, under the same review discipline as your golden datasets. Rerun them whenever you change chunking, embeddings, filters, or the structure of the knowledge base. We expect improvements to retrieval to move overall quality more than most instruction edits do, and this set is how you check that claim for your own agent.
What we are studying next
The current pattern treats retrieval as a step that runs before writing. We are testing designs where the agent can retrieve again mid-task when it notices a gap, and where results carry a confidence signal the agent can reason about.
The open question is how to give agents that freedom without returning to the original problem: context that grows until the important facts get lost in it.
How to apply this: take ten factual questions your content agent needs to answer often and check whether retrieval returns the correct, current entry in its top three results. Fix the sources and filters before you tune the agent.
Related research
The marketing data contract: schemas agents can depend on
When upstream data changes without warning, agents break quietly. A data contract defines what agents can rely on and who is responsible for keeping it true.
Load, latency, and rate limits: scaling agents for traffic spikes
Agents that run smoothly in normal weeks can stall when volume spikes. Planning for load, latency, and rate limits keeps them running when it matters.
Real-time signals help only when the response fits the signal
Agents can respond to buying signals in minutes instead of days. That speed helps only if the signals are meaningful and the responses are appropriate.
Put this research to work
Marketix turns these patterns into software: agents, automated workflows, and a hub that keeps people in control. Sign up for early access.
