How we run a research sprint on a new agent pattern
AI-assisted research. This note was prepared with the assistance of AI tools. It may contain errors and is not professional advice. See our Legal Disclosures.
New agent patterns appear faster than anyone can adopt them, and most arrive with a demo rather than evidence. Our job as the research team behind Marketix is to find out which ones hold up in real marketing work, and we do that through small, structured experiments.
This post describes the two-week research sprint format we use to test a new pattern. We share it as a method, so teams can adapt it for their own questions about agents. It describes how we work, not what any particular sprint found.
Day 1: frame the question
Every sprint starts with a single question, written as a sentence that evidence can answer. The question decides everything that follows: what we build, which cases we test, and what counts as a result. A vague question produces a prototype that proves nothing.
Good questions are narrow and name what would be measured. These are the kinds we aim for.
- "Can a planner agent produce campaign plans that reviewers approve with minor changes?"
- "Does retrieving product facts on demand reduce factual errors compared with preloading them?"
- "Can a checker agent catch claims that our brand rules prohibit before a human reviewer sees the draft?"
A bad question is broad, such as "Can agents run marketing?" It has no clear test and no clear failure.
Before any work starts, we also write down what result would change our minds in each direction. If the planner's plans need heavy edits, we stop. If they need only light edits on most cases, we continue. Writing this first keeps us from reading whatever happens as a success.
Days 2 to 3: map the workflow
Next we document the real workflow the pattern would support. That means its inputs, outputs, the people involved, and the points where someone makes a decision. We map the workflow as it runs today, including the workarounds, rather than as the process document describes it.
This step can end a sprint early, and we treat that as a useful outcome. If the workflow changes every week, or the data the agent would need does not exist, there is nothing stable to build on. Stopping on day three costs far less than discovering the same problem on day ten.
The map also tells us where the risk sits. A step that touches customers or budget gets more scrutiny in evaluation than a step that only produces an internal draft. Our impact and risk matrix is the tool we use to sort these steps.
Days 4 to 7: build the smallest useful prototype
We build only what the question needs. The goal is evidence, not a product, so anything that does not help answer the question waits. In practice, the prototype follows a few rules.
- Keep it small: one agent, or the fewest agents the pattern requires.
- Prefer read-only tools: the agent proposes actions rather than taking them wherever possible.
- Log everything: record every input, decision, and output so we can trace results later.
- Skip the polish: build no interface beyond what reviewers need to score outputs.
The temptation to build more is constant. A feature that would make the demo nicer adds time and adds variables, and each new variable makes the result harder to interpret. We hold the line because a smaller prototype gives a cleaner answer.
Read-only tools deserve a note. Most of the questions we test are about quality of judgment, and a proposal can be judged as well as an action. Keeping the agent read-only lets us run messy cases without any risk to real systems.
Days 8 to 11: evaluate
We run the prototype against a prepared set of realistic cases. The set includes messy ones on purpose: incomplete briefs, conflicting facts, and requests that should be refused. A pattern that only works on clean inputs will not survive real work.
Reviewers score each output with a rubric written before they see any results. We record edit time where the task involves editing, because the minutes a reviewer spends fixing a draft are a direct measure of what the agent saves. Our posts on a starter rubric for content agents and on measuring human edit rate describe both tools.
Where we can, we compare against a baseline. That might be the current manual process or a simpler automation that already exists. Without a baseline, a good score tells us the agent is competent but not whether it is worth the added complexity.
Days 12 to 13: analyze and write up
We write a short internal note for every sprint, in a fixed order. The fixed structure makes notes comparable across sprints and keeps the conclusion separate from the evidence.
- The question, exactly as written on day one.
- What we built, including what we chose not to build.
- What happened, with specific examples of good and poor outputs.
- What we think it means, and how confident we are.
- What we would test next.
Negative and inconclusive results get the same care as positive ones. A clear account of why a pattern failed is often more useful than a positive result, because it tells others where not to spend time. We write the confidence section plainly, including what the sample was too small to show.
Day 14: decide
Each sprint ends with one of three decisions. We make the decision on the last day, even when the evidence is mixed, because an open-ended sprint tends to drift into a project.
- Continue: run a follow-up sprint on a sharper version of the question.
- Build: move the pattern into the Marketix product, sometimes with a partner team that can use it.
- Stop: record what we learned and close the question for now.
A pattern that holds up over several sprints becomes a candidate for our published frameworks. One that stops may come back later if the tools or the data change. The full cycle is shown in our research loop diagram.
Adapting the format for your team
The format works for any team with one question and two weeks. You may not need every stage at full length. A team testing a small change to an existing agent might compress the workflow map into an afternoon, and spend the saved days on evaluation.
The parts we would keep regardless are the written question, the decision rule set in advance, and the write-up. Those three keep a sprint honest when the results are not what anyone hoped.
What we are studying next is how to run sprints on patterns that involve several agents at once. Handoffs between agents add cases that a single-agent test set does not cover, and we are still working out how to build evaluation sets that capture them.
How to apply this: pick one question about agents that matters to your team, write it as a testable sentence, and set your decision rule before you build. Give yourself two weeks, build the smallest thing that can answer it, and write up the result whatever it is.
Related research
Field notes, April 2026: agentic marketing moves from demos to workflows
Our April notes on agentic marketing: why we expect the hard questions to shift from what agents can do to whether teams can trust, measure, and run them.
Field notes, May 2026: protocols, evaluations, and the rise of the agent operations role
Our May notes on agentic marketing: why we expect a role to form around running agents day to day, and why permissions now matter more than connections.
From retainers to systems: replacing your agency with AI
Why the agency retainer model is breaking, and how to move to an AI-run marketing system you own.
Put this research to work
Marketix turns these patterns into software: agents, automated workflows, and a hub that keeps people in control. Sign up for early access.
