Field notes, May 2026: protocols, evaluations, and the rise of the agent operations role
AI-assisted research. This note was prepared with the assistance of AI tools. It may contain errors and is not professional advice. See our Legal Disclosures.
Agentic marketing in May is starting to look like an operations problem. As agents become easier to connect and deploy, the questions we keep returning to are about who runs them, how their quality is tracked, and what they are allowed to touch.
These are our monthly notes: themes we are tracking, questions we are studying, and what we expect. They are working hypotheses from our research and prototypes, not measurements of the market, and we hold them loosely.
A role is forming around running agents
The theme we find most interesting this month is organizational rather than technical. Once an agent moves past a pilot, someone has to be responsible for its day-to-day performance. That person reviews logs, triages failures, updates instructions, tracks quality, and decides when the agent can take on more.
We expect this work to settle into a recognizable function, which we informally call agent operations. The title will vary. It might sit in marketing operations, in a growth team, or with a technical marketer. What we expect to stay constant is the set of responsibilities.
The skills it combines rarely sit together today. The role needs process design, data literacy, editorial judgment, and enough comfort with technical systems to read a run history without help. A person who has only one of these tends to fix the symptom they can see. A person with all four can trace a weak draft back to a stale fact or a vague brief.
Consider a hypothetical team running three agents: one for research briefs, one for first drafts, and one for weekly reporting. Without an owner, each agent is checked only when someone notices a problem. With an owner who spends a few hours a week on them, failures get logged, patterns get named, and instructions change for a reason. The agents are identical, and the difference is whether anyone is learning from them.
What the agent operations job contains
The responsibilities we expect this role to hold are concrete enough to list. Teams that want to test the idea can use them as a draft job description.
- Review the run history: read a sample of agent runs each week, including the ones that did not fail.
- Triage failures: sort problems into bad inputs, weak instructions, tool errors, and model limits, since each has a different fix.
- Own the instructions: keep agent instructions versioned, with a note on why each change was made.
- Track quality: maintain a small set of measures, such as edit rate and rejection rate, and watch their direction.
- Recommend autonomy changes: propose when an agent should gain or lose independence, with evidence attached.
The last item matters most to us. Moving an agent up or down the autonomy ladder is a decision, and decisions need an owner. Without one, autonomy tends to drift upward by habit rather than by evidence.
Evaluation moves from ad hoc to routine
Last month we described evaluation as the likely bottleneck of the year. This month the theme we are watching is how evaluation becomes a habit rather than an event. The practices involved are small: a saved set of test cases, a rubric that says what good looks like, and a record of how much reviewers change agent output.
None of these are glamorous. We expect them to mark the difference between teams that improve their agents steadily and teams that change things and hope. A rubric turns a vague sense that drafts got worse into a specific claim someone can check.
The measure we think is most underrated is human edit rate. It costs almost nothing to collect, because reviewers are already editing. It also tracks what a team cares about: how much work the agent saves. We described how to set it up in measuring human edit rate.
A pattern worth testing is to run the saved test cases every time an instruction changes, before the change goes live. If the agent operations role exists, this becomes part of its weekly routine. If it does not, we expect the step to be skipped under deadline pressure.
Connection is easy, permission is hard
As standard tool protocols spread, connecting an agent to a system is becoming routine work. We covered what these protocols change for marketing teams in MCP for marketers. Our working view is that integration is turning into plumbing, and the harder conversations are moving to permissions.
The questions that follow are practical. Which actions should an agent be able to take in the CRM? Should an agent built into a vendor's platform be allowed to modify live campaigns? Who approves a change to those permissions, and where is that approval recorded?
We expect teams that answer these questions early to move faster later. The reason is that permissions are cheap to set at the start and costly to retract after an agent has been running for months. People come to rely on what an agent does, even when it was never meant to do it.
The design choice we keep coming back to is narrow tools. An agent that can call "update contact field" with a short list of allowed fields is easier to reason about than one with general write access. Our post on tool design for agents argues for fewer, clearer, safer actions, and we think permission reviews get simpler when the tools themselves are small.
Questions we keep returning to
Several open questions came up repeatedly in our research this month. We do not have settled answers yet, and we list them so readers can test them alongside us.
- Rejected work: what does a good handoff look like when a human reviewer rejects an agent's output, and how should the reason reach the agent's instructions?
- Uneven costs: how should teams budget for model spend when usage grows unevenly across agents?
- Pipeline fit: which content types benefit from a multi-agent pipeline, and which are better served by one agent with a good brief?
- Role placement: does agent operations work best inside marketing, or closer to the teams that run the underlying systems?
Our current leaning on the last question is inside marketing. The judgment calls in the role are mostly editorial and strategic, and they need someone who knows what a good campaign looks like. That is a starting view, not a finding.
What we are watching next month
June is the midpoint of the year, and we expect it to bring a round of reviews. These are the signals we plan to watch.
- Mid-year reviews: whether programs can show measured results from their first half of agent work, or only describe them.
- Summer planning: how summer campaign plans assign work to agents and how review is staffed.
- Comparison questions: how answer engines handle product comparison questions, and what that means for content planning.
The open question we carry into June is whether the agent operations role will be created on purpose or discovered by accident. If it is created on purpose, we expect it to come with time, tools, and authority. If it is discovered, we expect it to arrive as extra work for whoever happens to care, and to burn out quickly.
How to apply this: name one person as the owner of each agent you run, and give them a weekly hour to review its runs. Then list every action each agent can take, and confirm someone approved each one.
Related research
How we run a research sprint on a new agent pattern
A look inside our two-week research sprint format: how we frame a question, build the smallest useful prototype, and decide what we learned.
Field notes, April 2026: agentic marketing moves from demos to workflows
Our April notes on agentic marketing: why we expect the hard questions to shift from what agents can do to whether teams can trust, measure, and run them.
Field notes, June 2026: halfway through the year of orchestration
Our June notes on agentic marketing: the midyear move from single agents to coordinated systems, and the operating discipline that move requires.
Put this research to work
Marketix turns these patterns into software: agents, automated workflows, and a hub that keeps people in control. Sign up for early access.
