← Research library

Approval queues that don't become bottlenecks: keeping review real and fast

AI-assisted research. This note was prepared with the assistance of AI tools. It may contain errors and is not professional advice. See our Legal Disclosures.

Responsible agent programs in marketing put a person between the agent and the customer, at least at first. That approval step is what makes early deployment safe. It is also the part of the system most likely to fail quietly.

The trouble starts a few weeks in. Agents produce work faster than people can review it, the queue grows, and reviewers start skimming. Oversight that exists on paper but not in practice is worse than none, because it creates false confidence. In our research, we treat the approval queue as a system to design with the same care as the agents feeding it.

Why queues fail

An approval queue is simple: work arrives, a person decides, and work leaves. It fails when arrivals outpace decisions for long enough that people change how they decide. Nobody chooses to rubber-stamp. Reviewers respond to pressure the way anyone would, by spending less time on each item.

Three forces push in that direction. Agent output scales with compute, while review scales with people. Review is often added to someone's existing job rather than given its own time. And most approval tools show a reviewer the output without the context needed to judge it, so careful review takes longer than it should.

The result is a queue that looks healthy by its own measures. Items get approved and the backlog clears. What has changed is that the approvals no longer carry information.

A worked example

Consider a hypothetical team with a social agent that drafts 30 posts a day for one reviewer. In the first week, the reviewer reads each post closely and rejects about one in five. That pace takes most of a morning.

By the fourth week, the agent covers a second product line and drafts 60 posts a day. The reviewer's other work has not shrunk. Review time per post falls by half, the rejection rate drops to almost nothing, and the backlog stays flat.

On a dashboard, this looks like an agent that has improved. It may be. It may also be a reviewer who has stopped checking.

The team cannot tell the difference from approval counts alone. They need the design choices and signals below.

Principles for approval design

The goal is to make each review decision cheap enough that reviewers can afford to make it properly. We recommend four principles.

Review in proportion to risk. Not every output needs the same scrutiny. A customer-facing claim deserves careful reading, while an internal tag change might only need a weekly sample. Sort outputs into two or three review tiers based on consequence, and write the tier into the workflow so it does not depend on a reviewer's judgment in the moment.

Make the decision easy to make. Reviewers should see exactly what is changing, why the agent proposed it, and what sources it used. A side-by-side view of current and proposed state saves time on every item. If a reviewer has to open three systems to check one claim, the claim will eventually go unchecked.

Batch similar items. Reviewing 20 similar social posts in a row is faster and more consistent than reviewing them scattered across the day. Batching also makes repetition visible, which single-item review hides.

Set a service level. Decide how quickly approvals must happen for each tier, and measure it. If items routinely wait longer than the target, either add reviewers or reduce volume. Lowering the standard should not be one of the options.

Automated checks help here too. Banned terms, missing disclosures, and spending limits can be checked before an item reaches the queue, as we describe in guardrails as code. Each mechanical check removed from human review frees attention for the judgment calls only a person can make.

Signals your queue is failing

Approval data carries its own early warnings. We suggest reviewing these five signals weekly from the queue's logs.

  • Near-total approval: approval rates near 100% alongside short review times.
  • Growing backlog: the queue is longer at the end of each week than at the start.
  • Out-of-area approvals: reviewers approving items outside their expertise to keep work moving.
  • Unchanged resubmissions: rejected items that come back without the issue fixed.
  • Downstream catches: errors found after publication in items that passed review.

Any one of these suggests that review has stopped working as a real check. The last signal matters most, because it measures the queue against what reached customers. We pair it with the human edit rate, described in measuring human edit rate, since a falling edit rate can mean better drafts or tired reviewers.

Use rejections as training data

Every rejection says something about the agent. Capture a short reason chosen from a fixed list, with an optional comment. Here is a list we suggest starting with.

  • Factual error: a claim is wrong or unsupported.
  • Off brand: tone or wording does not fit the voice guide.
  • Wrong audience: the content does not suit the segment or channel.
  • Compliance issue: a required element is missing or a rule is broken.
  • Other: anything else, with a required comment.

Keep the list short so reviewers use it. Over time, the reasons show you what to fix in the agent's instructions, context, or tools. A cluster of factual errors points to retrieval. A cluster of off-brand rejections points to the voice specification.

We expect teams that review these reasons monthly to see the same mistakes repeat less often. That loop is the main way an approval queue makes agents better rather than only catching them.

Earn your way to sampling

As an agent proves itself, move from reviewing everything to reviewing a sample. One path we suggest: review all outputs, then half, then a fixed random sample plus anything the agent flags as uncertain. Each step frees reviewer time for the work that still needs it.

Treat each step as a decision backed by evidence. Write down the rejection rate, the downstream error count, and the period they cover. Agree in advance on the condition that would send the agent back to full review. This mirrors how agents move up the autonomy ladder, where approve to act is one rung and act and notify is the next.

Sampling should stay random. A reviewer who picks which items to check will pick the ones that look interesting, which are rarely the ones that fail. Let the system choose the sample, and have it add every item the agent marked as low confidence.

What to measure

Three measures tell you whether the queue is doing its job. Time to decision, per tier, shows whether you are meeting the service level. Rejection rate by reason shows what the agent gets wrong. Downstream error rate shows whether review catches what it should.

Read them together. A falling rejection rate with a flat downstream error rate is good news. A falling rejection rate with rising downstream errors means review has weakened. The failure budget for each agent gives you the threshold that should trigger a change.

We are still studying how to keep reviewers calibrated over time. Two reviewers given the same item can reach different decisions, and that spread is hard to see in normal operation. Whether periodic calibration sets, where every reviewer judges the same known items, can keep a team consistent at high volume is the next question we plan to test.

How to apply this: sort your agent outputs into two or three review tiers by consequence, and set a response time for each. Add a structured rejection reason to every approval tool, and review the reasons as a team each month.

Related research

Put this research to work

Marketix turns these patterns into software: agents, automated workflows, and a hub that keeps people in control. Sign up for early access.