← Research library

Self-critique loops improve drafts, but review stays independent

AI-assisted research. This note was prepared with the assistance of AI tools. It may contain errors and is not professional advice. See our Legal Disclosures.

As content agents move from experiments into production pipelines, teams look for cheap ways to raise quality before a person ever sees a draft. The self-critique loop is the most common: the agent produces a draft, reviews it against criteria, identifies problems, and revises.

It mirrors what good writers do, and it often helps. It also has limits that are easy to miss, because a loop that runs without errors looks like a loop that works.

This post covers where self-critique helps, where it falls short, how to design a loop that earns its cost, and how to measure whether it does.

When self-critique helps

Self-review works best when the review step can verify something concrete. In our research, three kinds of check suit it well.

Checkable criteria. Examples include a length limit of 150 words, a required disclosure, the correct event date, and product claims that match the fact sheet. The agent can check these reliably, especially when the criteria and the source facts are in front of it.

Structural improvements. Agents are reasonably good at noticing that a draft lacks a clear call to action, buries the main point, or repeats itself. These are visible on the page and do not require knowledge the agent lacked when writing.

Format compliance. Checking that output matches a required structure, such as all fields present and the right types, suits self-review. It is close to mechanical, which is also why some of it belongs in code.

The common thread is that the answer is in the draft or in material provided alongside it. The review step is reading, not recalling.

Where it falls short

Self-critique struggles when the problem is something the agent could not see the first time. Four limits recur.

Shared blind spots. If the agent did not know a fact was wrong when writing, it often will not notice when reviewing. The same knowledge gap affects both steps.

Favorable self-assessment. Agents tend to rate their own work well, especially on subjective criteria like tone or persuasiveness. A review that always passes is not adding information.

Endless revision. Without limits, agents may revise again and again, sometimes making things worse or drifting from the brief. Each pass can smooth away something specific that the brief asked for.

Cost and time. Each loop adds model calls. For high-volume tasks, the extra cost adds up quickly; our post on cost per task shows how to account for it.

Designing a good loop

A useful loop is narrow, bounded, and recorded. We recommend four design rules:

  1. Use explicit criteria: give the review step a checklist, not a vague instruction to improve the draft.
  2. Separate the reviewer: use a review step with different instructions and, ideally, access to the source facts the writer may have ignored.
  3. Limit iterations: allow one or two revision rounds, then escalate to a person.
  4. Record the critique: store what the review found and what changed.

Separation does most of the work. A reviewer prompt that holds the fact sheet and a claims checklist catches errors the writer prompt missed, because it is looking at different material. The same prompt asked to reconsider its own draft mostly confirms it.

Recorded critiques have value beyond the single draft. Over a few weeks they show which criteria fail most often. Those patterns tell you where the writer's instructions or reference material need work, which fixes the cause instead of correcting the symptom each time.

A worked example

Consider a hypothetical agent that writes event invitation emails. Its review checklist has five items: under 150 words, event date matches the brief, registration link present, required disclosure present, and every product claim matches the fact sheet.

On one draft, the review step finds the word count at 170 and the disclosure missing. It revises, and the second check passes. The record shows both failures, so the team can see over time whether length and disclosure are recurring problems.

On another draft, the agent states that a feature is available on all plans. The fact sheet says otherwise, but the review step was given only the draft and the checklist. It passes the claim.

An independent checker with the fact sheet catches it. The loop helped with the first draft and could not help with the second, because the second error required knowledge the loop did not have.

Self-critique versus independent checks

For important content, combine self-critique with checks that do not share the writer's blind spots. We use three layers: a separate checker agent with a narrow job, mechanical guardrails in code, and human review.

Self-critique improves the draft before it reaches those checks; it does not replace them. Rules that can be expressed as code, such as banned phrases or required disclosures, belong in guardrails as code, where they run the same way every time. We describe the full structure in multi-agent content pipelines.

A practical rule follows. If an error would be costly to publish, do not rely on a step that shares the writer's knowledge to catch it.

Measuring the benefit

Test whether self-critique helps for your specific task instead of assuming it does. Run your golden dataset with and without the loop, and compare four things:

  • Quality score: results on your rubric, such as the one in our starter rubric for content agents.
  • Error catch rate: the share of known errors in the test set that the loop finds.
  • Cost per task: model spend with and without the loop.
  • Time per task: latency added by each round.

Look at results by criterion, not just overall. A loop may lift format compliance while doing nothing for factual accuracy, which tells you to keep the loop and add an independent fact check. For some tasks, the improvement will justify the cost. For others, it will be marginal, and a cheaper mechanical check will do the same job.

The question we are studying next is when a second model makes a better reviewer than the first. Different models may have different blind spots, which could make cross-model review catch more. We expect the answer to depend on the task, and we are testing it criterion by criterion.

How to apply this: if you use self-critique, replace generic review instructions with a concrete checklist, give the review step the source facts, and cap it at two revision rounds. Then measure, criterion by criterion, whether the loop improves results on your test set.

Related research

Put this research to work

Marketix turns these patterns into software: agents, automated workflows, and a hub that keeps people in control. Sign up for early access.