← How it works

Evaluation gates

Stage 1Offline evalsGolden dataset andrubricStage 2Red teamDeliberate failurehuntingStage 3Shadow modeReal inputs, no realactionsStage 4Limited rolloutAssist or approve toactStage 5ProductionMonitoring and failurebudgetMeets qualitybar on test setGateSevere failuresfixedGateMatches humanbaselineGateWithin failurebudgetGateNew failures in production become new test casesWork only moves right when a gate's criteria are met
  1. Stage 1Offline evalsGolden dataset and rubric
  2. Gate: meets quality bar on test set
  3. Stage 2Red teamDeliberate failure hunting
  4. Gate: severe failures fixed
  5. Stage 3Shadow modeReal inputs, no real actions
  6. Gate: matches human baseline
  7. Stage 4Limited rolloutAssist or approve to act
  8. Gate: within failure budget
  9. Stage 5ProductionMonitoring and failure budget
  10. New failures in production become new test cases at stage 1.
Evaluation gates. Work moves right only when a gate's criteria are met; production failures become new test cases.

Before an agent touches live work, it passes through a series of stages separated by gates. Each gate has explicit criteria, and failures found later feed back into the earliest tests.

What each part does

01

Offline evals

The agent runs against a golden dataset of realistic and difficult cases, scored with a calibrated rubric.

02

Red team

A small group deliberately tries to make the agent fabricate, go off brand, violate policy, or misuse tools.

03

Shadow mode

The agent works on real inputs alongside people, with every external action blocked, and outputs are compared.

04

Limited rollout

The agent takes a live role at the assist or approve-to-act level, with full review.

05

Production

The agent runs at its earned autonomy level with weekly indicators and a failure budget.

06

Gates

Each gate has written criteria: meeting the quality bar, fixing severe failures, matching the human baseline, staying within budget.

07

Feedback

Every real failure becomes a new test case, so the same problem cannot return unnoticed.

See the architecture in the product

Marketix is built on these diagrams: specialist agents, governed connectors, and a hub that plans, routes, and records every run. Sign up for early access.