10 to the 23 AI logo
Stephen Lieberman
Through 1023AI
Contact

Practice Library

Governance patternstructural

Put a verifier on the agent

Attach an independent checking step to the least-supervised operator — in PAN runs, the single highest-leverage move.

What it changes

dampenedFailures adopted by people or agentsPAN Lab model result: ≈46% of the harm that persists removed in the published runs (vs ≈6% for a model upgrade alone).

Who can pull it

Deploying organizationHarness builderDeveloper

What it looks like institutionally

Autonomous and lightly-supervised operators are where adopted errors concentrate: nothing stands between the model's output and action. Attaching a verifier — a human check, a second independent system, a mandatory ground-truth lookup — inserts correction exactly where correction was absent.

The general principle: rank operators by how unsupervised they are, and spend verification there first. A verifier on an already-well-supervised workflow buys little; the same verifier on the automated pathway can dominate every other option.

PAN Lab comparisons make the point sharply — see the ledgered scenario result below — but the qualitative logic stands on its own: correction capacity matters most where it is currently zero.

Ledgered PAN-run results used above

In the published runs, over a supervised-plus-agent scenario, adding a verifier to the autonomous agent removed roughly 46% of the harm that persists and a coordinated governance package roughly 43%, while upgrading the model alone removed only about 6%.[†]

Addresses: Unsupervised error adoption · Automation without correction. Test a version of this lever in the PAN Lab.

Cases

Documented cases behind this pattern

Michigan MiDAS

Automated unemployment-fraud adjudication with no human review — the canonical high-error, zero-correction structure.

Deciding whether this lever fits your deployment?

Which patterns matter, and in what order, depends on your system's actual shape. Ranking your options on evidence, with what can backfire stated, is engagement work.

Work with 1023AIBrowse the Practice Library

Sources & Evidence

Claims made on this page and what supports them. The full registry lives in Evidence.

ScenarioIn the published runs, over a supervised-plus-agent scenario, adding a verifier to the autonomous agent remove…

In the published runs, over a supervised-plus-agent scenario, adding a verifier to the autonomous agent removed roughly 46% of the harm that persists and a coordinated governance package roughly 43%, while upgrading the model alone removed only about 6%.

From the published runs: PAN social-work governance guidance, lever-ranking comparison.