The model is the easy part. The gate is the design.

A system that cannot say how sure it is forces you to check everything

The economics of an automated step rest on one number: the share of outputs that go through untouched. A system that returns an answer and nothing else sets that share at zero, because the team has no basis for treating one output differently from another. They check all of it, which is the original work plus a subscription.

So the first design requirement is that every output arrives with a stated confidence, and the routing is three ways rather than two: accept without review, send to a person, or refuse and return the item. The third route is the one most often left out, and leaving it out drops the worst cases into the review queue beside the merely uncertain ones, where they are the most expensive items to handle and the least salvageable.

Where the threshold goes is an arithmetic question, not a matter of comfort. On one side is the cost of a wrong answer passing through, which depends entirely on whether it is caught internally in an hour or by a customer in a month. On the other is the cost of a review. Set the threshold where those meet, then check what you assumed: take a few hundred outputs, grade them by hand, and look at how accurate the system was inside each confidence band.

That calibration exercise is where the surprises are. A confidence number a language model reports about its own output is not automatically meaningful, and plenty of them are cheerfully wrong in the top band. Where you can, derive confidence from something external instead: whether an extracted value matches a record in a system that already holds it, whether two independent passes agree, whether the output satisfies a structural check. Those signals are boring and calibrated by construction.

Then re-sample quarterly. Inputs drift, vendors change models underneath you without asking, and a threshold set against last year’s behavior stops describing anything. It takes half a day.

Design the output for the review, not for the demo

Review has to be cheaper than doing, and it has to stay cheaper as volume grows. That is a design constraint on the output rather than an instruction to reviewers, and it is the constraint most often missed because a demo is watched once by people who are paying attention.

• Show the three fields that are usually wrong first, and the twenty-seven that are usually right behind a fold. Which three is an empirical question, and the calibration sample answers it.
• Put the source beside the claim — the sentence in the input that produced the value, not a link to the document. A reviewer who has to open the source is doing the extraction again.
• Show a difference when the item has a previous version, so the reviewer reads what changed rather than the whole record.
• Make rejection structured: a reason from a short list, not free text nobody aggregates. The list is how the gate teaches you where it is set wrong.
• Let a reviewer correct and accept in one action. A gate that only accepts or rejects sends work back around the loop, roughly doubling the handling time for the items that were already expensive.

The anti-pattern is an approval screen that shows the full input next to the full output and a button. It looks like review, and it is the original work with an extra click, so within a month the button is pressed without reading — and a gate everyone passes is worse than no gate, because it produces a record saying the work was checked.

Batch approval deserves the same suspicion. It is legitimate where the reviewer has assessed the batch on a property the batch shares, and a rubber stamp everywhere else. If it exists, measure how often it is used: the number is usually higher than anyone expects, and it is the cheapest early warning that the threshold is set too low.

The exception queue is where this usually fails

The common failure of an automated workflow is not a wrong output. It is a wrong output nobody owned, in a queue that was nobody’s queue, because the workflow was designed up to the point of automation and no further. The items in it are by definition the unusual ones, which makes them the ones where delay costs most.

• A named owner. Not a team and not a rota — one person, changed deliberately when it changes.
• A stated turnaround, so that an item that is stuck becomes visibly stuck rather than merely old.
• An escalation path for what the owner cannot resolve alone, with a name on that too.
• Cover for when the owner is away, agreed before the first holiday rather than during it.
• A route back into the threshold. What the queue keeps rejecting is the best data anyone has about where the gate is set wrong, and in most implementations it is thrown away.

One diagnostic is worth watching above the others: whether the queue grows faster than volume. If it does, the design is wrong in one of two ways. Either the threshold is too aggressive and ordinary items are being sent for review, or review costs more than doing and the queue is where work goes to be deferred. Those look identical on a dashboard and take ten minutes to separate by watching somebody work through five items.

Four numbers that say whether the gate is healthy

• Automatic acceptance rate — the share passing through untouched. The number the project was for, and the one most often not instrumented.
• Override rate inside the reviewed band — how often a reviewer changes something, which is whether the review is doing work.
• Queue age at the ninetieth percentile rather than the average, because the average is dominated by easy items and hides the ones that are stuck.
• Escaped defects — wrong outputs found downstream, by a later step or by a customer. The only one measuring the thing you were afraid of.

An override rate near zero means either the threshold is too conservative and safe items are being reviewed, or nobody is reading. Those two are indistinguishable in the data and easy to tell apart by sampling, and they call for opposite responses, so it is worth the sample rather than the assumption.

One last thing, and it is the point of the piece. Teams describe the human step as temporary — an interim measure until the system earns trust. In most workflows worth automating it is permanent, because the residue after automation is precisely the judgment that could not be specified. Designing it as temporary is how it ends up with no owner, no measurement and no place in anyone’s job description, which is the same failure as not designing it at all, arrived at more expensively.