Start with the routine task you want to stop checking
The agent has prepared the next update, but it is waiting for you to approve it. If every result lands in the same queue, producing more results gives you more to read. The useful question is which of those checks still prevents a mistake that matters.
In our public account, an order agent began by writing drafts. Its manager later trusted it with standard cases, while exceptions still went to a person. The account describes the direction of travel; it does not publish the case counts, acceptance criteria or error rates that justified the change.[S1] We can use that observation to frame a decision method, but not to claim a proven threshold.
Consider a separate, illustrative task: an agent sends the internal sales team a status update from the order system. The team wants fewer approvals for routine messages. That is a narrower decision than letting the agent promise a delivery date to a customer or change an order. Write down which message, recipient and source record the proposed release covers before discussing how much review to remove.
A later sample cannot stop a mistake that has already taken effect
For the internal status update, ask what happens if the message is wrong and nobody reads it until the next sample review. Can the team correct it before someone acts on it? Does the error affect only the message, or could it trigger a promise, payment or change elsewhere? The answer determines whether later detection is acceptable.
OpenAI’s agent guide uses factors such as reversibility, permissions and financial impact to assess tool risk, and recommends human intervention for high-risk actions.[S3] Our proposal is to retain prior approval wherever the consequences of an unnoticed error are unacceptable. A year of good behaviour does not, by itself, remove that boundary.
Keep the boundary in the actual workflow. If the agent may send an internal update but must not change an order, its access and routing should reflect that. An unfamiliar request or missing source record goes to a person before delivery. Reducing checks on one permitted task must not quietly expand what the agent can do.
Require results and working handovers for the exact task being released
Before changing the approval step, assess the current setup against a written definition of a correct update. In the example, that includes the right order, the current source status, the intended internal recipient and no unsupported promise. Test ordinary requests as well as missing records, conflicting statuses and requests outside the permission boundary.
Keep the inputs and criteria fixed when comparing a change, repeat cases where behaviour varies, and check the actual outcome rather than the agent’s success message. These are principles from Anthropic’s evaluation guidance.[S2] Our production acceptance guide covers that test setup; the decision here is whether its evidence supports a particular reduction in ongoing review.
Inspect the handovers too. Did an unclear case stop before a message was sent? Did the right colleague receive the original request and unresolved question? Count wrong results, appropriate handovers and missing work separately, as in our quality guide . Record the number of cases assessed, the types covered and what remains untested. A total labelled “passed” is too little to base this decision on.
Keep routine sampling separate from checks on unusual cases
Once the process owner has approved the bounded change, someone must still look at real work. We propose two separate views: a random sample from the permitted routine workload, and targeted checks of unusual inputs, complaints and near misses. The first helps avoid reviewing only memorable cases. The second helps investigate specific risks. Do not combine their counts into one supposed overall error rate.
NIST defines random sampling through equal chances of selection.[S4] In this example, define the pool as all eligible internal status updates in a stated period, then select from that pool rather than asking the agent to show its best work. Keep failed or missing runs visible separately. Write down the selection method, sample size, review frequency and reviewer before using the results.
No universal percentage makes this safe. A rare but serious failure may not appear in a small sample, and changes in the workload can make yesterday’s sample uninformative. If the decision needs a statistical assurance claim, have the sampling plan designed for that claim and its assumptions. This guide does not supply one.
For day-to-day operation, combine those human checks with alerts for missing records, failed delivery and unexpected actions. Anthropic recommends combining evaluations, production monitoring and periodic human review.[S2] Our audit-trail guide explains what to retain so a reviewer can connect the request to the result.
Record who may reduce review and what brings it back
The process owner makes the release decision; the reviewer supplies evidence. If those are different people, name both. Our responsibility guide explains that distinction. Involve the relevant control owners where the task requires their approval.
Use this proposed decision record for one task. Empty evidence or an unresolved failure means keeping the existing check while the gap is addressed.
| Record before changing review | For the illustrative internal status update |
|---|---|
| Exact permission | Named source system, supported order statuses and internal recipients; no order changes or customer promises. |
| Evidence and remaining gaps | Version tested, period, case counts, task coverage, errors and verified exception handovers. |
| Decision and responsible people | Who approves the narrower review, who assesses samples, and when the decision will be revisited. |
| Ongoing checks | Defined routine sample, separate targeted checks, alerts and time allowed to investigate. |
| Return to prior approval | Specified unacceptable error, broken handover, missing logs or an unassessed change to the task or system. |
Test the return path before relying on it. The team must be able to pause the affected automatic step, handle work waiting in the queue and identify results that may need correction. A model, permission or source-system change needs a fresh assessment of the affected scope. It must not inherit the old release merely because the agent kept the same name.
Start with one task that is already being checked. Bring its recent results and exception handovers to the person who accepts the work. Decide what evidence is still missing, and change only the check that evidence supports. Then measure total review and correction effort across the team, so the apparent saving does not simply become someone else’s queue.
Evidence and limits of the method
- The MING source reports a qualitative change in supervision. It does not publish the underlying promotion criteria, case-level logs or before/after error and time measurements.
- The internal order-status task, routing diagram and decision record are illustrative proposals. They are not a claim about that agent’s exact permissions or an approved company-wide policy.
- Technical guidance supports the method’s ingredients, not a universal release threshold, guaranteed saving or permission to remove a required human approval.
