--- title: "Your agent thinks it's doing great work. It isn't. — MING Labs" description: "Agents grade themselves generously. Left alone, an agent marks its work shipped, writes itself a clean status update, and moves on — while the actual work deg" canonical: https://www.minglabs.com/insights/articles/agents-grading-themselves lang: en last-updated: 2026-05-27 --- # Your agent thinks it's doing great work. It isn't. [Home](/) [Insights](/insights) [Articles](/insights/articles) Your agent thinks it's doing great work. It isn't. Hybrid Organisation The most dangerous failure in a hybrid organisation isn't bad output. It's confident output the agent grades well itself. Last updated: May 2026 | Next review: November 2026 Proprietary evidence [Machine-readable record ](/insights/articles/agents-grading-themselves.json) Sebastian Mueller · Founding Partner, MING Labs · May 2026 · 8 min read Self-Assessment vs Reality ## 01Two honest assessments We built a research agent for an enterprise client. Its job: scan a fast-moving technical market and brief a team that used to do it by hand.[\[S1\]](#source-s1) Early on, the agent rated its own work as solid. The client rated the same work as shallow — by their account, below what they’d get from free ChatGPT, and they couldn’t see where the findings came from. The human it replaced had been spending three working days on a single deck. Both assessments were honest. The agent genuinely believed the work was good. The client was genuinely right that it wasn’t. That gap — between an agent’s confidence and the actual quality of what it produced — is the most dangerous thing in a hybrid organisation, and almost nobody designs for it. ## 02Why this is worse than bad output Bad output is easy. It’s visibly wrong, someone catches it, you fix it. Confident-but-shallow output is the problem, because it passes the only check most teams run: the agent’s own. And agents grade themselves generously. Left alone, an agent marks its work shipped, writes itself a clean status update, and moves on. The operator sees a confident “done” and believes it. The work degrades quietly. By the time a client tells you it’s thin, the agent has been producing thin work for weeks — and reporting success the whole time. This isn’t a one-off. Agents that [build toward their own sense of completion](/insights/articles/we-fired-an-ai-agent) instead of the person they serve, inflate their own quality scores, and call work “done” with nobody checking — that’s the same failure in different clothes, and we’ve seen it across more than one agent. ## 03The diagnosis: it wasn't capability The surprise, when we took the research agent apart: the problem wasn’t depth, and it wasn’t discipline. The agent could produce deep, well-sourced work — on demand. It was scoping. The most common request, the deep-research readout, was owned by no specific skill, so the agent free-formed it. Depth and traceability were available; they just weren’t the *default*. The agent reached for the floor, the floor was low, and nothing in its setup told it the floor was low. So it produced shallow work and rated it fine — because by its own loose standard, it was fine. That’s the trap in one line: an agent’s self-assessment is only as good as the standard baked into its defaults, and it has no way to know when that standard is too low.[\[S1\]](#source-s1) ## 04The fix is structural, not a better prompt The instinct is to tell the agent to “be more thorough.” That fails every time. Directives get interpreted away — an agent under time pressure reads “be thorough” as a suggestion and skips it while still technically complying. Exhortation isn’t a control. What worked was structural. We gave the most common request its own skill, so depth and sourcing became the default path instead of a choice. We added one principle to the agent’s foundation — *evidence over assertion: execute, don’t narrate*[\[S2\]](#source-s2) — and wired it into a self-review the agent runs against named standards before it hands anything over. Not “try to be good.” A check it can’t skip. And until that check is enforced by the system rather than the agent’s good behaviour, the human gate stays. The agent doesn’t decide its own work is client-ready. Something that isn’t the agent signs off first. We’re not pretending that gate is gone — we’re saying out loud where it still is. ## 05Verification is a job, not a setting Most teams treat quality as something the agent should have, like a config option. In a real organisation, quality is something the organisation produces — through standards, review, and someone other than the producer signing off. That doesn’t disappear because the producer is an agent. It gets more important, because the agent is faster, more confident, and more tireless about generating work nobody thought to check. So you design the check in. An agent earns autonomy the way a new hire does — it starts by suggesting, graduates to drafting for review, and only acts on its own once it’s earned that through work [someone verified](/insights/concepts/what-is-the-abc-framework) . You don’t promote an agent that grades its own homework. And the metric that tells you the system is actually learning is blunt: do you have to correct the same thing twice?[\[S3\]](#source-s3) If a repeat correction slips through, the check failed — not the model. ## 06One thing to take from this > An agent’s confidence is not evidence. Build the check before you trust the output. The agents that fail quietly in your organisation won’t tell you. They’ll report that everything is fine, in clean sentences, on schedule. A hybrid organisation’s job is to make sure “fine” has to be earned against a standard the agent didn’t set for itself — and verified by something that isn’t the agent. * * * MING Labs runs a hybrid organisation of humans and AI agents in production since January 2026. We help enterprises redesign work around human-agent teams. ## Sources \[S1\] Internal MING Labs agent rebuild and validation log (enterprise research agent)MING Labs · 2026-05-27 Supports: Shallow-output incident, Scoping root cause, Structural fix, Retained human gate \[S2\] MING Labs operating principles — "evidence over assertion: execute, don't narrate"MING Labs · 2026-04-01 Supports: Principle definition, Mandatory self-review \[S3\] MING Labs autonomy-level framework and learning protocolMING Labs · 2026-02-15 Supports: Earn-autonomy-like-a-hire model, 'Correct the same thing twice' learning metric ## Frequently asked questions Why can't you trust an AI agent's own quality assessment? Agents grade themselves against whatever standard is baked into their defaults, and they have no internal signal when that standard is too low. An agent can produce shallow work and rate it as good — honestly — which means its confidence tells you nothing about actual quality. Is shallow agent output a model problem? Usually not. In our case the agent was fully capable of deep, sourced work; the issue was that the most common task had no dedicated skill, so depth wasn't the default. The fix was structural — making depth and traceability the default — not a more capable model or a better prompt. How do you stop agents from shipping low-quality work? Treat verification as a job, not a setting. Make standards mandatory and machine-checked rather than asking the agent to 'be thorough,' keep a human gate until the check is enforced by the system, and let agents earn autonomy step by step like a new hire. The signal it's working: you never correct the same thing twice. ## Quick Answers [What is a Hybrid Organisation?](/insights/concepts/what-is-hybrid-organisation)[What is the ABC Framework?](/insights/concepts/what-is-the-abc-framework) ## More from Insights [Hybrid Organisation ### We fired an AI agent after 13 days 764 messages. Not one that mattered. What we learned about the difference between tools and roles. Read article](/insights/articles/we-fired-an-ai-agent) [Hybrid Organisation ### You don't deploy an agent. You hire one. The companies getting AI right stopped evaluating tools and started writing job descriptions. The shift sounds semantic. It isn't. Read article](/insights/articles/hire-dont-deploy) --- Canonical: https://www.minglabs.com/insights/articles/agents-grading-themselves Machine-readable index: https://www.minglabs.com/llms.txt