--- title: "How do I measure whether AI speeds up software delivery? — MING Labs" description: "Measure AI coding gains after review and rework. An invented comparison and a task record separate human effort, delivery time and accepted results." canonical: https://www.minglabs.com/insights/answers/how-do-i-measure-ai-software-delivery-productivity lang: en last-updated: 2026-11-08 --- # How do I measure whether AI speeds up software delivery? [Home](https://www.minglabs.com/) [Insights](https://www.minglabs.com/insights) [Answers](https://www.minglabs.com/insights/answers) How do I measure whether AI speeds up software delivery? Answer Measure the time until a useful change is accepted, together with the human effort needed to get it there. Include review, rework and failed attempts; compare similar tasks under the same quality requirements. Our proposed method and invented example show why faster coding can still mean more work for the team. Record later defects separately before judging the result.[\[S1\]](https://www.minglabs.com/insights/answers/how-do-i-measure-ai-software-delivery-productivity#source-s1) Invented example, not a measured result Faster coding can leave more work. person-minutes Without AI**180** With AI**190** CreatingChecking and finishing The assumed creation time falls from 120 to 45 minutes. Review, rework, testing and release rise from 60 to 145. Total human effort rises from 180 to 190 person-minutes. Both bars use the same scale. Source: MING Labs, 8 November 2026. Last updated: November 2026 | Next review: December 2026 Proposed measurement method [Machine-readable record ](https://www.minglabs.com/insights/answers/how-do-i-measure-ai-software-delivery-productivity.json) By MING Labs · Editorial team ## A finished draft is only part of the work. Imagine adding an overdue-invoice filter to an internal application. The code appears quickly, but somebody still has to check which invoices qualify, correct the edge cases and release the change. If the finance team cannot trust the results yet, the task is still open. This is an invented example. We propose agreeing the expected behaviour before the comparison starts: an unpaid invoice past its due date appears; a paid or not-yet-due invoice does not. Apply the same requirements to comparable changes with and without AI. The reviewer records acceptance and the reason for rejection, rather than judging whether the code merely looks plausible.[\[S1\]](https://www.minglabs.com/insights/answers/how-do-i-measure-ai-software-delivery-productivity#source-s1) The assumed 75-minute saving in creation is outweighed by 85 additional minutes of review, rework and release. The AI-assisted change uses ten more person-minutes in total. These inputs demonstrate the accounting; they do not predict what will happen in your team.[\[S1\]](https://www.minglabs.com/insights/answers/how-do-i-measure-ai-software-delivery-productivity#source-s1) | Human effort in the invented example | Without AI | With AI | | --- | --- | --- | | Creating the change | 120 min | 45 min | | Reviewing it | 30 min | 90 min | | Rework | 20 min | 35 min | | Testing and releasing | 10 min | 20 min | | Total human effort | 180 min | 190 min | ## Does your team need a clearer acceptance path? Discuss one development task and the evidence needed to call the result usable. [Discuss my development workflow](https://www.minglabs.com/contact?type=project&topic=software-productivity) Tell us what you have in mind. We will respond personally. ## Keep effort, waiting and quality visible. A reviewer can spend half an hour on a change that waited all afternoon. Record both quantities. In our illustration, the unassisted task runs from 09:00 to 14:30, or 330 elapsed minutes; the assisted task runs from 09:00 to 16:00, or 420. Those spans include waiting and are separate from the 180 and 190 person-minutes of active work.[\[S1\]](https://www.minglabs.com/insights/answers/how-do-i-measure-ai-software-delivery-productivity#source-s1) A usable comparison needs a record of each task’s starting point, effort, elapsed time and outcome. Add the defect follow-up when the agreed observation window ends. | Record for each task | What belongs in the record | | --- | --- | | Comparable starting point | Task class, expected behaviour, risk, repository familiarity, developer experience and tool/model version. | | People’s effort | Active minutes for creation, review, rework, testing and release across everyone involved. Allocate shared setup explicitly. | | Delivery time | Start and acceptance timestamps, including waiting. Keep machine runtime separate. | | Useful result | Accepted, rejected or abandoned, with the reviewer and reason. Retain unsuccessful attempts. | | Later quality | Defects, their severity and repair effort during a window agreed in advance. An unobserved window is unknown, not zero defects. | Failed attempts change the answer. Suppose a second AI-assisted attempt takes 95 person-minutes and is rejected. The batch now consumed 285 minutes for one accepted change, not 190. With no accepted result, effort per accepted change is undefined; report the effort and zero accepted results separately. Later repair effort also stays visible.[\[S1\]](https://www.minglabs.com/insights/answers/how-do-i-measure-ai-software-delivery-productivity#source-s1) ## Compare similar work before drawing a team-wide conclusion. Group comparable tasks before assigning them to the two approaches. Do not give AI all the easy jobs or ask someone to solve the same already-understood problem twice. Record task differences and repeat the comparison. Random assignment can strengthen a study; an observational team record does not isolate AI’s causal effect. Current research makes that caution practical. In February 2026, METR explained why its later developer experiment did not provide a reliable estimate of the effect: who participated, which tasks they submitted and concurrent agent use complicated the comparison. Its May report on a survey of 349 technical workers, conducted from February to April 2026, described perceived benefits. These were self-reports rather than observed gains.[\[S2\]](https://www.minglabs.com/insights/answers/how-do-i-measure-ai-software-delivery-productivity#source-s2)[\[S3\]](https://www.minglabs.com/insights/answers/how-do-i-measure-ai-software-delivery-productivity#source-s3) More output can also mean different work. A dashboard that is now cheap to build might help the team, but its hypothetical manual build time is not automatically saved business value. Record why the delivered change was needed and what it enabled. METR’s separate analysis explains this distinction between task substitution and value.[\[S4\]](https://www.minglabs.com/insights/answers/how-do-i-measure-ai-software-delivery-productivity#source-s4) ## Start with one recurring change your team can judge. Choose a familiar task class with the developer and the person who accepts the result. Write the acceptance cases and defect-review date before starting; use the record above for both approaches. Review the completed sample together: did useful changes arrive sooner, use less human effort and meet the same quality bar? Where the answers differ, identify the stage that needs attention before expanding AI use. That is the operating question behind a [hybrid organisation](https://www.minglabs.com/hybrid-organisation): how people and agents deliver useful work together. We suggest this record as a starting method. It does not certify code security, price the tools or establish a staffing saving. Citable short version MING proposes comparing AI-assisted and unassisted software work by accepted useful changes, total human effort and elapsed delivery time, with the same acceptance criteria and a declared defect-review window. Count rejected attempts in the effort total. This is a measurement method, not evidence that AI universally accelerates or slows development. Invented worked example, checked arithmetic and primary research with stated limitations. No measured MING productivity gain. ## Where does your AI-assisted delivery need more clarity? Bring one recurring software task and the acceptance decision that is unclear. We can discuss how people and AI work together and what evidence would support that decision. [Discuss my development workflow](https://www.minglabs.com/contact?type=project&topic=software-productivity) Tell us what you have in mind. We will respond personally. [See what running an agent costs ](https://www.minglabs.com/insights/answers/what-does-it-cost-to-run-an-ai-agent-in-production) ## Sources \[S1\] Software delivery measurement example MING Labs · 8 November 2026 Supports: 180 = 120 + 30 + 20 + 10; 190 = 45 + 90 + 35 + 20 person-minutes. A rejected 95-minute attempt raises the AI-assisted batch to 285 minutes for one accepted result. Original proposed method and invented arithmetic. No measured team result, benchmark or client data. \[S2\] [We are Changing our Developer Productivity Experiment Design](https://metr.org/blog/2026-02-24-uplift-update/) METR · 24 February 2026 Supports: The authors consider their later experiment an unreliable measure of the current effect size; parallel agents complicate time recording. Follow-up to a randomized developer study; authors identify participant/task selection and time-recording limitations. \[S3\] [Measuring the Self-Reported Impact of Early-2026 AI on Technical Worker Productivity](https://metr.org/blog/2026-05-11-ai-usage-survey/) METR · 11 May 2026 Supports: Respondents report benefits; perceived speed, value and measured productivity are different quantities. Survey in February to April 2026, 349 technical workers including 87 software engineers; self-reports, not observed performance. \[S4\] [Task Substitution and Uplift](https://metr.org/blog/2026-05-08-task-substitution-and-uplift/) METR · 8 May 2026 Supports: A change in the tasks undertaken can separate speed gains from the value of the work. Analytical discussion, not a measured universal effect. ## Frequently asked questions Can I use lines of code or completed tickets as my productivity measure? They describe activity or throughput, but tasks vary in difficulty and usefulness. Record task class, acceptance and defects alongside volume. A pile of easy changes should not stand in for delivery of a valuable feature. How do I record work with several agents running at once? Record each person’s active time without counting the same minute twice. Add effort across different people, even when they work simultaneously. Keep agent runtime and elapsed delivery time separate from person-minutes. How long should I run the comparison? Choose the task classes, acceptance criteria and defect-review window before starting. Gather repeated comparable tasks and report the range and sample size. One worked example or a short run of easy tasks cannot establish a reliable team-wide gain. Does saved time mean we can reduce the team? No. Establish whether the capacity was freed at the right time and used for valuable work. Staffing, demand and realised cash savings require separate evidence; shorter coding time alone supports none of those conclusions. ## Available actions - **Copy answer for my AI**: copies the [AI briefing](https://www.minglabs.com/insights/answers/how-do-i-measure-ai-software-delivery-productivity.md) to the clipboard in the browser. --- Canonical: https://www.minglabs.com/insights/answers/how-do-i-measure-ai-software-delivery-productivity Machine-readable index: https://www.minglabs.com/llms.txt