Follow the customer request, not just the first model call.
Imagine an agent drafts replies about delivery dates. It reads the customer’s question, looks up an order and prepares a reply for a colleague to approve. When the order number is missing, it should ask for it. If you compare only the first model call, you miss the second lookup, another attempt at the answer and the colleague who repairs it.
We would start with a record for each request: the task, model and settings, all usage and tool charges, retries, result, review time and acceptance decision. Tokens are the units used to process text; input and output can have different rates. Read the provider’s billing fields and reconcile them with the actual charge. A subscription allowance needs its own accounting; a token count is not automatically an extra invoice.
Define a usable reply before testing. In this example it uses the right order, gives only a supported date and asks for missing information. Record a correct handoff separately from a completed reply. The same definitions must apply to both versions.[S1]
Remove repeated work before taking away information the task needs.
A useful first experiment is to return the order status and expected date instead of the entire order history. Another is to remove a repeated instruction or ask for the specific customer reply rather than a long internal summary. Change one thing so you can explain the result. OpenAI’s optimization guide follows a baseline, change and quality-check sequence; its simulated savings are not a forecast for your workload.[S2]
Keep the order’s identity, relevant exception and source date. If the shorter record hides a split shipment, the agent may produce a confident but unusable answer. The person approving the reply should see the same evidence and use the same criteria in both versions. Compare routine requests and important exceptions separately, so a better average cannot conceal a new serious mistake.
There is a concrete reason to test this. In an April 2026 postmortem, Anthropic reported reverting a reduced reasoning default and a verbosity instruction after quality problems. It also fixed a bug that repeatedly removed earlier reasoning. These were product-layer issues; the report says the API was unaffected. It is a counterexample to assuming that less thinking or less text is always an improvement.[S4]
Caching helps repeated input; check that reuse actually happens.
If many requests begin with the same instructions, prompt caching can reuse that prefix. Put stable material together and keep the changing order details separate where the provider’s interface permits it. Anthropic’s documentation distinguishes cache writes, reads and uncached input. A changed prefix or expired cache can prevent reuse; output still has to be generated.[S3]
For the delivery reply, inspect a cold request and later repeated requests. Compare billed totals, including any cache-write charge. Do not pad a short prompt merely to reach a cache threshold. If repeated lookups, retries or human correction dominate the cost, a cache discount may address only a small part of the problem.
A €3 model saving is lost when correction adds €30.
Our invented example compares the same 100 initial requests. We choose rates of €2 per million input tokens, €8 per million output tokens and €60 per review hour. These are arithmetic assumptions, not current supplier prices or a MING offer. All token totals include additional attempts; do not add the retry count to the bill again.[S1]
The example keeps the starting workload and assumed rates fixed; every outcome below is invented.
| For 100 initial requests | Baseline | Less repeated context | Too much context removed |
|---|---|---|---|
| Input tokens (millions) | 2 | 1.5 | 1 |
| Output tokens (millions) | 0.5 | 0.25 | 0.375 |
| Additional attempts | 20 | 12 | 25 |
| Model usage | €8.00 | €5.00 | €5.00 |
| Review and correction (minutes) | 60 | 60 | 90 |
| Review and correction | €60.00 | €60.00 | €90.00 |
| Combined cost | €68.00 | €65.00 | €95.00 |
| Accepted drafts | 90 | 90 | 85 |
| Combined cost per accepted draft | €0.76 | €0.72 | €1.12 |
Model cost is input millions × €2 plus output millions × €8. Review cost is minutes ÷ 60 × €60. Divide their sum by accepted drafts: €68 ÷ 90 is about €0.76; €65 ÷ 90 is €0.72; €95 ÷ 85 is €1.12. The last variant has the same €5 model bill as the leaner one, but its assumed extra correction makes each accepted draft more expensive.
These scenarios show how to calculate the decision; they do not show that removing context caused these outcomes in a real test. Nor does an equal acceptance count establish equal quality: inspect the individual errors and any non-negotiable requirements. The full monthly budget also includes costs outside this example, covered in our answer about operating an AI agent .
Keep the change only after reviewing the failures as well as the bill.
Run the current version and the proposed version on the same representative tasks under comparable conditions. Record the model version, settings and source records so another person can repeat the comparison. Where responses vary, use repeated runs and a held-back set; a single successful demonstration is not evidence that the failure rate has stayed the same.
Before the comparison, agree which failures block release and what variation in quality, review time and response time is acceptable for this task. Keep unauthorised actions and invented delivery promises out of the savings calculation: a cheaper failure is still a failure. Our quality-measurement answer explains why correct completion and correct escalation need separate counts.
Evidence and limits
- The requests, rates, token totals, retries, review times and acceptance counts are invented. No model benchmark or customer saving was measured.
- The proposed method combines public technical guidance with our own arithmetic. It offers no universal quality threshold, sample size or guaranteed saving.
- Only model usage and review are included. Tool fees, implementation, hosting and ongoing support are excluded. Zero accepted results make unit cost undefined.
Choose one repeated task and one change worth testing. Save its current requests, agreed checks and complete cost record before editing the prompt. That gives the team a defensible reason to keep the change, revise it or restore the previous version.
