The Verification Tax: The Hidden Cost of AI Output

On the third day of a course we ran for a client's technical team, we split the room in two and gave both halves the same ticket from their own backlog. One half worked normally. The other half was told to use AI as aggressively as they liked.

The AI half finished first, comfortably. They also spent most of the following afternoon on it, which is the part that does not show up in anybody's demo.

Finished is not the same as done

What happened was not a failure. The output was good. It compiled, it ran, it looked like something a competent person would have written — and that was exactly the problem, because it also contained one assumption about how their pricing tiers worked that was true in 2023 and has not been true since.

Nobody spotted it by reading. They spotted it because a senior developer, made uneasy by how quickly the work had arrived, went through it line by line against the actual business rules. That review took longer than writing the code would have.

So the honest accounting for that ticket is: production time down by about eighty per cent, verification time up by rather more, net result slower — and the verification was done by the most expensive person in the room. That is not a saving. It is a transfer, from a cheap activity to an expensive one, disguised as speed.

The only equation that matters

We wrote this on the whiteboard and it stayed up for the rest of the week.

Net gain = time saved producing − time spent verifying − the cost of what you fail to catch.

Every conversation about AI productivity that ignores the second and third terms is marketing. Note that the third term is not a time cost at all; it is a risk, paid at unpredictable moments, usually to somebody who is not you. A wrong figure in a board pack, a mishandled edge case in a payment path, a confident paragraph in a client proposal that commits you to something you cannot deliver.

Once a team is thinking in those terms, the useful question stops being can AI do this task — the answer is almost always some version of yes — and becomes how cheaply can I know whether it did it correctly. That question sorts work into two piles very quickly.

Cheap to verify, expensive to verify

Verification is cheap when the task has what we ended up calling an oracle: something outside your own judgement that tells you whether the answer is right.

Cheap: code with a test suite that actually covers the behaviour. A query whose result you can reconcile against a total you already trust. A config change you can apply to a staging environment and watch. A migration you can run twice and diff. A summary of a document you have read. A translation between two formats where you hold both. In each case the check is mechanical, fast, and does not depend on how convincing the output sounds.

Expensive: prose that asserts facts about your business. A summary of a document nobody has read. A recommendation with no counterfactual. A config that will not be exercised until the failover you are hoping never happens. Anything where the only available check is a knowledgeable person reading carefully — because that is precisely the resource you were trying to spend less of.

The pattern is not about difficulty. It is about whether being wrong is detectable before it is expensive.

Ask for output you can check

The best habit the team took away was not a prompt. It was a preference: when there is a choice, ask for the form of output with the fastest oracle.

If you need a rule implemented, ask for the tests first, read those, and then write or generate the implementation against them. Reading tests is fast, and a wrong test is far more obvious than a wrong implementation. If you need a figure, ask for the query rather than the number — a query you can run, and a number you can only believe. If you need a recommendation, ask for the two options that were rejected and why; the reasoning is checkable in a way that a conclusion is not.

None of that is a trick to make the model behave better. It is a way of arranging the work so that being wrong is loud rather than quiet.

Never accept work you could not have produced

This was the rule the team wrote for themselves at the end of the week, and it is a better rule than anything we brought with us.

It is not a rule about ability, it is a rule about position. If you could have written it yourself, given time, then reviewing it is genuine review: you know where the difficulty was, you know which line to look at hardest, and you would notice the assumption that quietly does not hold. If you could not have written it, you are not reviewing — you are hoping, in a way that reads exactly like reviewing from the outside, including to you.

That distinction is what makes the same AI output completely appropriate in one person's hands and reckless in another's. It is also why the tools help experienced people considerably more than they help beginners, whatever the marketing implies.

The tax is highest exactly where the temptation is

There is an unhappy symmetry here. The work you least want to do is often the work you are least equipped to check.

The unfamiliar corner of the codebase. The compliance document you have been avoiding. The vendor's API you have never used. The regulation you half understand. These are the tasks that make people reach for an assistant, and they are precisely the tasks where the verification cost is highest, because you have no standard of your own to measure against. It is the same reason we keep arguing that knowing where AI does not belong is worth as much as knowing where it does.

The workable version is not to avoid those tasks. It is to change what you ask for. In territory you do not know, ask for orientation rather than output: what are the parts of this, what usually goes wrong, what should I be reading, what would a specialist ask me. Then do the work yourself, more quickly than you would have. You have used the tool to reduce your ignorance instead of to hide it.

What this means for the ROI conversation

Businesses measuring AI value by hours saved on production are measuring the easy half and reporting it as the whole. The real number is net of verification, and verification is usually done by senior people whose time is the scarce resource in the first place.

It is worth measuring properly, because the honest number is still good — often very good — for the right tasks. Track two things for a month on a handful of real jobs: how long the production took, and how long the checking took. The tasks where that ratio is healthy will surprise you, and so will the ones where it is not. Then push the work toward the first list. That is a more useful exercise than any tool evaluation, and it is the practical end of the argument we made about where AI actually earns its keep.

The honest summary

Speed of production is the least interesting property of these tools. What determines whether they pay is the cost of knowing you can trust the result — and that cost varies enormously by task, by person and by the form you asked the output to take. Choose work with a fast oracle. Ask for shapes you can check. Refuse output you could not have produced yourself. Do that and the savings are real and durable. Skip it and you will be very fast at generating things somebody else has to read carefully, which is a strange thing to have optimised for.