How to measure ROI of AI automation, honestly

The question arrives in a predictable shape. Something was automated six months ago, everyone agrees it helped, and now someone senior wants the return in writing. The people who built it start assembling a figure, and within an hour they are multiplying an estimated time saving by an estimated hourly rate, which is the moment the exercise stops being measurement.

This is not a local embarrassment. It is the central finding of the published research, and it is worth starting there because it tells you what kind of problem you have.

The gap the surveys keep finding

Stanford’s 2026 AI Index reports that organisational adoption has reached 88 per cent. Adoption is close to universal and is no longer an interesting number about anyone.

Returns look nothing like that. McKinsey’s State of AI survey, fielded in May and June 2026 across 1,719 participants, found roughly 37 per cent attributing any EBIT impact to AI, a figure essentially unchanged on the previous year, and about 6 per cent qualifying as high performers, meaning they credit AI with at least five per cent of EBIT. Meanwhile 80 per cent said it had improved their own productivity.

Eighty per cent feeling faster and six per cent able to find it in the accounts is the whole problem in two numbers. The survey’s own regional spotlight is blunter about it: most respondents cannot quantify the impact at all, which makes this a measurement gap rather than a technology gap. That distinction decides what you do next.

Why saved minutes do not become money

A tool that saves eleven people twenty minutes a day has saved almost four hours daily, and in most organisations that money never appears anywhere. Nobody was removed, no contractor was cancelled, no deadline moved. Each person absorbed their twenty minutes into a day that was already full.

That is not a trick of accounting, it is what actually happened. Scattered minutes are real for the people living them and invisible to the business, and a return calculated from them will not survive a finance review.

Saved time converts into money in three situations only: a cost you stop paying, a volume you now handle without adding headcount, or a delay that was costing revenue and no longer does. If your automation does not produce one of those three, say so plainly and defend it on quality or risk instead. That is a stronger position than a number nobody believes.

Four numbers that survive scrutiny

These are measurable in any process worth automating, and all four can be read from systems rather than from how people feel.

Cycle time is the one to lead with, because it is hard to argue with and it includes the waiting that time-and-motion estimates always miss. Exception rate is the one that predicts whether this will still be running in a year: if the share needing a human does not fall over the first two months, the approach is wrong and more tuning will not save it.

Rework is the number most often left out, and leaving it out is how an automation posts a saving while quietly moving work downstream. We made the same point about extraction pipelines in our note on automating data entry from emails, where a system that is 95 per cent accurate with no way to tell which 95 is worse than the manual process it replaced.

The baseline is the whole exercise

Almost every unmeasurable automation project has the same root cause: nobody recorded what the old process cost while it was still running. Once it is gone, the comparison can only be reconstructed from memory, and memory is generous about processes it was glad to lose.

Two weeks of the four numbers above, taken before anything is built, is the cheapest work in the project and the only part that cannot be done later. It also tends to change what gets built, because a fortnight of real timestamps usually shows the delay sitting somewhere other than where everyone assumed.

If the process is too informal to measure, that is the finding. A process nobody can describe consistently is not ready to be automated, and our note on internal tools and when to stop using Excel covers where that line sits.

Costs that get left out of the numerator

Returns get overstated mostly through omission, and the same three items go missing. Usage billing is the first: the McKinsey survey found about 20 per cent of respondents reporting that operating costs, token costs among them, had constrained their use. That is not a rounding error, and unlike a licence it rises with success.

The second is review labour. If a person checks the output, their time belongs in the cost of the automated process, not in the old one. The third is maintenance: a model or an upstream format changes, and someone has to notice and fix it. Projects that skip this line are the ones that quietly stop being used.

Why redesign shows up in the returns

The clearest pattern in the McKinsey data is about process rather than technology. Roughly three-quarters of the high performers reported fundamentally redesigning workflows, against about one in four of everyone else.

That matches what the work looks like up close. Automating a step inside a process designed around a human doing that step gets you the step back and nothing else, because the approvals, handoffs and batching around it were all built for the old constraint. The returns come from removing the waiting, and the waiting is usually not in the step you automated.

This is also why reporting is such a common first project and such a common disappointment. Generating the report faster does not help if the decision it feeds still waits for a monthly meeting. We scoped that honestly in reporting automation for a small business, and the same caution applies to what belongs on a dashboard for a business owner: faster numbers are only worth something if somebody acts on them sooner.

When not to bother measuring

Measurement has a cost, and for small automations it can exceed the saving. If something took an afternoon to build and removes a weekly annoyance, instrumenting it is waste. Keep the four numbers for work where the spend is real or where someone will eventually ask for the return in writing, which is most things above a few weeks of effort.

Two honest limits on everything above. The survey figures are self-reported and directional, not audited, and they describe large organisations more than small ones, so treat them as the shape of the problem rather than as your benchmark. And some automations are worth doing with no measurable return at all: work that is accurate instead of fast, or a risk removed rather than a cost. Our note on what to trust a support chatbot with is mostly about that trade, and the reason to be explicit is that an unmeasurable benefit defended honestly survives longer than a measurable one that was invented.

A workable sequence

• Take two weeks of cycle time, exception rate, cost per item and rework before building anything.
• Decide in advance which of the three conversion routes applies: a cost stopped, volume absorbed, or a delay that was costing revenue.
• Put usage billing, review labour and maintenance in the cost of the new process.
• Look at the whole process for the waiting, not just the step you can automate.
• Report the four numbers and the honest limits, and resist multiplying estimated minutes by estimated rates.

The uncomfortable implication of the 88 per cent and the 6 per cent sitting next to each other is that adoption was the easy part and nobody is being graded on it any more. The number that will be asked for is the one that needed a baseline, and the baseline has to be taken before the thing you are proud of exists.