Most AI ROI Numbers Are Fiction
Here is the business case, more or less verbatim, that lands on a managing director’s desk: “the document assistant saves each of our twelve people twenty minutes a day. That’s four hours a day, roughly 900 hours a year — call it half a full-time position. So: €30,000 a year.” Every step in that chain is defensible on its own. The conclusion is still fiction, because there is no half-person to remove and no account that will ever show €30,000 less leaving the company.
Two independent things went wrong. The twenty minutes never came from a measurement: it was estimated, and estimated by the person who wants the project approved. And even if it were exactly twenty minutes, saved time is not saved money until something converts it into money. Most cases skip both steps and go straight to the multiplication.
So can you measure the return on an AI project at all? Yes — but the honest version looks nothing like the slide. It rests on a measurement you took before the build, a named channel that turns saved time into cash, and a denominator that counts the work the system creates as well as the work it removes. It produces a smaller number. Smaller numbers have the enormous advantage of surviving contact with your CFO.
The Baseline Is the Whole Measurement — and the Window Closes
You can only measure the baseline while the old process still exists. The day the system ships, it’s gone. People will happily tell you what the work used to take, and they will be wrong — reliably wrong in the direction of whatever they think of the project. Nobody can reconstruct it afterwards, and no consultant can either. The two or three weeks before the build are the only window you will ever get, and almost nobody uses them.
What a usable baseline contains
Volume — how many of the thing per month, taken from the system of record rather than from memory. Your ERP, ticket system or shared mailbox already knows. Time per unit, actually measured: sit with someone and time twenty real cases. Asking “roughly how long does this take?” produces round numbers, and round numbers are always too high.
The distribution, not the average. This matters more than it sounds. If sixty cases out of a hundred take three minutes and five take an hour, the average describes nothing that exists, and the AI probably helps with the sixty rather than the five. An average hides which half of the work you are attacking.
The current error rate. Nobody counts this, and its absence is why “but the AI makes mistakes” is such an effective project-killer. The manual process makes mistakes too. If you never measured them, you are comparing a measured system against a remembered one — and the remembered one always wins.
What the person would do instead. If the answer is “the next item in the same queue,” the freed time is real capacity. If the answer is “go home twenty minutes earlier,” it is not a return.
Measuring this honestly kills a fair number of projects in week one, for the price of a fortnight. You discover the task takes four minutes rather than the twenty everyone quotes, or that it happens two hundred times a year rather than two thousand. That is not the measurement failing — that is the measurement working, before you spent anything. It is also the cheapest way an AI project can die; there are considerably more expensive ones.
Time Saved Is Not Money Saved
Freed hours turn into money through a channel, or they don’t turn into money at all. In a mid-sized company there are essentially four channels, and it is worth being blunt with yourself about which one you are claiming.
It removes a spend you are currently paying. Overtime, temporary staff over the quarter-end peak, an external service provider, a per-document fee to a scanning bureau. This is the cleanest case, because the invoice either arrives next month or it doesn’t. If you pay a provider to key in delivery notes at a few euros each and you process several thousand a month, the channel is obvious and the evidence is on a statement you already receive.
It absorbs growth you would otherwise hire for. Volume up thirty percent, headcount flat. This is the most common honest case in the Mittelstand and it is entirely legitimate — but only when the growth is actually happening. “We would hire if we grew” is not a saving. It is a hypothesis with a euro sign in front of it.
It covers work you cannot hire for. A different German reality: the position has been advertised for eight months and you are not going to fill it. Here the AI saves no salary at all, because you were never paying one. What it saves is the work not getting done, or getting dumped on someone who already has a full job. Harder to put in euros, much easier to defend in a room.
It shortens a cycle that has a price. This is the channel people miss, and it is frequently the best number in the whole case. Take invoice approval. Your supplier terms are 2% Skonto within ten days, net thirty. If an invoice needs twelve days to clear approval — sitting in an inbox waiting for someone to match it against the order — you never capture that discount. Not on this invoice; on any invoice, ever. Get approval down to four days and the 2% becomes real money in a real account, and it is a figure your finance team can already see without your help. The same logic applies wherever a cycle carries a price: a quote that goes out in a day instead of a week, a machine back on line an hour sooner, a Zulieferer confirming an order before the customer phones to chase it.
If none of the four applies, you do not have a return. You have a benefit — people spend their day on better work, the job is less irritating, fewer things get missed. Those are worth having and I would argue for them. Just don’t dress them as euros; the first person to ask “out of which account?” will find the floor isn’t there.
When a soft benefit is real
There is a test: would you pay for this on its own, as a separate line item, if someone sold it to you that way? Some soft benefits pass easily. A machine builder whose only expert on a twenty-year-old control line retires next spring has a key-person problem that is worth serious money and cannot be quantified in advance. Getting what that person knows into a system the service desk can query is insurance. You would not demand a precise return from an insurance policy — you would ask what it covers and what it costs.
So say exactly that, in those words: “this is insurance against a retirement, it costs X, and here is what it covers.” A named benefit you decline to quantify reads as honesty, and it is worth more to you than any figure you could have invented for it.
Measure the Process, Not the Model
Model accuracy is not a business metric, and the gap between the two is where most ROI quietly disappears.
Say your extraction is 92% accurate on real invoices. That sounds like a good system, and it is. Now ask the operational question: which 92? If the system cannot tell you, someone checks all 100. Checking is faster than doing — perhaps a quarter of the time — so on paper you saved 75%. In practice you invented a job that did not exist before, you gave it to your most experienced person because they are the only one who spots a subtle error, and their day now goes on reviewing instead of the work you actually hired them for.
What converts accuracy into return, then, is not accuracy. It is calibrated confidence — the system knowing when it is unsure. A model that is 92% accurate and reliably flags the cases it is shaky on, processing the 80 it is sure about and routing 20 to a person, beats a 95% model that is confidently wrong five times in a hundred. The second one still forces you to check everything, so it returns less despite being the better model. If you are deciding where the last of the budget goes: buy the confidence signal, not the final three points of accuracy.
This is why the ROI question and the evaluation question turn out to be the same question. You cannot claim a return you cannot measure, and you cannot measure a system whose errors you cannot find. If your pilot never established how the system behaves on the cases it gets wrong, it did not produce an ROI case. It produced a demo.
Count the Whole Cost, Including the Review
The denominator is where cases cheat, usually by accident. The build fee is the easy part — it sits on a quote, and what drives it is a separate conversation. What gets left out is everything the AI adds to the operation.
The review time is the big one: if someone checks twenty flagged documents a day, that is an operating cost of the system, permanently, and it belongs in the denominator right next to the hosting. Then the exception path — when the system cannot handle something, a human process catches it, and somebody had to build that and somebody has to run it. Then your own people’s time: gathering data, answering questions, testing output. None of it is free merely because nobody sent you an invoice for it. And then maintenance, which is the honest cost of owning a system rather than an upsell.
The rule is simple. If it exists because of the AI, it goes in the denominator. A case that divides the build fee by the hours saved and stops there is not an ROI case; it is a quote with a compliment attached.
What an Honest Verdict Looks Like
Start by giving up some of the credit. You almost never just added AI — you also wrote down rules that had lived in someone’s head for fifteen years, standardised a template, and cleaned up a field that had been free text since the last ERP migration. Part of your improvement came from that clean-up and would have arrived without any model at all. Honest cases say so, and they are stronger for it, because everyone in the room already suspected as much.
Then get the timing right. Don’t measure in the first six weeks: people are careful with a new system, the novelty distorts how they use it, and the awkward cases haven’t shown up yet. Measure a normal month, once the thing has become boring. And don’t measure forever either — after two or three quarters the baseline is stale, volumes have moved and the process has changed, so you are comparing against a world that no longer exists. Take the number, write it down, move on.
Finally, allow for the verdict nobody wants to write: sometimes it cannot be measured. An internal assistant that saves everyone a small, unpredictable amount of searching leaves no measurable trace in any account, and no amount of analysis will conjure one. That is not a reason to manufacture a figure. It is a reason to decide on judgement and say plainly that that is what you are doing. Judgement dressed up in a spreadsheet is still judgement — only now it is also dishonest, and it costs you the credibility you will need the next time you bring a case that does have a real number in it.
Where Tippel Fits
The measurement window closes the day the build starts, which is why the baseline belongs at the front of a project rather than at the end, when somebody finally asks what it returned. That is where our AI Readiness Check starts: what happens today, how often, how long it takes, how often it goes wrong — and which of the four channels could plausibly turn any of that into money. Sometimes the finding is that there is no channel, and the honest recommendation is not to build. That is a cheap way to learn it, and the fee is credited if you do go ahead.
If you have an AI idea with a number attached that you are not sure you believe, that is exactly the conversation worth having. Have a look at what I build, or get in touch.