AI Projects Fail Early and Quietly
The uncomfortable thing about an AI post-mortem is how boring the cause usually turns out to be. You go in braced for a hard technical problem — retrieval that wouldn’t rank, a model that couldn’t hold the domain vocabulary, latency nobody could fix. What you find instead is a scope decision taken in week one, in a forty-minute meeting, by three people who had never sat next to the person doing the work. The technology did what it was asked. The project died anyway.
This is what makes the failure expensive rather than merely disappointing. A project that fails technically fails loudly: the thing doesn’t work, you find out in week three, you stop, you’ve lost three weeks. A project that fails on a week-one decision doesn’t fail loudly at all. It produces a demo that impresses the management, a build that hits its milestones, a status update every fortnight — all of it genuine work, competently done, aimed at the wrong target. You find out at go-live, after you’ve paid for the whole thing.
So the more useful question isn’t “why do AI projects fail?” It’s “why does it take six months to notice?” The answer is that every symptom of the real failure looks exactly like a healthy project until the moment it doesn’t.
The Failures That Look Like Progress
These are the ones I keep running into. They have nothing to do with model choice, and every one of them is set in motion before anybody writes code.
You automated the part that was already cheap
Picture a Zulieferer whose Vertriebsinnendienst handles incoming orders arriving as PDFs and email text. The obvious project: extract the fields, push them into the ERP, give the clerks their time back. The pilot works — high-nineties extraction on clean orders — and everyone calls it a success.
Then it goes live and saves almost nothing, for a reason that was sitting in numbers nobody counted. Say 200 orders a week. Perhaps 140 of them were already clean: the clerk glanced at each and keyed it in inside two minutes. That’s under four hours a week, all told. The other 60 were the actual job — an obsolete article number, a quantity that contradicts the frame contract, a delivery date the shop floor cannot hit, a customer who writes “wie immer” in the notes field. Fifteen minutes each, and someone who knows that customer. That’s fifteen hours a week.
The project automated the four hours.
There’s a second-order effect here that surprises people, and it’s why these projects get quietly abandoned rather than merely underperforming: automation concentrates the difficulty. The clerk used to work a mix, and a run of easy orders was a breather between the hard ones. Now the easy ones are gone and the day is nothing but exceptions, back to back, with the same headcount and the same hours. Harder work, and a system to blame for it. That is how you get a technically successful project that the department stops using within a quarter.
The honest version of this project is either much smaller — four hours a week is still four hours a week, take it and don’t call it a transformation — or much harder: aim at the exceptions, which means the system needs the frame contract, the customer’s habits, and the shop-floor calendar. That’s a real project, and possibly a good one. It is not the project that got scoped in week one.
There’s a sponsor, but no owner
Nearly every stalled project has a sponsor: the MD, who read something convincing, or the IT lead, who was told to do something about AI. What it lacks is an owner — someone whose own numbers improve if this works, and who will therefore fight for it when it’s eighty percent done and still annoying.
That gap has a specific shape in the Mittelstand. The MD wants it. IT builds or buys it. The Vertriebsinnendienst or the Serviceleitung — the people whose process is actually changing — hear about it when it’s ready for testing. They didn’t ask for it, they don’t own it, and they’re the ones absorbing the disruption. So they don’t sabotage it. Something worse happens: they’re perfectly polite about it, they use it when someone’s watching, and it dies of indifference. No meeting is ever held to cancel it. It just stops being mentioned.
An AI project is a process change with software attached. If nobody who owns the process owns the project, you have bought software for a change that was never going to happen.
The model works, and nobody decided what happens when it doesn’t
Picture a regional insurer classifying and routing incoming post — claims, policy changes, complaints. The pilot lands at 85% correct, everyone agrees that beats the temp who covered it in August, and it ships.
The question the demo never raised: what happens to the other 15%? A misrouted complaint doesn’t announce itself. It sits in the wrong queue for three weeks and returns as an angry phone call or a missed Frist. The manual process was no more accurate — but it had something the model didn’t: a human who felt uncertain, and who carried the odd envelope over to someone else.
That’s the part worth taking away. For anything with a human near the loop, a system that knows when it doesn’t know beats a system that is simply more accurate. A classifier that routes 80% confidently and drops 20% into a review queue is a better production system than one that routes 100% at 85% accuracy — because the first one’s errors are contained and the second one’s are loose in your process, aging. Accuracy is what you demo. Calibration is what you operate.
The Data Was There. Nobody Opened It.
The classic data failure isn’t dirty data or too little of it. Those are at least visible, and they’re the subject of preparing your data for AI. The failure that kills projects is subtler: the data exists, so everyone assumes it’s usable, and nobody actually looks inside until month three.
Say a machine builder wants to predict which service call needs which spare part. The ERP has a Fehlerursache field on every service order going back nine years — tens of thousands of rows. That looks like a dataset.
Then you open it. Sixty percent of the rows say “Sonstiges”, because that’s the first entry in the dropdown and the fitter is completing the form on a phone, in a car park, at half six. A good chunk of the rest is free text in a shorthand that two long-serving fitters can read and nobody else. The real diagnosis was never in that field. It was in the fitter’s head, and maybe on the paper copy.
The field was designed to close the service order, not to describe it. Data collected to close a process does not describe the process — and nearly every internal dataset in a mid-sized company was collected to close something. This costs an afternoon to check in week one and half the budget to discover in month three. Open the actual table. Look at the actual values. Count the distinct ones.
How to Spot All of This in Week One
Every failure above is cheap to detect and expensive to discover. Detecting them needs no data scientist and no architecture diagram — it needs five questions, none of which are about AI:
- Who does this work today, and can I sit with them for a day?
- Take the last 200 cases: how long did each one take? Are the hours in the routine cases or the awkward ones?
- Whose numbers improve if this works — and do they know the project exists?
- When the system is wrong, who notices, how long does that take, and what does the mistake cost?
- Show me the field you want the model to learn from. Not the schema. The values.
If you can’t get straight answers to these, that is the answer, and it’s worth more than another workshop. This is also the difference between a pilot that produces a verdict and one that produces a demo — how to run an AI pilot covers the mechanics of getting to a real go/no-go.
What This Doesn’t Protect You From
Two honest caveats, because a checklist that claims to prevent all failure is just a different kind of sales pitch.
First, none of this saves you from a genuinely hard technical problem. Some things simply aren’t ready, and no amount of good scoping changes that — reliable extraction from hand-annotated 1980s Werkstattzeichnungen will burn your budget however carefully you set it up. That’s a separate judgement, and it’s the one you hire an engineer for rather than a workshop.
Second, these failures aren’t equally fatal. A missing owner is recoverable mid-project, if the MD is genuinely willing to hand it over rather than just say so. Bad ground-truth data is sometimes recoverable by starting to record it properly now and building next year. The exception problem usually isn’t: if the hours are in the long tail and the project is aimed at the head, no amount of engineering makes the arithmetic work. That one has to be caught before you start — which is exactly why an afternoon of counting is the highest-return work in the whole project. Note that this is a different question from what an AI project costs: a well-aimed project can still be expensive, and a cheap one aimed at nothing is still wasted.
Where Tippel Fits
None of this is an argument against doing AI projects. It’s an argument for spending the first week on questions instead of architecture, and for being willing to hear that the answer is no.
That is what the AI Readiness Check is: a paid, timeboxed week or two that runs exactly these questions against your real process and your real data, and ends in an honest go/no-go rather than a proposal. Sometimes the useful outcome is “not this use case — the one next to it is where the hours are.” That answer costs a fraction of finding out in month six, and the fee is credited if you go ahead. If you’d rather talk it through first, get in touch.