The Hard Part Isn't Reading the Document
Show a modern model a scanned supplier invoice and it will give you the invoice number, the net total and the VAT rate. That demo takes an afternoon, and it is the reason so many document AI projects get approved and then quietly go nowhere. Reading is the part that got solved. What did not get solved is trust: the model returns the same confident JSON whether it read the total correctly or hallucinated a plausible number off a smudged fax.
This matters because the value of document processing is not accuracy — it is the share of documents that never touch a human. If a machine builder's accounts payable team handles 2,000 supplier invoices a month and the system extracts every field at 95% accuracy but cannot tell you which 5% are wrong, someone still has to check all 2,000. You have bought nothing. A system at 90% accuracy that reliably flags the documents it got wrong lets 90% flow straight through, and that is the whole return.
First, Check Whether You Need AI at All
Before anyone builds an OCR pipeline for invoices, open the PDFs and look inside them. Since 1 January 2025 every domestic B2B company in Germany must be able to receive structured e-invoices, with the obligation to issue them phasing in from 2027 for companies above €800,000 turnover and from 2028 for everyone. Your inbound invoice mix is shifting under you while you scope the project.
Two formats matter. XRechnung is pure XML. ZUGFeRD (and its French twin Factur-X) is a hybrid: a PDF/A-3 that looks like an ordinary invoice but carries the complete invoice data as an embedded XML attachment, both conforming to the EN 16931 semantic model. I have watched teams run a ZUGFeRD PDF through OCR and a vision model — reconstructing, at cost and with error, data that was sitting in an attachment inside the same file, exactly correct, free to parse.
So the first stage of any inbound document pipeline is not a model. It is a check: does this PDF have an embedded EN 16931 XML? If yes, parse it and skip everything else. That branch is deterministic, 100% accurate, and needs no review queue. AI is for the remainder — the scans, the emailed PDFs from suppliers who will not move for years, the delivery notes and order confirmations that no standard covers. Build the boring branch first and the AI branch gets smaller, cheaper and easier to defend.
The Shape of a Pipeline That Holds
What survives production is not one clever prompt over a document. It is a chain of narrow stages, each of which can be measured and each of which can refuse.
OCR sets the ceiling nothing else can raise
Every downstream stage inherits the text layer's mistakes, and a language model does not degrade gracefully on bad input — it fills the gap with something plausible. A "1" lost to a fax artefact does not produce an error; it produces a confident wrong amount. Tesseract is fine on clean digital text and unreliable the moment there are table borders, stamps or a skewed scan. Cloud services like Azure Document Intelligence, AWS Textract or Google Document AI are markedly better at layout and tables, and they mean shipping supplier invoices to a US-headquartered processor — a decision to make deliberately, not discover in an audit. On-premise, PaddleOCR and docTR are credible, as are vision-language models that skip the separate OCR step entirely, at meaningfully higher hardware cost. Test candidates on your ugliest 200 documents, not on a clean sample.
Classify before you extract
The instinct is one prompt that reads any document. Resist it. Route first — invoice, credit note, delivery note, order confirmation, dunning letter — then extract with a schema written for that type. Classification is the cheap, boring, robust half: few classes, easy to measure, easy to correct. It also gives you an honest "unknown" class, which is how documents you never anticipated get parked instead of silently mangled into an invoice shape. Extraction errors are expensive and quiet; classification errors are cheap and loud. Push as much of the decision as you can into the loud half.
Structured output guarantees shape, not truth
Constrained decoding against a JSON Schema — supported natively by most serving stacks now — is genuinely useful. It removes a whole class of parsing failures and guarantees your net total is a number and your date is a date. It is also where teams get comfortable too early. Schema-valid JSON that says €1,847.00 when the invoice says €1,347.00 is perfectly valid and completely wrong. Validity is a property of the output format; correctness is a property of the world. And you rarely need to invent the schema: for invoices, §14 UStG defines the mandatory fields and EN 16931 defines the semantics. Use them.
Confidence Is Not What the Model Says It Is
Ask a model how sure it is and it will tell you 0.95, in the same tone whether it is right or wrong. Token probabilities are a measure of how ordinary the text looked, not of whether the field is correct. Building a review threshold on self-reported confidence produces a system that reviews the wrong documents.
Real confidence comes from agreement between independent checks that have nothing to do with the model. On an invoice, you have an unusual luxury: the document largely verifies itself, and your ERP verifies the rest. Do the line items sum to the net total? Does net plus VAT at the stated rate equal the gross? Does the VAT ID resolve to a known supplier in the master data, and does the IBAN match the one on file for that supplier? Does the PO number exist, and is the amount within tolerance of the goods receipt? Each check is deterministic and cheap. A document where all of them agree is safe in a way no confidence score can express — and when one disagrees, you not only know to route it, you know which field to show the reviewer first.
This is also the honest answer to "what threshold should we use?". You do not pick a number from the model. You define the error budget, run the checks in shadow mode against what your team actually booked, and read off where the line falls. That comparison against a real baseline is the same discipline any serious estimate of cost and benefit depends on.
Where It Meets the ERP
The extraction is a few weeks. The integration is the project. A booking in SAP, Dynamics, Sage or DATEV is not a JSON field — it is a posting against a cost centre and an account, tied to a purchase order and a goods receipt, in a system that will be audited. Three constraints deserve naming before you commit to a date. Postings must be idempotent, because the same invoice will arrive twice, once by email and once by post, and duplicate detection needs a real key rather than a filename. Corrections must be reversals, not edits: GoBD requires that a booking, once made, remains traceable and unalterable, so an AI that silently rewrites a posting is a compliance defect and not just a quality issue. And the system needs a Verfahrensdokumentation describing how documents are processed and how errors are handled — writing it forces you to answer questions the project would otherwise leave vague.
Run it in shadow mode first, for weeks, posting nothing. Let it produce its proposal alongside what the team actually books, and compare. Shadow mode is the cheapest measurement you will ever take, and it is where you find out that some suppliers put the order number in the reference field, others bury it in the line item text, and one puts it nowhere except the PDF filename. That kind of discovery is why the data work usually outweighs the model work.
What I Wouldn't Do
I would not start with contracts. They are the use case everyone asks for and the worst first project: low volume, high variance, no arithmetic to check against, and consequences that make anything less than human review unacceptable. The economics are the opposite of invoices, where volume is high, the schema is fixed by law, and the document verifies itself. Contracts are a fine second project once the pipeline and the review workflow exist. As a first one, they teach you nothing cheaply.
I would not fine-tune a model as an opening move. Fine-tuning is a plausible answer to "our documents are genuinely unusual", but it is almost never the reason a pipeline is underperforming — bad OCR, a missing document type and a schema that does not match the ERP account for most of it, and all three are cheaper to fix. I would also not let anything post unattended in month one, however good the numbers look, and I would not build a review UI that shows a reviewer a form of thirty fields when three are in question. If a human has to re-read the whole invoice to check the machine, the machine saved nothing. Deciding where a use case like this sits against the others on your list is its own question — that is what the use case survey is for.
Where Tippel Fits
Document processing is the use case I most often recommend starting with, and the one where I most often argue for a smaller scope than the client came in with: one document type, one inbound channel, shadow mode, an honest straight-through rate. That is close to the AI Readiness Check — a paid, timeboxed piece of engineering on your real documents, ending in a number and a go/no-go rather than a demo, with the fee credited if you build. On-premise is a normal answer here rather than an exotic one; supplier invoices are exactly the data that is awkward to send abroad, and the build is done directly by the engineer who scoped it.
If you have a stack of invoices, orders or delivery notes that someone types into a system by hand, that is a good conversation to have. Get in touch and bring your ugliest scans.