Why a Hallucinated Citation Is a Landmine
A general-purpose chatbot asked whether an AI system meets the EU AI Act will give you a fluent, confident, well-structured answer — and somewhere in it will be a reference like “under Article 14(4) the provider must…” where the sub-paragraph either says something else or is not there. The prose reads exactly as convincingly whether the citation is real or invented. That is the whole danger: a language model hallucinates most dangerously not when it is obviously wrong, but when it is wrong in the register of an expert.
In most applications a wrong reference is an annoyance. In a compliance report it is a liability you sign your name under. Someone acts on that report — a data protection officer, an auditor, a board deciding whether a system can go live. A citation that does not resolve, or resolves to text that says the opposite, is not a small quality issue; it is the failure that discredits every correct finding around it. So when we built KomplAI — Tippel’s live checker for the EU AI Act, DORA and GDPR — the design started from the output we could not afford to ship, and worked backwards. Everything else is downstream of one rule: no legal reference reaches the report on the model’s word alone.
Why RAG Alone Doesn’t Close the Gap
The obvious first answer is retrieval-augmented generation: put the legal texts in a vector store, retrieve the relevant passages, and have the model answer from them. We do use retrieval, and you should too — a RAG pipeline grounded in EUR-Lex is far better than a model answering from training-data memory. But retrieval narrows the problem; it does not eliminate it, and the gap it leaves is exactly where compliance work lives.
Two failure modes survive retrieval. First, miscitation: the model retrieves the correct passage, reasons about it correctly, and then writes the wrong article number next to it — a transcription slip that no amount of retrieval quality prevents, because it happens after retrieval. Second, misreading: the model retrieves a passage and overstates it, treating a recital (which is interpretive context) as if it were an operative obligation, or reading a conditional duty as an absolute one. Retrieval put the right text in front of the model. It did nothing to guarantee the sentence the model then wrote is faithful to that text. If you want to understand why grounding helps but is not a guarantee, the mechanics are worth reading in full on how RAG actually works.
The lesson generalises past compliance. Retrieval controls what the model sees; it does not control what the model asserts. Anywhere the assertion has consequences, you need a step that checks the assertion itself — not the context it was drawn from.
Verify at Generation Time
The step we added is deliberately unglamorous: after the model drafts a finding with a citation, and before that finding is allowed anywhere near the report, the citation is resolved and re-checked programmatically. This is the part most demos skip because it adds work and slows things down. It is also the part that turns a plausible-sounding tool into one whose output you can defend.
Resolve every citation against EUR-Lex
The first gate is deterministic and boring, which is the point. The model’s output is parsed for every legal reference — “Art. 9(2)(a) AI Act”, “Recital 60”, a DORA article, a GDPR paragraph — and each one is normalised into a canonical identifier and looked up in the authoritative EUR-Lex text. If the reference resolves, we have the actual provision in hand for the next step. If it does not resolve — the article number does not exist, the sub-paragraph is out of range, the instrument is wrong — the finding does not get printed with an invented citation. It is flagged. A gap the user can see beats a fabrication they cannot.
The genuinely hard part here is not the lookup; it is the parsing and the versioning. Legal references arrive in loose, multilingual, abbreviated forms, and you have to map every variant onto the same canonical target. And EUR-Lex carries consolidated versions alongside the original Official Journal text, with article numbering that can shift as an instrument is amended — so “resolve” only means something once you have pinned which version you are resolving against. That plumbing is unglamorous and it is most of the work.
The adversarial re-check
Resolving a citation proves the article exists. It does not prove the finding is true. Article 11 being a real provision tells you nothing about whether the retrieved system actually breaches it. So the second gate is an adversarial pass: for each finding, a separate step is pointed at the resolved source text and the law and asked, in effect, to break the finding — does the cited provision actually support this conclusion, or is the claim an overreach, a misread recital, a conditional duty stated as absolute? A finding that survives being argued against is worth far more than one a compliant model was happy to agree with. It is the difference between “the model said so” and “the claim was tested against the text and held.”
What It Costs — and When It Pays
None of this is free, and it is worth being honest about the bill. You pay in latency and tokens: every finding triggers extra resolution and a second model pass, so a verified report costs roughly the report generated twice. You pay in complexity: a citation parser, a version-pinned corpus, a resolver, and an explicit flagging path are far more moving parts than “ask the model and format the answer.” And you pay in maintenance: EUR-Lex changes, instruments get amended, and keeping the mapping current is ongoing work rather than a one-time build. For KomplAI, this is why EU AI Act, DORA and GDPR reviews that used to be days of expert reading come back in minutes — but the minutes are only trustworthy because of the machinery behind them.
So the honest guidance is: this pattern is not for every LLM feature. If you are building a brainstorming assistant or a first-draft writer, verification-at-generation is overkill and the latency is not worth it. It earns its cost in exactly one situation — when a wrong factual claim has consequences. Compliance and legal are the obvious cases; medical, financial, and safety-critical outputs are the same shape. The test is simple: if a confident, well-phrased, wrong citation in your output would cause real harm or real liability, you need the deterministic checker. If it would just be mildly embarrassing, you do not.
The Reusable Pattern, and Where Tippel Fits
Strip away the compliance specifics and the pattern is portable: ground every factual claim an LLM makes in an authoritative source of truth, and verify it programmatically at the moment of generation. Retrieval decides what the model sees; a deterministic check decides what it is allowed to assert; an adversarial pass decides what survives scrutiny. The model’s job is to propose well. The system’s job is to make sure nothing it proposes reaches the user as fact until something other than the model has confirmed it. That is what “verifiable” means in practice, and it is the part a thousand thin wrappers skip.
KomplAI is the version of this we built for ourselves, in production, and would build the same way again. If you have an LLM system whose output someone has to stand behind — a report, an assessment, an answer that gets acted on — the same architecture applies, and it is most of what we do at Tippel. If you want to work out whether your use case genuinely needs verification-at-generation or would be over-engineered by it, that question is exactly what the AI Readiness Check is for — or just get in touch and describe the output you need to be able to defend.