A number that exists in no source
Generate the Q3 board summary from these 14 source documents.
‘Q3 revenue grew 23% to $4.2M, driven by enterprise expansion [finance-q3.pdf, p.4].’ — but finance-q3.pdf has three pages, and the only revenue line in it reads: ‘Q3 revenue: $3.8M, up 18% YoY.’
This is the most dangerous failure in the series, precisely because it doesn’t look like one. The number is plausible. The prose is confident. And there’s a footnote — so the reviewer’s eye relaxes, the figure walks into a board deck, and a decision gets made on a number the model invented. Retrieval did its job; the correct document was right there in the context. The model read it, ignored the real figure, made up a better-sounding one, and stapled a citation to it.
The escalation from Act III’s last lie is the disguise. The critic merely failed to catch an error. Here the system manufactures the error and forges the mark of credibility that sells it. The footnote isn’t evidence — it’s a costume. Lowering the temperature won’t undress it, and “add a grounding instruction” is what the last three teams tried. The fix lives at a specific seam, and the citation itself is the clue that tells you which one.
Why it happens
A citation the model writes is prose, not proof — until something checks that the cited source actually says it.
We collapse two different things into the word “cite.” One is attribution: this number was copied from that exact span, here’s the pointer. The other is what a language model does by default: produce text that has the shape of an attributed claim — a figure, a bracket, a filename, a page — because that’s what board reports look like. The second is generation, all the way down. The bracket is as invented as the number; it just happens to name a real file, which is what makes it lethal.
So when a grounded system fabricates, the citation tells you where to look. Retrieval clearly worked — the source is named, the source exists. So ask the usual question and watch it resolve fast: was the answer-bearing context present? Yes — the real figure was in the model’s context. It had the truth and emitted a falsehood. That puts the fault squarely on the generation side. Localize it.
Where does it break?
A bad answer splits one way: the model either never got the answer-bearing context, or itgot it and still failed. Answer that, and you've halved the search space. Then localize within the failing half.
- ingest
- chunk
- embed
- retrieve
- rerank
- assemble
- prompt
- generate
- post
Descend the pipeline
The real document was in context — so this is a generation-side failure. Descend: confirm the source, watch generation fabricate, then prove a verifier catches it.
retrieve✓ cleared Was the correct source document actually in the model’s context?
finance-q3.pdfwas retrieved and present in the prompt, its revenue line intact: “Q3 revenue: $3.8M, up 18% YoY.” The truth was available at generation time. This is not a retrieval miss — the model had the right number in front of it.generate✕ fault here Where does the number come from — copied from a span, or produced as fluent text?
Inspect the generation step. The model isn’t asked to extract figures and attribute them; it’s asked to write the summary. So it writes — and “23% to $4.2M” is simply a more board-pleasing sentence than the real one. Then, because board summaries carry footnotes, it generates a footnote:
“…grew 23% to $4.2M [finance-q3.pdf, p.4]” — the figure, the file, and the page are all sampled tokens. p.4 doesn’t exist; the real figure is on p.2. Nothing in this step ever opened the source to check.
The citation was never derived from a span — it was written, the same way the number was. The fault is the generation stage: unconstrained, it produces attributed-looking prose with no actual attribution underneath. The forged footnote is the fingerprint.
attribution check✓ cleared Cross-check — run a verifier that opens each cited source and confirms it contains the claim. Does it catch the fake?
Add one post-generation pass: for every
figure [source, page], fetch that span and check the number appears in it.Claim “$4.2M” vs finance-q3.pdf p.4 → page 4 does not exist; nearest figure on p.2 is $3.8M. ✗ Unsupported — claim dropped, real figure substituted.
The fabrication can’t survive contact with its own citation. That’s the whole point: a citation is only worth anything when something downstream actually follows it. The model’s vectors and reasoning were never the issue — the issue was an open-loop generation step that no one ever asked to prove itself.
Flip the fix
Theory's cheap. Take the same broken system and flip the fix yourself — watch the trace change and the eval scores move.
Generate the Q3 revenue line for the board summary. (source says $3.8M / +18%)
“Q3 revenue grew 23% to $4.2M.” — p.4 doesn’t exist; the source says $3.8M / +18%.
“Q3 revenue grew 23% to $4.2M [finance-q3.pdf, p.4].” — confident, cited, false.
Make it a standing check
The fix is two complementary clamps. Constrain generation to copy figures from retrieved spans and emit the span id it copied from — so a number can’t exist without a real source behind it. And verify every cited claim after generation, fetching the cited span and confirming it contains the value, dropping or flagging anything unsupported. Grounding stops most fabrications; verification catches the rest. But fixing this report is a bug fix. Closing the class is the teardown.
The failure mode is unverified attribution — confident, well-cited text that nothing ever checks against its sources. It’s invisible to every fluency and coherence metric, because the output is fluent and coherent; it’s just false. So instrument attribution as a hard gate:
- A cited-claim support test: extract every
(claim, source)pair from generated reports and assert the source contains the claim. Run it in CI on a fixed set, and at request time as a guardrail. Any unsupported citation fails the report, loudly. - Track fabrication rate — the share of figures with no supporting span — as a first-class metric, not a vibe. A report generator that scores beautifully on readability and 30% on this number is a liability, and only this number says so.
That closes the safeguards act: you’ve seen the reviewer wave a lie through, and the lie forge its own credentials. Every trust mechanism you bolt on becomes a place for the deception to hide. Which leaves one final question, and it’s the one this whole series has been walking toward — the instrument you’ve been told, five times, to build and trust. What happens when that is the thing that’s broken?
- Why does an LLM invent a number and still attach a real-looking citation?
- Because to the model the citation is just more text in the shape of a report — the figure, the filename, and the page number are all sampled tokens, none derived from an actual lookup. The footnote happens to name a real file, which is exactly what makes the fabrication convincing.
- How do you stop hallucinated citations in generated documents?
- Two clamps. Constrain generation so every figure must be copied from a retrieved span and carry that span's id, and add a post-generation verifier that opens each cited source and confirms it contains the claimed value, dropping anything unsupported. Grounding prevents most; verification catches the rest.
- How do you verify a cited claim that's a calculation, not a copied figure?
- Capture the inputs and the operation, not just the output: require the model to show the source figures it used and the arithmetic, then re-compute and check. A derived number is grounded only if every input traces to a real span and the math reproduces.