Case study
How Kairos stamps fabricated citations — a probe, not a promise
LLM research tools invent references. Kairos runs a deterministic gate that adjudicates every [PMID:/DOI:/KG:] marker before it reaches the page — and a standing probe that feeds it fabricated citations on every change. The probe has never let a fabrication pass. Here is the mechanism, the record, and the boundary.
The failure mode: models write citations, then hope
A research assistant that retrieves evidence and then synthesizes an answer can emit a plausible-looking [PMID:12345678] that was never in the evidence pool. The model writes the marker first and checks the pool second — or not at all. In our own audits, fabricated markers were a recurring failure class: they look like provenance while being the opposite of it.
The gate: every marker adjudicated before it renders
Since v0.20.0, every citation marker in free-form reasoning text passes through a deterministic adjudication before rendering: markers found in the evidence pool are kept; markers that are out-of-pool but verifiably real (checked against NCBI) are kept and labeled external; hallucinated or unparseable markers are stripped from the text with the sentence's meaning preserved. The gate only removes false markers — it never breaks the dossier, and it never silently rewrites your literature. It is on in production, and the dossier shows what was removed.
The probe: we feed it fabrications on purpose
A gate is a claim until something tries to break it. Our standing adversarial probe builds a deliberately poisoned document — five fabricated citations and a fabricated 12 g/L titer — and runs it through the review layer. Healthy means every fabricated item is stamped WARN or FAIL, never PASS. The probe runs in CI on every change to the review layer, and it prints its verdict and exit code into the build log: if a fabricated claim ever receives PASS, the build fails — that failure is the alert.
The record
The two probes of 2026-07-27 were both caught — session verdict FAIL, fabricated claim stamped. The standing probe that entered CI on 2026-09-20 was healthy on its first run: all five fabricated citations and the fabricated 12 g/L titer were stamped unverified, never PASS. This page is a snapshot; the probe re-runs on every change, and the build log is the running record.
What this does not cover
The gate adjudicates citation markers, not prose: an uncited sentence can still be wrong. The review layer checks whether a cited source actually supports a claim — same organism, same target, entailed or contradicted — not whether the claim is true in the world. And no dry-lab tool substitutes for wet-lab validation. We state this on every page because a probe that sounds like a guarantee is just another fabrication.