Self-verify — does the cited value actually appear at the cited URL?

About one in six of our shipped citations was cited-but-wrong. This is what closed that class.

Model output that includes a URL as evidence is often a lie. Not maliciously — the model has been trained to produce citation-shaped text, and citation-shaped text with a plausible URL is the local minimum for the loss function. The URL is real, the value at the URL is not what the model claims it is, and the reader has no way to know without opening a browser tab.

We measured the problem before we built the check. An internal audit of the runtime’s output found roughly 18% of cited scalar values in a cited-but-wrong class: a plausible value, a real product URL, and the value not on that page. A transposed spec. A sibling SKU’s number. A hallucinated figure wearing a real-looking citation.

services/runtime/lib/self-verify.mjs closes the gap without adding a network call. The run has already read its evidence — page text and structured data, held during the run and keyed by source URL. Before a cited scalar value ships, we join the citation back to that cached evidence and look for the value’s significant tokens in the cited page’s text. The rule is positive contradiction only: a value is nulled when we hold the cited page’s text and none of the value’s tokens appear in it. Absence of evidence is not contradiction — a cited URL whose page the run did not retain passes untouched, because unverifiable is not the same as wrong. And the check refuses to contradict against a near-empty or error page: it needs a minimum of cached text before it is allowed to null anything.

The matching has to be tolerant in the right directions. Numeric tokens carry both decimal forms, so 1,09 and 1.09 agree; digit separators collapse, so 1 500, 1,500, and 1500 agree; and a model-converted value matches its page-unit form only with the unit token adjacent — a value stored in kilograms matches the page’s 10 g, while a bare scaled number anchored to nothing matches nothing, because refutation power is the point and unanchored number matches would dissolve it. Each lane is a small pure function with its own fixture in the test file, and each lane can only add match opportunities — the safe direction for a guard whose worst failure mode is destroying a legitimately grounded value.

The check knows its own limit. It verifies citation fidelity — the value appears on the cited page — not item identity: a value scraped verbatim from the wrong product’s page passes by construction. A separate lane closes that hole by requiring the run’s own SKU or brand anchor in the evidence window around the matched value; when the anchor is missing it downgrades confidence and never nulls, and it stays off by default until it clears the same A/B gate every other mechanism passes.

The base check runs on by default in the live engine, and it is pure string work over text already in memory — no fetch, no model call. When it nulls a value, the record carries an audit note saying so. A blank with a reason beats a number we cannot stand behind.

Trust is a working feature. It is built out of small checks that keep firing.

The code is published for reading: packages/self-verify in bargo-lv/primitives — source and tests, all rights reserved.

Correction (2026-08-20): an earlier version of this note described the check as re-fetching the cited URL at ship time, with a latency cost attached; the shipped design has never fetched at ship time — it joins against page text the run already read — and the latency figure is withdrawn with the claim.