Status: Design draft (pre-codex-review)
Date: 2026-05-12
Target release: v3.7.3
Scope: Two orthogonal additions to v3.7.1/v3.7.2 trust-provenance — (L3-1) per-citation quote-or-locator gate, and (L3-2) contaminated-source advisory signals on literature_corpus[] entries.
Precedent doc: 2026-04-30-ars-v3.6.8-trust-provenance-and-drift-transparency-spec.md (D1–D5), 2026-05-10-ars-v3.7.2-trust-provenance-hardening-spec.md (H1–H2)
External motivation: Zhao, Wang, Stuart, De Vaan, Ginsparg, Yin (2026-05) "LLM hallucinations in the wild: Large-scale evidence from non-existent citations" (arXiv:2605.07723) — 146,932 hallucinated citations estimated for 2025 alone across arXiv / bioRxiv / SSRN / PMC; inflection point mid-2024; 85.3% of preprint hallucinations survive into the published record; Google Scholar increasingly indexes citation-only entries with no real publication backing them.
v3.6.8 / v3.7.1 / v3.7.2 hardened the trust-chain layer between user, source artifact, and AI-produced descriptions. The frontmatter trust-strength schema (source_acquired / source_verified_against_original / human_read_source), the 4-cell finalizer matrix, the Two-Layer Citation Emission discipline, and the wrapper-side Scope Report enforcement together establish a defensible answer to "is this reference real, is its description correct, and has anyone actually read it?"
What that stack does NOT yet answer is "does the cited reference actually support the claim the prose is making?" — what the external literature calls citation faithfulness or claim-citation alignment. Three layers exist:
source_verified_against_original boolean.Two empirical anchors confirm L3 is the load-bearing gap:
(a) Zhao et al. 2026-05 (arXiv:2605.07723). A corpus-scale audit of 111M references across 2.5M papers finds: - A sharp post-2024-mid rise in non-existent (L1) citations, estimated at 146,932 fabricated references in 2025 alone across four major corpora. - Existing safeguards leak: arXiv moderation catches 21.2%, bioRxiv→PMC pipeline lets 85.3% of preprint hallucinations through to the published record. - Google Scholar increasingly indexes "citation-only entries" — references that cannot be linked to any real publication but already appear as references in other papers — creating a feedback loop that contaminates downstream retrieval. - Section Discussion, paragraph 7: "The harder problem — detecting claims that misrepresent the content of real sources, or hallucinated assertions embedded in unstructured prose — remains far less tractable, and the gap between what verifiers can detect and what LLMs can produce is wide and probably widening." This is L3 in the authors' own words.
(b) ARS 2026-04-30 production session (cited in v3.6.8 §1). The session found a bibliography_v2.md derivative artifact containing a fabricated quote attributed to D14 (OECD/UNESCO 2005). The fabricated quote was caught only because the user manually retrieved the original PDF and grep'd for the quoted phrase — i.e., a manual L3 check. v3.6.8 §3.1 D1 (frontmatter trust-strength split) prevents the fabricated quote from being silently amplified through downstream prose, but does not by itself surface the fabricated quote. The downstream synthesis_agent rule "Verbatim quotes only within the verified phrase boundary" (line 170) names the L3 discipline but is not mechanically enforceable in current shape — no finalizer or gate verifies that an in-text citation carries a verifiable quote anchor.
(c) Retrieval-side contamination. Zhao et al. Fig. 3c documents a steep rise in "citation-only" Google Scholar entries since 2024 — non-existent references that nevertheless appear as standalone bibliographic entries in Scholar after enough citing-paper accumulation. ARS retrieval_agent currently treats Semantic Scholar / OpenAlex / Google Scholar results as authoritative for the existence check. Post-2024-mid this assumption no longer holds for the long tail.
v3.7.3 therefore adds two orthogonal protections:
- L3-1 (claim faithfulness locator): every in-text citation MUST carry either an extracted verbatim quote (≤25 words from the cited source) OR a precise locator (page / section / paragraph identifier in the cited source). Bare citations without either are MED-WARN-NO-LOCATOR and refused by the terminal gate.
- L3-2 (contaminated-source advisory): literature_corpus[] entries gain an optional contamination_signals object carrying two boolean fields — preprint_post_llm_inflection (year ≥ 2024 AND venue type is preprint server) and semantic_scholar_unmatched (Semantic Scholar lookup returned no match by title or DOI). These signals are advisory — they surface in the cite-time finalizer report but do not by themselves block emission. The corresponding marker is LOW-WARN-CONTAMINATED-SOURCE, which is acknowledgeable via /ars-mark-read <slug> per v3.6.8 §3.6 precedent.
v3.7.3 explicitly does NOT attempt the full L3 audit (running an LLM-as-judge against ref full-text for each (claim, ref) pair). That stays v3.8+ scope. v3.7.3 is the locator layer — the minimum infrastructure that makes the full L3 audit possible later by giving each citation a stable anchor to verify against.
Close two attack surfaces orthogonal to v3.7.1/v3.7.2:
L3-1 locator gate. Every in-text citation emitted by synthesis_agent, draft_writer_agent, report_compiler_agent MUST be accompanied by a structured quote-or-locator anchor. The Two-Layer Citation Emission (v3.7.1 §3.3 Step 1) gains a third layer (locator). The finalizer 4-cell matrix extends to 5-cell along a new axis. Terminal gate refuses MED-WARN-NO-LOCATOR markers.
L3-2 contaminated-source advisory. Adapter-produced and AI-produced literature_corpus[] entries gain optional contamination_signals fields. bibliography_agent sets preprint_post_llm_inflection at ingest time based on year + venue type. bibliography_agent calls Semantic Scholar (per references/semantic_scholar_api_protocol.md) at acquisition time and sets semantic_scholar_unmatched when no hit. The cite-time finalizer surfaces these as LOW-WARN-CONTAMINATED-SOURCE markers.
Target outcome: the v3.6.8 fabricated-quote incident, replayed under v3.7.3, surfaces a MED-WARN-NO-LOCATOR at the finalizer pass (because the fabricated quote either lacks a page locator or carries one the audit cannot verify). The terminal gate refuses output until the user re-emits the citation with a valid (<kind> ≠ none) anchor or removes the citation. NO-LOCATOR is NOT acknowledgeable via /ars-mark-read — see §3.1 Terminal gate extension + Q5 (resolved).
literature_corpus[] entries without contamination_signals validate cleanly (optional field). The bibliography_agent ingestion path adds the field for new entries; old entries can be migrated via a follow-up batch script if the user wants, but no automatic upgrade fires.Pattern: When synthesis_agent / draft_writer_agent / report_compiler_agent emit an in-text citation under v3.7.1 Two-Layer Citation Emission, the citation carries only the slug reference <!--ref:slug-->. No structured anchor records which passage in the cited source supports the surrounding claim. The synthesis_agent rule "Verbatim quotes only within verified phrase boundary" exists in prose (synthesis_agent.md:170) but is not mechanically extractable and is not enforced by any gate. An author or downstream auditor cannot mechanically verify that a claim is anchored to a specific passage in the cited source.
Mechanism — three sub-axes:
(A) Emission axis. Current Two-Layer Citation has form Smith (2024) <!--ref:smith2024-->. No slot for a quote or page locator. The agent can write any claim adjacent to the citation; the slug is opaque to claim-content alignment.
(B) Resolution axis. The cite_provenance_finalizer_agent's 4-cell matrix reads only frontmatter trust state. It cannot detect "claim-without-anchor" because the missing data is in the prose, not the frontmatter.
(C) Gate axis. Terminal gate (cite_provenance_terminal_gate_agent for deep-research, formatter_agent for academic-paper/pipeline) only inspects markers for ok / LOW-WARN-acknowledged status. A bare citation with no claim-anchor passes the gate today.
Protection — v3.7.3 Three-Layer Citation Emission:
Extend the v3.7.1 Two-Layer form by adding a third hidden layer carrying the locator:
<visible layer> <!--ref:slug--><!--anchor:<kind>:<value>-->
Where <kind> ∈ {quote, page, section, paragraph, none} and <value> is the corresponding payload:
| kind | value format | example | semantics |
|---|---|---|---|
quote |
URL-encoded verbatim text from cited source, ≤25 words | quote:When%20publishers%20bypass... |
strongest anchor; can be grep'd against retrieved source |
page |
page number or range, e.g. 12 or 12-14 |
page:12-14 |
medium anchor; locates passage but does not verify content |
section |
section identifier, e.g. 3.2 or Results |
section:3.2 |
medium anchor; relies on cited source's stable section labels |
paragraph |
paragraph index from start of section, e.g. 3 |
paragraph:3 |
weakest structural anchor; brittle to revision |
none |
empty | none: |
explicit no-anchor declaration; forbidden in production, used only by bibliography_agent during ingest before any prose exists |
Three firm rules:
synthesis_agent / draft_writer_agent / report_compiler_agent emission, every visible citation MUST be followed by both <!--ref:slug--> AND <!--anchor:<kind>:<value>--> where <kind> ≠ none. Emitting <!--anchor:none:--> in production prose is a contract violation detected by the finalizer.<kind> = quote, the URL-decoded value MUST be ≤25 words by whitespace split (per v3.6.7 word_count_conventions.md). Quotes exceeding 25 words MUST be replaced by page or section locator pointing to the passage.<!--anchor:...--> from its own working knowledge of the cited source. It MUST NOT read frontmatter to discover anchor candidates (v3.6.7 partial-inversion discipline). When the agent cannot produce a verifiable anchor, it emits <!--anchor:none:--> and the finalizer marks the citation MED-WARN-NO-LOCATOR (gate-refused).Finalizer matrix extension (4-cell → 5-cell):
The v3.6.8 §3.3 4-cell matrix gains a precedence-zero check on <!--anchor:...-->. If <kind> = none, the resolution is MED-WARN-NO-LOCATOR regardless of other trust state. Otherwise the v3.6.8 4-cell matrix proceeds:
| Anchor present | source_acquired | source_verified | human_read | Resolution |
|---|---|---|---|---|
no (none) |
— | — | — | MED-WARN-NO-LOCATOR (precedence-zero; supersedes other cells). Marker text: [UNVERIFIED CITATION — NO QUOTE OR PAGE LOCATOR]<!--ref:slug--> |
| yes | false | — | — | HIGH-WARN-NO-ORIGINAL (unchanged from v3.6.8) |
| yes | true | false | — | MED-WARN-NOT-CROSS-CHECKED (unchanged) |
| yes | true | true | false | LOW-WARN (unchanged, plus contamination-signal annotation if any contamination_signals present — see §3.2) |
| yes | true | true | true | ok (unchanged) |
NO-LOCATOR is MED severity (not HIGH) because the citation may still point at a real verified source; it is just unanchored. The fix is locator emission, not source acquisition. Treating NO-LOCATOR as HIGH would conflate two distinct defects.
Terminal gate extension:
formatter_agent (academic-paper / pipeline) and cite_provenance_terminal_gate_agent (deep-research) MUST refuse to emit final output when any [UNVERIFIED CITATION — NO QUOTE OR PAGE LOCATOR] marker remains, AT THE SAME PRECEDENCE as the v3.6.8 HIGH-WARN-NO-ORIGINAL refusal. Remediation path: the user re-emits the citation with a valid (<kind> ≠ none) anchor. /ars-mark-read does NOT clear NO-LOCATOR — the precedence-zero rule stops BEFORE applying the 4-cell matrix that consumes human_read_source, so an unread/read distinction cannot mutate the resolution. This asymmetry is intentional: a locator is a structural property of the prose, not an evidence-state property of the source. v3.7.3 codex review P2-2 closure (2026-05-12); see also Q5 (resolved).
Pattern: bibliography_agent and retrieval_agent accept Semantic Scholar / Google Scholar / OpenAlex / publisher lookup results as authoritative for the existence check. Post-2024-mid, two distinct contamination vectors are documented in the external corpus-scale literature (Zhao et al. 2026-05 Fig. 1a-d, Fig. 3c):
Neither signal individually proves a specific reference is hallucinated. Both are advisory. But surfacing them at cite-time gives the user the information needed to decide whether to acquire the source (source_acquired: true path), drop the citation, or acknowledge the risk and proceed.
Mechanism — two ingestion-time checks:
(A) preprint_post_llm_inflection boolean. Set by bibliography_agent at ingest time. Computed as year >= 2024 AND venue is preprint server (arXiv / bioRxiv / medRxiv / SSRN / Research Square / Preprints.org / ChemRxiv / EarthArXiv / OSF Preprints / TechRxiv). The venue list is closed (per §6 maintenance note). The threshold year 2024 follows Zhao et al. inflection-point analysis; bibliography_agent prompt cites the source so future amendments to the threshold are traceable. v3.7.3 gemini review F6 expanded the list from 6 to 10 venues.
(B) semantic_scholar_unmatched boolean. Set by bibliography_agent when a Semantic Scholar API lookup (per references/semantic_scholar_api_protocol.md) returns no match by either DOI or title. The check fires only when obtained_via is NOT manual (a user-curated entry is exempted; the user has already vouched for it). The lookup uses the existing protocol's title-similarity threshold; reaching no-match means S2 has no indexed paper matching the entry.
Protection — v3.7.3 contamination signals schema + cite-time surfacing:
literature_corpus_entry.schema.json gains an optional contamination_signals object:
contamination_signals:
preprint_post_llm_inflection: true | false # year >= 2024 AND venue is preprint server
semantic_scholar_unmatched: true | false # S2 lookup returned no match
Both fields are optional within the object; the entire contamination_signals object is optional at the entry level. An entry with the object absent is treated as "signals not yet computed" (no advisory fires). An entry with the object present but both fields false is treated as "signals computed, no contamination evidence" (no advisory fires).
Finalizer extension:
When the 5-cell matrix resolves to ok or LOW-WARN but the entry's contamination_signals has any field true, the finalizer adds an annotation suffix to the resolved marker:
| Base resolution | contamination_signals | Annotated marker |
|---|---|---|
| LOW-WARN | none / both false | <!--ref:slug LOW-WARN--> (unchanged) |
| LOW-WARN | preprint_post_llm_inflection: true | <!--ref:slug LOW-WARN CONTAMINATED-PREPRINT--> |
| LOW-WARN | semantic_scholar_unmatched: true | <!--ref:slug LOW-WARN CONTAMINATED-UNMATCHED--> |
| LOW-WARN | both true | <!--ref:slug LOW-WARN CONTAMINATED-PREPRINT+UNMATCHED--> |
ok |
any contamination flag | <!--ref:slug ok CONTAMINATED-...--> (same suffix shapes) |
Contamination annotations are advisory — they do not change the gate decision. ok CONTAMINATED-... still passes the terminal gate; LOW-WARN CONTAMINATED-... is acknowledgeable via /ars-mark-read <slug> exactly like plain LOW-WARN.
Two firm rules:
<!--anchor:none:--> and is rejected by the production gate per R-L3-1-A).none.contamination_signals field is owned by bibliography_agent (AI-set) per the existing AI-owned fields convention; adapters MAY set it but are not required to.shared/contracts/passport/literature_corpus_entry.schema.json — no change in this step (anchor lives in prose, not entry frontmatter).deep-research/agents/synthesis_agent.md — extend "Two-Layer Citation Emission (v3.7.1)" section to "Three-Layer Citation Emission (v3.7.3)". Add anchor format spec, 5 kind enum, R-L3-1-A/B/C firm rules. The PATTERN PROTECTION (v3.6.7) block stays byte-equivalent; the new content is in a separate ## Three-Layer Citation Emission (v3.7.3) heading following the existing Two-Layer heading.academic-paper/agents/draft_writer_agent.md — same extension. Mirror the synthesis_agent section.deep-research/agents/report_compiler_agent.md — same extension.deep-research/agents/cite_provenance_finalizer_agent.md (and academic-paper peer if separate) — extend 4-cell matrix to 5-cell. Add NO-LOCATOR precedence-zero check. Update output marker shapes.deep-research/agents/cite_provenance_terminal_gate_agent.md — extend refusal-pattern list with [UNVERIFIED CITATION — NO QUOTE OR PAGE LOCATOR].academic-paper/agents/formatter_agent.md — same extension.shared/contracts/passport/literature_corpus_entry.schema.json — add optional contamination_signals object (two optional booleans). Update inline description with cross-reference to this spec doc.scripts/check_literature_corpus_schema.py — accept the new field (additive; existing fixtures without the field stay valid).deep-research/agents/bibliography_agent.md — add a new section ## Contamination Signal Computation (v3.7.3) describing the two checks. The check fires at ingestion time, not retroactively. The preprint venue list is closed and enumerated inline.cite_provenance_finalizer_agent.md — extend marker shape to include CONTAMINATED-... suffix when entry has any contamination_signals.*: true. Suffix is annotation only (does not change gate decision).CHANGELOG.md — add v3.7.3 entry under ## [Unreleased]..claude/CLAUDE.md — add ## v3.7.3 Key Additions section.ARS v3.7.3 ships when:
literature_corpus_entry with contamination_signals: {preprint_post_llm_inflection: true, semantic_scholar_unmatched: false} passes check_literature_corpus_schema.py. Entry without the field also passes (backward compat).<!--ref:slug--> + <!--anchor:quote:...--> for at least one citation, <!--anchor:page:...--> for another. No bare Two-Layer citation in production prose.contamination_signals.preprint_post_llm_inflection: true produces a finalizer output where the resolved marker for that entry carries the CONTAMINATED-PREPRINT suffix. The terminal gate emits the output (advisory does not block).feedback_codex_workflow_consolidated.md review-vs-challenge split, review confirms contract; challenge probes contract gaps in a follow-up round).claim_ref_alignment_audit_agent that runs LLM-as-judge over (claim, ref full-text) pairs for a sampled subset of citations. Anchors emitted by v3.7.3 become the input to this audit. Open question: how to sample (random N%? all quote: kind? all post-2024 preprints?). Defer to v3.8 spec.literature_corpus[] and computes contamination_signals for entries that lack the field. v3.7.3 ingests new entries with the field; legacy entries can be migrated by re-running bibliography_agent or by writing this script. Defer.bibliography_agent.md. New preprint servers entering the ecosystem require a spec amendment. Track as a recurring maintenance task.docs/design/2026-04-30-ars-v3.6.8-trust-provenance-and-drift-transparency-spec.md) — D1 frontmatter trust-strength split, D3 cite-time provenance check, finalizer/gate architecture.docs/design/2026-05-10-ars-v3.7.2-trust-provenance-hardening-spec.md).docs/design/2026-04-29-ars-v3.6.7-downstream-agent-pattern-protection-spec.md) — partial-inversion discipline that R-L3-1-C inherits.feedback_ars_bibliography_agent_hallucination_patterns.md — 5 upstream patterns (JSIE prestigious anchoring, etc). L3-2 contamination signals are orthogonal but complementary.feedback_ars_verification_provenance_layer.md — three-field trust-strength split that L3-1 extends with a fourth axis (locator presence).feedback_ars_audit_scope_disclosure.md — audit-scope disclosure precedent that L3-2 advisory shape inherits (surface, don't block).feedback_codex_workflow_consolidated.md — review-vs-challenge split governing the codex round count target.feedback_complex_spec_review_inventory_pattern.md — invariants index + state inventory discipline for review rounds.| Risk | Mitigation |
|---|---|
Agents emit <!--anchor:none:--> reflexively to satisfy the contract, defeating the gate's purpose. |
R-L3-1-C mandates that none triggers MED-WARN-NO-LOCATOR (gate-refused). Agents that reflexively emit none produce gate-refused output; the user notices immediately. The contract is shaped so "emit none" is not a free pass. |
quote: anchors fabricated by the agent (quote that does not appear in the cited source). |
v3.7.3 does not detect this. v3.8 L3 audit closes it by grep-against-retrieved-source. v3.7.3 establishes the anchor channel; v3.8 verifies anchor content. |
Page numbers in page: anchors hallucinated. |
Same — v3.7.3 establishes the channel; v3.8 verifies. v3.7.3 is a necessary precondition for v3.8, not a substitute. |
| Contamination signals fire on legitimate post-2024 preprints, creating noise. | Advisory only. User retains discretion via /ars-mark-read. Per Zhao et al., the entire post-2024 preprint corpus DOES carry elevated baseline risk — the noise IS the signal. |
Semantic Scholar API rate limit / downtime making semantic_scholar_unmatched unreliable. |
Existing semantic_scholar_api_protocol.md covers degradation. When the API is unavailable, bibliography_agent omits the semantic_scholar_unmatched field rather than setting it to false (which would imply "checked and found"). Absence ≠ negative confirmation. |
| Spec-doc-only delivery (no implementation in v3.7.3 PR-A) creates spec/impl drift. | This spec is committed alongside the agent-prompt and schema changes in the same PR (PR-A). Lint additions in PR-B. CHANGELOG ties them. No spec-only release. |
| Three-Layer Citation breaks existing v3.7.1 finalizer for non-upgraded prose. | Backward compat: a Two-Layer Citation with no <!--anchor:...--> is treated as anchor=none (MED-WARN-NO-LOCATOR). Existing v3.7.1 corpus prose gates as MED-WARN until re-emitted under v3.7.3 contract. This is intentional — old prose IS unanchored and SHOULD surface as such. |
| Contamination check delays bibliography_agent ingest. | The S2 lookup is already in the v3.3 pipeline (semantic_scholar_id recording). v3.7.3 adds one bit of state derived from the same lookup. Marginal cost zero. |
paragraph index be 0-based or 1-based? 1-based matches human convention; 0-based matches code conventions. Tentative answer: 1-based. Resolve at codex round-1.%20 etc.) for quote payloads vs. HTML entity encoding vs. just escaping the HTML comment terminator. URL-encoding is most robust; HTML entity is more readable. Tentative answer: URL-encode, since the anchor lives in an HTML comment and --> would otherwise terminate it. Resolve at codex round-1.bibliography_agent.md or extracted into a shared/references/preprint_venues.md for reusability? Tentative answer: extract to shared/, since retrieval_agent and citation_compliance_agent may also consume it. Resolve at codex round-1.year >= 2024 (loose, catches January 2024 papers that predate the inflection) or (year, month) >= (2024, 6) (precise but requires month parsing)? Tentative answer: year >= 2024 for simplicity; January–May 2024 papers carry slightly inflated false-positive but the noise cost is low. Resolve at codex round-1./ars-mark-read <slug> does NOT clear NO-LOCATOR. The precedence-zero rule stops BEFORE applying the trust-state matrix, and /ars-mark-read only mutates human_read_source (a matrix input). The earlier tentative answer ("reuse /ars-mark-read for one acknowledgment surface") was a documented but unimplementable promise — the finalizer rules cannot satisfy it. The only NO-LOCATOR remediation is re-emitting the citation with <kind> ≠ none. The asymmetry is intentional: a locator is a structural property of the prose, not an evidence-state property of the source./ars-mark-read acknowledgment).END SPEC