← Về thư mục
📄 / / academic-research-skills / docs / design / 2026-06-08-262-cross-paper-contradiction-design.md

#262 — Structured cross-paper contradiction inventory (synthesis layer)

Status: design (2026-06-08) Issue: #262 (Kong et al. 2026 META #255, Tier B) Target agent: deep-research/agents/synthesis_agent.md Paper anchor: Kong et al. 2026 (arXiv:2605.18661, L. Kong, "Roadmap & User Guide") §7.4.2 — "systems increasingly retrieve and summarize individual papers well, but still struggle with multi-paper relational reasoning, methodological lineage, and cross-paper contradictions." Sibling precedent: #261 (figure/table fidelity, prose-layer) and #214 (synthesis sub-claim inventory, prose-layer) — both shipped prose-layer-only because producer and consumer are LLM agents, not a deterministic parser.


Problem

synthesis_agent (Phase 3, Analysis) already has prose-level contradiction handling:

What is not yet present, per Kong §7.4.2:

  1. A structured, inspectable inventory of which paper-pairs were assessed and what the verdict was — the current prose narrative-discusses contradictions (reconcilable and irreconcilable alike) but does not make the set of assessed pairs and the unresolved / checked-clear pairs enumerable for the scholar to confirm.
  2. Explicit candidate-pair scoping — the current pass relies on the LLM noticing contradictions ad-hoc during synthesis. For a large corpus there is no stated way to bound which pairs get looked at, and no honest statement of what the scan does and does not cover.

(A calibration reference is discussed later as an optional, non-shipping sanity-check — it is not a deliverable of this change.)

This is the cross-paper-relational sibling of #214 (which decomposed within-review sub-claims) and #261 (figure fidelity). All three enumerate before asserting "addressed."

Scope decision (chosen: prose-layer-only — deliberately departs from the issue's literal acceptance)

The issue's acceptance text overclaims, and following it literally would manufacture the exact "false rigor" the repo rejected in #261. This section records why we depart from the issue wording and ship the lighter form.

The issue acceptance says "Synthesis output schema adds contradiction_pairs[] block" and the rationale invokes "downstream agents (formatter, integrity_verification) cannot programmatically check." Read literally that points to a JSON Schema + lint invariant + CI gold gate. But:

This is the same scope call #261 made — its rationale was that machine-validating the YAML shape proves field presence, not fidelity, and adds an unrequested maintenance surface, whereas #213 earned schema+lint only because it has a deterministic machine consumer. #262 has no machine consumer either.

Decision: prose-layer-only. Extend the existing synthesis_agent contradiction section with a YAML-shaped markdown inventory block + a scoping subsection + a worked examples/ file. No JSON Schema, no lint invariant, no CI gold fixture. Vocabulary is aligned with the issue (contradiction, conditional_difference) to avoid drift, but no schema architecture is imported.

The departures from the issue's literal acceptance, to be stated plainly in the PR body:

Issue acceptance (literal) What ships Why
"Synthesis output schema adds contradiction_pairs[] block" YAML-shaped markdown block in the existing Contradictions section No deterministic consumer; schema = false rigor (#261 precedent)
"Calibration gold set … accuracy ≥ 0.75" as acceptance No calibration artifact ships; any future/manual calibration is recorded out-of-band (model/date/prompt + confusion matrix + per-class recall) and stays non-blocking 20 LLM-judged pairs is too small and too nondeterministic across runs to be a hard CI gate; wiring it as one would be false precision

Field model (codex-corrected — the issue's draft model is not MECE)

The issue's draft field model has four defects, all carried into this corrected model:

  1. conflict_type mixed two orthogonal axes. The issue enum {contradictory, conditional_difference, resolved_in_synthesis} folds a conflict nature (contradictory, conditional_difference) together with a resolution status (resolved_in_synthesis). These are separated into two fields.
  2. insufficient_overlap was referenced but not in the enum (the resolution_in_manuscript required-when clause names it). It is now a first-class pair_assessment value.
  3. No evidence pointers. a_finding / b_finding as free text invites the agent to state claims the papers do not make. Each side gains an evidence pointer grounded in the corpus context.
  4. candidate_basis was implicit. Recording why a pair was checked makes the scan's coverage inspectable (and exposes same-camp bias — see scoping).

The shipped block (markdown, inside the existing Contradictions & Resolutions section):

cross_paper_tensions:
  - pair_id: CP-001                      # synthesizer-assigned, stable within the synthesis
    paper_a: "<citation_key or ref slug>"
    paper_b: "<citation_key or ref slug>"
    candidate_basis: "shared RQ subtopic | shared construct/outcome/measure | opposite finding direction | bibliographic coupling | scholar flag | agent-noted cross-cluster"
    overlap_topic: "..."
    a_finding: "..."
    a_evidence_pointer: "..."            # where in the corpus context Paper A's finding is grounded
    b_finding: "..."
    b_evidence_pointer: "..."
    pair_assessment: "contradiction | conditional_difference | no_material_conflict | insufficient_overlap"
    resolution_status: "resolved_in_synthesis | flagged_unresolved | not_applicable"
    resolution_pointer: "Synthesis Report > Contradictions & Resolutions, ¶N"   # required when resolution_status == resolved_in_synthesis
    scholar_confirmation: "pending | confirmed | disputed"   # scholar-set; agent emits 'pending'

Field rules:

Candidate-pair scoping (recall-limited heuristic — NOT a sound algorithm, NOT MECE)

The issue framed scoping as "cluster-then-pairwise, O(K²) per cluster." That framing overstates what an LLM agent does: it does not execute an O(K²) enumeration; it is given a candidate-selection heuristic in prose. Calling it an algorithm with complexity guarantees would be dishonest unless a deterministic script built the clusters — and none does. So the design states it as bounded, recall-limited candidate generation.

Two failure modes are real and are disclosed in the output, not hidden:

The heuristic, stated as guidance:

  1. Generate candidate edges (not disjoint clusters). Include a pair if it shares an RQ subtopic, shares a construct/outcome/measure, shows opposite finding direction, is bibliographically coupled, or is scholar-flagged.
  2. Bibliographic coupling and shared-RQ are inclusion signals; never use them to exclude a pair from consideration.
  3. Deduplicate by sorted (paper_a, paper_b).
  4. Emit a Coverage Note: number of papers, candidate pairs considered, classes of pairs not exhaustively checked, and the explicit recall limitation ("this is a scoped advisory scan, not complete pairwise contradiction detection").

Advisory-only / narrative-side discipline (inherited, not new)

synthesis_agent already runs under a strict narrative-side contract: it emits, it does not audit; it does not simulate audit steps; it does not read entry frontmatter (synthesis_agent.md § PATTERN PROTECTION + the citation-emission narrative-side rules). The contradiction inventory inherits all of it:

Relationship to existing machinery (additive, no duplication)

Calibration (non-blocking manual reference, not a CI gate)

A calibration reference may be recorded to sanity-check contradiction-judgment quality: a small set of paper pairs (e.g. contradictory / conditional / compatible) judged by the agent, reported as a confusion matrix + per-class recall, with the model id, date, and prompt recorded. It is not wired into CI and is not a pass/fail acceptance gate — 20 LLM-judged pairs are too few and too nondeterministic across runs for a hard threshold. Recorded as a behavioral observation, not a guarantee.

Out of scope

Acceptance (corrected from #262 — see Scope decision for departures)

Verification