← Về thư mục
📄 / / academic-research-skills / docs / design / 2026-06-08-214-synthesizer-subclaim-design.md

#214 — Sub-claim inventory before consensus aggregation (partial-evidence trap, synthesis layer)

Status: design (2026-06-08) Issue: #214 (Kim et al. 2026 META #217) Target agent: academic-paper-reviewer/agents/editorial_synthesizer_agent.md Paper anchor: Kim et al. 2026 (arXiv:2605.20668v1) §F.3.2 — the partial-evidence trap is the single largest correctness-error class in AI meta-review (41/54, 76%). Sibling: #213 closed the citation-audit layer of the same trap; this is the editorial-synthesis layer.


Problem

§F.3.2 failure mode: a multi-part item gets a single binary verdict, so partial support is treated as full resolution.

In the editorial synthesizer, the general Synthesis Protocol Step 1 inventory is a (Dimension × reviewer) matrix whose "Key Weaknesses" cell holds whatever a reviewer said about weaknesses — and a single weakness bundle often packs several sub-claims (e.g. "statistical reporting is inconsistent AND mixed-model grouping is unclear"). Step 2 Consensus Classification then reaches CONSENSUS-4 / CONSENSUS-3 / SPLIT on the whole bundle, not on each sub-claim. A bundle where 4 reviewers agree on one sub-claim but only 1 raised the other is scored as one consensus verdict, and the minority sub-claim is folded into (or dropped from) the bundle's classification. Same shape as the documented failure: multi-part in, one verdict out.

This is the synthesis-layer sibling of #213. Both layers must enumerate before aggregating.

Scope decision (chosen: prose-layer-only — NOT a mirror of #213)

213 went heavy (new prompt verdict + normalized schema field sub_claim_breakdown[] + lint invariant INV-19 + gold fixtures + calibration subset metric). That weight was justified there because the citation judge emits machine-readable claim_audit_result rows consumed by a deterministic gate with downstream schema consumers.

The editorial synthesizer is different: its output is a human-facing Editorial Decision Letter + a Revision Roadmap consumed by academic-paper revision mode as prose. There is no schema / lint / gold-set substrate at this layer analogous to the citation gate, and the synthesizer is a prose-producing arbiter, not a row-emitting judge. Inventing a machine-readable sub-claim artifact + a lint to pin it here would be an unrequested abstraction with no deterministic consumer.

Decision: prose-layer-only. Change the synthesizer's reasoning + output prompt; add a worked examples/ before/after; confirm both forbidden-ops lists are not violated. No schema field, no lint invariant, no gold fixture. Vocabulary (sub_claim) is aligned with #213 to avoid drift, but #213's schema architecture is not imported.

The primary-key trap (and the faithful fix)

The issue says "Step 1 inventory primary key from Weakness to sub_claim." A literal read — make every report Dimension a sub-claim — breaks the non-weakness rows (Overall Recommendation, Confidence Score, # of Questions are not "sub-claims" and don't decompose). The faithful-to-paper fix decomposes the weakness/issues portion only, leaving the reviewer-summary rows intact.

So Step 1 splits in two:

| sub_claim_id | parent_weakness | reviewer_id | position | evidence_pointer | confidence |

Step 2 — consensus per sub-claim

CONSENSUS-4 / CONSENSUS-3 / SPLIT are computed per sub_claim_id, not per bundle. Mechanics unchanged except granularity:

DA-CRITICAL — unchanged

DA findings stay tracked independently of the 4-reviewer consensus count (#214 non-goal: do not change DA flow). DA-CRITICAL items are not decomposed into the Step 1b inventory; they keep their existing dedicated Decision-section treatment.

Revision Roadmap — sub-claim granularity

Step 5 Revision Roadmap construction: each roadmap item traces to a sub_claim_id (not just a weakness bundle), so a compound weakness whose sub-claims have different consensus levels produces separate, correctly-prioritized roadmap items instead of one blurred item. Roadmap-format compatibility with academic-paper revision mode input is preserved (sub_claim_id is additive provenance, not a format change).

Forbidden-ops compatibility (both lists)

  1. General Synthesis Protocol has no explicit forbidden-ops block; the governing constraint is the Phase-Boundary "you are not a 6th reviewer / do not produce new review comments." Sub-claim decomposition is re-organization of existing reviewer content (splitting what a reviewer already said into its atomic parts), not authoring new findings — compatible. The design adds a one-line guard: the synthesizer may only decompose claims a reviewer actually made; it must not introduce a sub-claim no reviewer raised.

  2. v3.6.2 Sprint Contract Synthesizer Protocol (the separate arithmetic mode, L46-69) is untouched. Its forbidden-ops (no aggregation rules beyond cross_reviewer_quantifier + severity, no averaging, etc.) govern failure_conditions[] / acceptance_dimensions[] evaluation and are orthogonal to the general protocol's weakness inventory. The new sub-claim protocol carries an explicit applicability line: "applies to the general Synthesis Protocol only; sprint-contract mode is unaffected."

Out of scope

Acceptance (from #214)

Verification