Date: 2026-04-27
Target release: v3.6.6 (academic-paper full-mode writer/evaluator sprint contract)
Roadmap anchor: academic-research-skills-ROADMAP.md §3.6.4 (description correction) + §3.6.5 exit criteria last item
Spec status: draft — §1 sanity-checked via codex round-1 (1 P1 + 7 P2 + 3 P3 applied), §2-§9 pending
Author: Cheng-I Wu
v3.6.6 (academic-paper/full writer/evaluator sprint contract)
Roadmap anchor: §3.6.4 description correction + §3.6.5 exit criteria last item Schema impact: Schema 13.1 backward-compatible add. Schema 14 explicitly rejected per Approach 1 decision.
shared/sprint_contract.schema.json Schema 13.1 upgrade:mode enum adds writer_full + evaluator_fullpre_commitment_artifacts, disagreement_handlingallOf / if mode startsWith reviewer_, including measurement_procedure.reviewer_must_output_before_paper, reviewer scoring-plan requirements, failure_conditions[].cross_reviewer_quantifier, and editorial failure_conditions[].actionpre_commitment_artifacts for writer_fulldisagreement_handling for evaluator_fullNote: agreement_audit is intentionally NOT a schema field. It is a runtime artifact (see item 10), not pre-registered acceptance criteria. Keeping it out of the schema preserves the v3.6.2 design philosophy that contract = "what good looks like before the work runs", not "what was observed during the work".
shared/contracts/writer/full.json + shared/contracts/evaluator/full.json contract templates.
shared/contracts/README.md update documenting writer/evaluator contract directories, mirroring v3.6.2 convention.
scripts/check_sprint_contract.py mode branching for reviewer, writer, and evaluator contracts; structure unchanged, logic extended.
scripts/test_check_sprint_contract.py extension + CI coverage for shared/contracts/writer/ and shared/contracts/evaluator/. The test module is the same one CI invokes via python3 -m unittest scripts.test_check_sprint_contract (workflow spec-consistency.yml); v3.6.6 extends rather than relocates it.
academic-paper/agents/draft_writer_agent.md Phase 4 hard gate:
acceptance_criteria_paraphrase + writer pre-commitment artifactsPhase 4b paper-visible drafting against the Phase 4a commitments
academic-paper/agents/peer_reviewer_agent.md Phase 6 hard gate:
contract_paraphrase + evaluator scoring/disagreement commitmentsPhase 6b paper-visible evaluation; verdict logic unchanged
academic-paper orchestration wiring for the actual two-call Phase 4 and Phase 6 split. The spec must name the orchestration file(s) that call draft_writer_agent.md and peer_reviewer_agent.md, or explicitly document that existing orchestration already supports separate paper-blind and paper-visible calls.
Model selection mirrors existing ARS_CROSS_MODEL env pattern: unset → warning + same-model; set → use that model ID for evaluator/cross-runtime comparison. No new env var.
Agreement audit split:
academic-paper/references/agreement_audit_protocol.md for the versioned protocol/templateexamples/ab_evidence/v3.6.6/agreement_audit_results.md for runtime example outputA/B evidence in tests/fixtures/v3.6.6-ab/ (per §6.2 / §6.5):
known_failure_mode + failure_evidence) for categorical signalROADMAP §3.6.4 description correction + new decision-log entry recording that v3.6.6 extends contracts to the existing writer/evaluator pair, rather than splitting a single agent.
CHANGELOG / .claude/CLAUDE.md / README / scripts/check_spec_consistency.py version bump sweep.
academic-paper/revision mode.
academic-pipeline orchestrator-level contract.
External academic-paper-reviewer skill integration; it already received reviewer contracts in v3.6.2 and is zero-touch in v3.6.6.
Internal/external two-reviewer merge. This is known technical debt and remains a v3.7.x follow-up.
New evaluator agent file; peer_reviewer_agent.md is the evaluator.
Schema 14. Schema 13.1 is the approved backward-compatible add.
New CLI flag, contract negotiation UI, or runtime contract editing.
Real retracted-paper fixtures or hard FNR/FPR calibration gates.
Mirror v3.6.2 pattern — schema evolution is backward-compatible, validator structure stays familiar, templates live under shared/contracts/, and the load-bearing runtime behavior is a two-call paper-blind/paper-visible hard gate.
Evaluator anti-drift — peer_reviewer_agent commits its contract paraphrase, scoring plan, and disagreement-handling plan before reading the full draft.
Writer pre-commitment — draft_writer_agent commits acceptance criteria and pre-commitment artifacts before drafting against the full paper context.
Cross-model compatible via existing env pattern — v3.6.6 supports cross-model/cross-runtime comparison through ARS_CROSS_MODEL without changing default same-model behavior when unset.
Open-ended task evidence discipline — A/B evidence uses process metrics and deltas, not absolute claims that a generated paper is objectively good.
Do not set an absolute paper-quality threshold; contract-on/off delta is the signal.
Do not introduce contract negotiation protocol; inherit fixed baseline template + bounded agent_amendments append slot from v3.6.2.
Do not change current Accept/Minor/Major/Reject verdict mapping.
Do not merge, replace, or coordinate the internal peer_reviewer_agent with the external academic-paper-reviewer skill.
academic-paper still has an internal peer_reviewer_agent, while the repository also has the external academic-paper-reviewer skill. v3.6.6 documents this as known technical debt and leaves merge/routing decisions to v3.7.x.§5 must verify whether existing academic-paper orchestration already supports separate paper-blind and paper-visible calls. If not, v3.6.6 must include the concrete orchestration change rather than treating the split as a prompt-only edit.
§3 must spell out the pre_commitment_artifacts field shape (subfields, validation rules) — currently named but not structurally specified.
§6 A/B evidence ship thresholds: initial estimates of +20 pp / 15% / +2 findings have been tuned after fixture sizing to H1 hard ≥ +10 pp / H2 soft ≤ 0% relative delta / H3 soft ≥ 0 mean count delta, with paper-C categorical pass/fail as a separate signal. Frozen in §6.4; further revision requires §6 spec amendment per §6.6 boundary.
Two rounds of decisions converged before this spec started: a 7-question clarification with the user (Q1-Q7), then a codex round-1 sanity check on §1 that surfaced four design-level corrections (D1-D4). The decisions table records both rounds with their consequences for §3 onwards.
| # | Question | Decision | Consequence |
|---|---|---|---|
| Q1 | How wide should v3.6.6 scope be? Mirror v3.6.2's reviewer-only first-test-case discipline, or include revision mode and the academic-pipeline orchestrator? |
Narrow: academic-paper/full mode only. No revision, no academic-pipeline orchestrator-level contract, no external academic-paper-reviewer skill (already on contract since v3.6.2). |
Schema 13.1 enum gains exactly two new modes (writer_full, evaluator_full). Two contract templates ship. revision and pipeline contracts deferred to v3.6.7+ / v3.7.x. |
| Q2 | What to do about the two-reviewer overlap (internal peer_reviewer_agent inside academic-paper Phase 6 vs the standalone academic-paper-reviewer skill)? |
Document as known technical debt; do not merge in v3.6.6. Spec §"Known limitations" calls it out; routing/merge decision deferred to v3.7.x. | v3.6.6 touches only the internal peer_reviewer_agent for the evaluator contract. External academic-paper-reviewer is zero-touch and keeps its v3.6.2 reviewer contracts untouched. |
| Q3 | How does the user select which model evaluates? Add a new env var, extend ARS_CROSS_MODEL, or embed in the contract template? |
Mirror existing ARS_CROSS_MODEL env pattern unchanged. Already a model-ID string (not boolean). Unset → warning + same-model evaluator. Set → use that model ID. No new env var. |
No new user-facing setup. Cross-model is opt-in, consistent with shared/cross_model_verification.md ("This is entirely optional. All ARS skills work with Claude Opus 4.7 alone… Continue with single-model verification — never block the pipeline on cross-model failure"). Design Goal 4 is worded as "compatible via existing pattern", not "default cross-model" (D3 below). |
| Q4 | What does A/B evidence look like for an open-ended task where no ground-truth quality score exists? | Process-metric only, delta-based main indicator, not absolute. Cross-model judge via codex gpt-5.5 + xhigh in verdict-only mode for LLM-judged metrics; programmatic verification via Semantic Scholar API Tier 0 for citation-grounded metrics. Three pre-declared directional hypotheses (H1-H3) plus a categorical sanity-check paper. | §6 specifies the fixture (six paper-A + one paper-C, §6.2), three process metrics (H1 acceptance dimension pass rate, H2 Tier 0 unverified citation rate, H3 deduped self-detected issue count, §6.3), the ship gate rubric (H1 hard +10pp / H2 + H3 soft / paper-C categorical, §6.4), and the artefact reproducibility invariants (§6.5). Absolute paper quality is explicitly not measured. |
| Q5 | Where do A/B fixtures come from? Real retracted papers, AI-generated synthetic, or human-written? | Re-use existing suite outputs as fixture inputs, evolved from the original Q5 decision (AI-generated synthetic + human-authored defect injection + expected_findings.yaml) after §6.2 fixture-sizing brainstorm. Six paper-A inputs are existing deep-research synthesis reports already produced by the suite (covering three paper types × two repetitions); one paper-C input is an existing v3.6.5 academic-paper full-run session log where the in-pair Phase 6 evaluator missed a hallucinated citation that the Stage 3 reviewer caught. Real retracted papers remain rejected (ethical/legal); AI-authored defect injection rejected (AI-evaluator collusion risk on top of authoring overhead). |
§6.2 specifies the seven-paper fixture composition (six paper-A + one paper-C). No expected_findings.yaml ground truth file — H1 / H3 use codex gpt-5.5 + xhigh verdict-only judge against the contract acceptance dimensions; H2 uses Semantic Scholar API programmatic verification; paper-C categorical signal uses the v3.6.5 session-log Stage 3 reviewer excerpt as the known-failure-mode evidence (§6.2 manifest failure_evidence field). |
| Q6 | Schema 13.1 backward-compatible add, Schema 14 new schema, or Approach 3 base+extension refactor? | Approach 1 — Schema 13.1 backward-compatible add. v3.6.2 §10 risk 7 explicitly predicted this path; Approach 3 violates v3.6.2's zero-touch promise; Schema 14 doubles the surface area without conceptual gain. | §3 specifies Schema 13.1 conditional fields. Existing reviewer contracts and validator structure remain untouched (D1 below tightens this). |
| Q7 | Does the spec correct ROADMAP §3.6.4's outdated "split single agent" description, or only describe what v3.6.6 actually does? | Correct ROADMAP §3.6.4 + sync with new decision-log entry. Truthfulness over change minimisation; ROADMAP is source of truth and stale text misleads future readers. | In-scope item 12 includes ROADMAP edit. Decision-log entry records that v3.6.6 extends contracts to the existing writer/evaluator pair, not splits a single agent. |
After §1 was drafted from Q1-Q7, a codex sanity check on §1 surfaced four design-level issues that survived clarification because they only become visible when the spec meets the existing schema and protocol. All four were applied to §1 and shape §3-§6.
| # | Issue codex surfaced | Decision | Consequence |
|---|---|---|---|
| D1 | Schema 13 has reviewer-specific required fields (measurement_procedure.reviewer_must_output_before_paper, failure_conditions[].cross_reviewer_quantifier, editorial enum on failure_conditions[].action). A bare mode enum addition will fail validation for writer_full / evaluator_full. The "13.1 simple add" framing was too optimistic. |
Apply v3.6.2's allOf / if mode startsWith reviewer_ conditional pattern to all four reviewer-specific fields. Mirror the technique v3.6.2 already used for cross_reviewer_quantifier (v3.6.2 spec §3.3). |
§3 specifies four reviewer-conditional gates (not one). Existing reviewer contracts validate identically (zero-touch promise preserved). New writer/evaluator contracts are unaffected by reviewer-specific fields. |
| D2 | disagreement_handling was originally proposed as a single field for both writer and evaluator. But "disagreement" is meaningless for a solo writer (no peer to disagree with). |
Split into two fields. Writer mode requires pre_commitment_artifacts (what to commit pre-paper). Evaluator mode requires disagreement_handling (how to resolve writer-evaluator disagreement). Each gated to its respective mode. |
Schema 13.1 adds two optional top-level fields, not one. Two new conditional gates: if mode == writer_full requires pre_commitment_artifacts; if mode == evaluator_full requires disagreement_handling. |
| D3 | "Cross-model by default" framing in original Design Goal 4 contradicts the actual decided behaviour (Q3: unset → same-model with warning, set → cross-model). The framing inflated the truthful claim. | Reword Design Goal 4 to "Cross-model compatible via existing env pattern." Truthful framing matches actual behaviour. | Design Goal 4 in §1 reflects the change. README / CHANGELOG sweep (in-scope 13) carries the same wording so no claim of "default cross-model" reaches users. |
| D4 | Codex P2 #18 flagged that agreement_audit.md placed under academic-paper/references/ mixes versioned source with runtime artifact, and recommended splitting protocol-doc from runtime-output. Reading that finding against v3.6.2's contract philosophy raised a deeper question: should agreement_audit be a schema field at all? Agreement-audit data is observed during the run, not pre-registered acceptance criteria (the v3.6.2 contract definition). Embedding observed data in the schema violates v3.6.2's load-bearing principle that contract = "what good looks like before the work runs". |
Extension of codex's P2 #18 finding (not codex's own recommendation): remove agreement_audit from schema entirely. Keep only the protocol document (versioned, in academic-paper/references/agreement_audit_protocol.md) and runtime example output (in examples/ab_evidence/v3.6.6/agreement_audit_results.md). Audit data lives in runtime artifacts, never in the contract. |
§1 in-scope item 1 lists only two new optional schema fields. The audit/runtime separation appears in in-scope item 10 (split-path delivery). Note that codex's original recommendation only addressed file location (split protocol from runtime output); the schema-removal decision is one step further. |
L1-L4 are tracked in §8 "Risks and open questions" (mirroring v3.6.2 spec §10 convention; v3.6.6 has fewer sections so risks and open questions live in the same section). They were raised during brainstorm or codex sanity check but settling them needs the schema or protocol detail in §3-§6.
Schema 13.1 is a backward-compatible extension of Schema 13. The schema file path stays shared/sprint_contract.schema.json. The title updates to "ARS Sprint Contract (Schema 13.1)". No file rename, no $id change.
This section closes open questions L1 (pre_commitment_artifacts shape) and L4 (acceptance_criteria_paraphrase vs pre_commitment_artifacts parity). L2 (orchestration) is closed by §5; L3 (final A/B thresholds) in §6.
Three categories of change. None remove or rename existing fields.
| Category | Change | Backward-compat impact |
|---|---|---|
| Enum extension | mode enum gains writer_full + evaluator_full (5 → 7 values) |
Reviewer contracts unchanged. Enum addition is non-breaking. |
| Existing-field gating | Four currently-required fields become reviewer-conditional: measurement_procedure, panel_size, failure_conditions[].cross_reviewer_quantifier (already conditional in §3.3 of Schema 13), failure_conditions[].action enum membership |
Reviewer contracts validate identically. Writer/evaluator contracts are no longer forced to carry reviewer-specific structure. |
| New optional fields | Two new top-level optional fields: pre_commitment_artifacts, disagreement_handling. Two new conditional gates require them for writer_full / evaluator_full. |
Reviewer contracts ignore both fields (still pass). Writer/evaluator contracts must carry the field that matches their mode. |
agreement_audit is explicitly not added to the schema (D4 in §2.2). Audit data is a runtime artifact, captured in examples/ab_evidence/v3.6.6/agreement_audit_results.md, never in pre-registered acceptance criteria.
"mode": {
"enum": [
"reviewer_full",
"reviewer_methodology_focus",
"reviewer_re_review",
"reviewer_calibration",
"reviewer_guided",
"writer_full",
"evaluator_full"
]
}
reviewer_quick remains excluded from the enum (v3.6.2 Q3-A' boundary). revision_full, evaluator_revision, pipeline_full, etc. are out of scope for v3.6.6 (v3.6.7+ / v3.7.x candidates).
Schema 13 has four reviewer-specific surfaces that must move from "required for all modes" to "required only for reviewer_* modes". This applies the v3.6.2 §3.3 allOf if mode startsWith reviewer_ pattern to additional fields.
measurement_procedure becomes reviewer-conditionalCurrent state (Schema 13): top-level required includes measurement_procedure, and the object's three subfields are all reviewer-specific (reviewer_must_output_before_paper, scoring_plan_schema, paraphrase_minimum_dimensions).
Schema 13.1 change: remove measurement_procedure from top-level required. Add it back conditionally:
{
"if": {
"properties": { "mode": { "pattern": "^reviewer_" } },
"required": ["mode"]
},
"then": {
"required": ["measurement_procedure"]
}
}
Writer and evaluator modes do NOT need measurement_procedure because their pre-commitment artifacts live in dedicated new fields (pre_commitment_artifacts / disagreement_handling) with different shapes — see §3.4.
failure_conditions[].cross_reviewer_quantifier (already conditional)Schema 13 §3.3 already gates this:
{
"if": { "properties": { "mode": { "pattern": "^reviewer_" } }, "required": ["mode"] },
"then": { "properties": { "failure_conditions": { "items": { "required": ["cross_reviewer_quantifier"] } } } }
}
No Schema 13.1 change needed. Writer/evaluator failure_conditions are not required to carry this field.
failure_conditions[].action enum membership becomes mode-conditionalCurrent state: action enum is hardcoded to five editorial_decision=* values. These are reviewer-only outcomes (an evaluator does not issue an editorial decision; a writer cannot fail itself into an editorial decision).
Schema 13.1 change: remove the base action enum entirely. Move the constraint to three allOf mode-conditional branches (reviewer / writer / evaluator). The base property keeps only "type": "string" as a structural placeholder.
This change is forced by JSON Schema semantics: if/then does not override the base — the base enum and the then enum intersect, and a value must satisfy both. A writer/evaluator-specific enum cannot override the base reviewer enum; both are simultaneously enforced and the writer/evaluator values are rejected. The only valid pattern is to drop the base enum and constrain entirely in allOf.
"action": { "type": "string" }
with the three allOf branches:
{
"if": {
"properties": { "mode": { "pattern": "^reviewer_" } },
"required": ["mode"]
},
"then": {
"properties": {
"failure_conditions": {
"items": {
"properties": {
"action": {
"enum": [
"editorial_decision=accept",
"editorial_decision=minor_revision",
"editorial_decision=major_revision",
"editorial_decision=reject_or_major_revision",
"editorial_decision=reject"
]
}
}
}
}
}
}
},
{
"if": {
"properties": { "mode": { "const": "writer_full" } },
"required": ["mode"]
},
"then": {
"properties": {
"failure_conditions": {
"items": {
"properties": {
"action": {
"enum": [
"writer_decision=accept",
"writer_decision=revise_in_phase_4b",
"writer_decision=escalate_to_evaluator"
]
}
}
}
}
}
}
},
{
"if": {
"properties": { "mode": { "const": "evaluator_full" } },
"required": ["mode"]
},
"then": {
"properties": {
"failure_conditions": {
"items": {
"properties": {
"action": {
"enum": [
"evaluator_decision=accept",
"evaluator_decision=accept_with_dissent_note",
"evaluator_decision=request_revision",
"evaluator_decision=flag_for_reviewer_stage"
]
}
}
}
}
}
}
}
Notes on the action vocabulary:
writer_decision=accept (F0 success — see §3.3.4), writer_decision=revise_in_phase_4b (writer self-repair within the same phase), writer_decision=escalate_to_evaluator (writer hits a condition it cannot self-repair, hands off to evaluator).accept / request_revision / flag_for_reviewer_stage) plus accept_with_dissent_note for cases where the evaluator's own pre-committed scoring_plan.per_dimension_criteria.what_triggers_warn matches on a high or normal-priority dimension and no mandatory dimension scores 'block' or 'warn' — the draft is acceptable overall, but a non-mandatory concern is logged rather than silenced. The accept_with_dissent_note action is referenced by disagreement_handling.disagreement_resolution.on_dimension_disagreement in §3.4.2 — both surfaces use the same vocabulary. Crucially, neither surface relies on numeric writer pre-commitments (the writer's pre_commitment_artifacts carries only acceptance_criteria_paraphrase per §3.4.1, not per-dimension numeric score predictions); evaluator-side dissent is anchored on the evaluator's own scoring plan plus the upstream structural artefacts the writer follows in Phase 4b (Structure Architect's approved outline, Argument Blueprint, Paper Configuration Record).failure_conditions contains F0 — concept stays universal, mechanics become mode-conditionalThe concept of a universal F0 accept condition stays — every contract must carry an explicit success condition so the synthesizer/evaluator/writer always has a "no-failure-fired" terminal state to emit. Writer F0 means "draft meets all writer pre-commitment artifacts"; evaluator F0 means "draft passes evaluator's pre-committed scoring plan"; reviewer F0 stays "every mandatory dimension scores 'pass'".
The mechanics must change. Schema 13 hardcodes contains.properties.action to "const": "editorial_decision=accept". With the §3.3.3 mode-conditional action enum, writer/evaluator F0 actions are writer_decision=accept / evaluator_decision=accept respectively. The hardcoded const will reject them. F0 contains must therefore become mode-conditional too:
"failure_conditions": {
"type": "array",
"minItems": 1,
"items": { /* unchanged */ }
}
Remove the inline contains from the base failure_conditions definition. Re-add it via three allOf mode-conditional branches that each pin F0's action to the mode's accept variant:
{
"if": {
"properties": { "mode": { "pattern": "^reviewer_" } },
"required": ["mode"]
},
"then": {
"properties": {
"failure_conditions": {
"contains": {
"type": "object",
"properties": {
"condition_id": { "const": "F0" },
"action": { "const": "editorial_decision=accept" }
},
"required": ["condition_id", "action"]
}
}
}
}
},
{
"if": {
"properties": { "mode": { "const": "writer_full" } },
"required": ["mode"]
},
"then": {
"properties": {
"failure_conditions": {
"contains": {
"type": "object",
"properties": {
"condition_id": { "const": "F0" },
"action": { "const": "writer_decision=accept" }
},
"required": ["condition_id", "action"]
}
}
}
}
},
{
"if": {
"properties": { "mode": { "const": "evaluator_full" } },
"required": ["mode"]
},
"then": {
"properties": {
"failure_conditions": {
"contains": {
"type": "object",
"properties": {
"condition_id": { "const": "F0" },
"action": { "const": "evaluator_decision=accept" }
},
"required": ["condition_id", "action"]
}
}
}
}
}
Reviewer contracts validate identically: the reviewer-mode contains clause is structurally identical to the Schema 13 inline contains. Writer/evaluator contracts get a mode-appropriate F0 accept pin.
panel_size becomes reviewer-conditionalCurrent state (Schema 13): top-level required includes panel_size with the base definition { "type": "integer", "minimum": 1 }. The field encodes the size of the reviewer panel for cross-reviewer quantifier evaluation (cross_reviewer_quantifier ∈ {any, majority, all} in §3.3.2 is meaningful only when more than one reviewer participates). Generator-mode contracts run a single agent each — draft_writer_agent for writer_full, peer_reviewer_agent for evaluator_full — so panel_size has no semantic anchor in those modes.
Schema 13.1 change: remove panel_size from top-level required. The base definition stays unchanged ({ "type": "integer", "minimum": 1 }) so any contract that does carry the field still validates structurally. Add it back conditionally for reviewer modes:
{
"if": {
"properties": { "mode": { "pattern": "^reviewer_" } },
"required": ["mode"]
},
"then": {
"required": ["panel_size"]
}
}
Writer and evaluator contracts do NOT carry panel_size. Each runs a single agent; the field would either be set to 1 (semantically empty) or omitted, and forcing the field on the template adds no validation power while leaking reviewer-specific vocabulary into generator-mode contracts. The cross-reviewer-quantifier conditional gate (§3.3.2 / allOf branch 1) requires cross_reviewer_quantifier only for reviewer modes; the field remains a declared optional property on failure_conditions[].items for all modes, so a generator-mode contract that mistakenly carries cross_reviewer_quantifier (for example, copy-pasted from a reviewer template) still passes schema validation. Shipped writer/full.json and evaluator/full.json templates omit the field by default, and §4 contract authoring guidance restates the omission. Schema-level strict prevention (declaring the property false for generator modes, or adding a top-level not clause) is deferred to v3.7.x — see §8.2 R1. The same require-only-not-forbid pattern applies symmetrically to measurement_procedure (§3.3.1) and is inherited from Schema 13's existing reviewer-conditional design; v3.6.6 keeps that posture rather than expanding the schema strictness surface in this release.
Both new fields below are inserted into the schema's top-level properties object, not nested under measurement_procedure or any other existing field. They are declared at top-level (so the outer additionalProperties: false does not reject contracts that carry them) but not added to the top-level required list — their requirement is mode-conditional (gated in §3.5).
pre_commitment_artifacts (writer-only, closes L1)"pre_commitment_artifacts": {
"type": "object",
"additionalProperties": false,
"required": ["acceptance_criteria_paraphrase"],
"$comment": "pre_commitment_artifacts carries only invariant, paper-agnostic policy: how many acceptance_dimensions the writer must paraphrase in Phase 4a paper-blind output. Everything paper-specific lives in upstream runtime artefacts and is read at Phase 4a / 4b runtime, not committed in the frozen contract baseline. Specifically: per-section word counts come from the Paper Configuration Record (intake_agent); section outline comes from the Structure Architect's approved outline (structure_architect_agent); CER chains come from the Argument Blueprint (argument_builder_agent). Embedding any of those in the contract baseline would either force orchestrators to mutate the frozen baseline before Phase 4a or hard-code IMRaD assumptions onto non-IMRaD paper types (literature reviews, theoretical, case study, policy brief — all supported by writer_full per draft_writer_agent's Paper Type Adjustments table). Writer dimensions D1 (section_completeness), D3 (argument_blueprint_fidelity), D4 (total_word_count), D5 (per_section_word_count) phrase their gates against those upstream artefacts rather than against contract-embedded values. The object remains a structured sub-object (rather than promoting acceptance_criteria_paraphrase to a top-level field) so v3.7.x can extend pre_commitment_artifacts with new invariant, paper-agnostic policy without an additional schema-architecture refactor.",
"properties": {
"acceptance_criteria_paraphrase": {
"type": "object",
"additionalProperties": false,
"required": ["minimum_dimensions"],
"$comment": "This object is a CONFIGURATION declaration, not the paraphrase content itself. It declares how many of the contract's acceptance_dimensions the writer must paraphrase in Phase 4a. The actual paraphrase text is a runtime artifact emitted by draft_writer_agent in its Phase 4a output (analogous to how reviewer paraphrase content lives in <phase1_output>, not in the contract). Mirrors reviewer measurement_procedure.paraphrase_minimum_dimensions semantics.",
"properties": {
"minimum_dimensions": {
"anyOf": [
{ "const": "all" },
{ "type": "integer", "minimum": 1 }
],
"$comment": "Default in shipped writer template: 'all'."
}
}
}
}
}
Required for writer_full only (gated in §3.5).
L4 resolution: acceptance_criteria_paraphrase is a subfield inside pre_commitment_artifacts, not a separate top-level field. This avoids the codex P3 #33 parity question — writer Phase 4a outputs ONE artifact (pre_commitment_artifacts) which currently carries one invariant, paper-agnostic subcomponent (acceptance_criteria_paraphrase). Earlier drafts of §3.4.1 also nested structure_plan and argument_plan here; those were evicted to upstream runtime artefacts (Structure Architect output, Argument Blueprint) per the schema $comment above and the design discussion in §4.2 / §8.2 R2. The container shape is preserved so v3.7.x can extend pre_commitment_artifacts with new invariant policy without an architecture refactor. Reviewer Phase 1 still uses the existing contract_paraphrase semantics under measurement_procedure.reviewer_must_output_before_paper. The naming asymmetry reflects that reviewer is paraphrasing for assessment, writer is paraphrasing for execution.
disagreement_handling (evaluator-only)"disagreement_handling": {
"type": "object",
"additionalProperties": false,
"required": ["paraphrase_minimum_dimensions", "scoring_plan", "pre_commitment_check_protocol", "disagreement_resolution"],
"properties": {
"paraphrase_minimum_dimensions": {
"anyOf": [
{ "const": "all" },
{ "type": "integer", "minimum": 1 }
],
"$comment": "How many of the contract's acceptance_dimensions the evaluator must paraphrase in Phase 6a paper-blind output (the contract_paraphrase artifact named in §1 design goal 2 / §4.3). Mirrors reviewer measurement_procedure.paraphrase_minimum_dimensions and writer pre_commitment_artifacts.acceptance_criteria_paraphrase.minimum_dimensions to keep the Phase-6a anti-drift gate hard-required at the schema layer rather than relying on prose-level commitments. Default in shipped evaluator template: 'all'."
},
"scoring_plan": {
"type": "object",
"additionalProperties": false,
"required": ["per_dimension_criteria"],
"properties": {
"per_dimension_criteria": {
"type": "array",
"minItems": 1,
"items": {
"type": "object",
"additionalProperties": false,
"required": ["dimension_id", "what_to_look_for", "what_triggers_block", "what_triggers_warn"],
"properties": {
"dimension_id": { "type": "string", "pattern": "^D[1-9][0-9]?$" },
"what_to_look_for": { "type": "string", "minLength": 1 },
"what_triggers_block": { "type": "string", "minLength": 1 },
"what_triggers_warn": { "type": "string", "minLength": 1 }
},
"$comment": "Mirrors reviewer measurement_procedure.scoring_plan_schema four-field structure. Evaluator commits these in Phase 6a paper-blind."
}
}
}
},
"pre_commitment_check_protocol": {
"type": "object",
"additionalProperties": false,
"required": ["check_writer_artifact"],
"properties": {
"check_writer_artifact": {
"const": "pre_commitment_artifacts",
"$comment": "Pinned to the single writer source for v3.6.6. Field shape exists (rather than inlining the const) for forward compatibility — when v3.6.7+ may add additional writer artifacts, this can relax to an enum without a breaking schema change."
}
}
},
"disagreement_resolution": {
"type": "object",
"additionalProperties": false,
"required": ["on_dimension_disagreement", "on_structural_drift"],
"$comment": "Both subfield enums use the fully-qualified evaluator_decision=* vocabulary so values can be passed directly into failure_conditions[].action without runtime transformation. Keeping a single canonical action vocabulary across §3.3.3 and §3.4.2 prevents drift between the two surfaces.",
"properties": {
"on_dimension_disagreement": {
"enum": [
"evaluator_decision=request_revision",
"evaluator_decision=accept_with_dissent_note",
"evaluator_decision=flag_for_reviewer_stage"
],
"$comment": "What the evaluator does when its own paper-blind scoring plan flags a dimension during paper-visible evaluation (the dimension hits its pre-committed what_triggers_block or what_triggers_warn anchors). The disagreement is between the evaluator's pre-committed criteria and its post-paper observation; it is NOT anchored on numeric writer pre-commitments (writer pre_commitment_artifacts carries only acceptance_criteria_paraphrase per §3.4.1; paper-specific section outlines, CER chains, and word counts are read from upstream runtime artefacts — Structure Architect output, Argument Blueprint, Paper Configuration Record — not from the contract). Default in shipped templates: 'evaluator_decision=request_revision' for mandatory dimensions; runtime evaluator may downgrade non-mandatory dimensions to 'accept_with_dissent_note' (see F4 in evaluator/full.json)."
},
"on_structural_drift": {
"enum": [
"evaluator_decision=request_revision",
"evaluator_decision=accept_with_dissent_note",
"evaluator_decision=flag_for_reviewer_stage"
],
"$comment": "What the evaluator does when the actual draft structure deviates from the Structure Architect's approved outline (the upstream runtime artefact from structure_architect_agent that the writer follows in Phase 4b). Drift threshold lives in the evaluator's runtime judgement, not the schema; the contract carries only the default verdict to apply when drift is observed."
}
}
}
}
}
Required for evaluator_full only (gated in §3.5).
Two new allOf branches added to the existing two:
{
"if": {
"properties": { "mode": { "const": "writer_full" } },
"required": ["mode"]
},
"then": {
"required": ["pre_commitment_artifacts"]
}
},
{
"if": {
"properties": { "mode": { "const": "evaluator_full" } },
"required": ["mode"]
},
"then": {
"required": ["disagreement_handling"]
}
}
The full Schema 13.1 allOf array therefore has twelve branches:
| # | Source | Branch |
|---|---|---|
| 1 | (existing) Schema 13 §3.3 | Reviewer cross_reviewer_quantifier requirement |
| 2 | (existing) Schema 13 §3.3 | override_ladder three-round shape |
| 3 | (new) §3.3.1 | Reviewer mode requires measurement_procedure |
| 4 | (new) §3.3.3 | Reviewer mode pins failure_conditions[].action to editorial enum |
| 5 | (new) §3.3.3 | writer_full pins failure_conditions[].action to writer_decision enum |
| 6 | (new) §3.3.3 | evaluator_full pins failure_conditions[].action to evaluator_decision enum |
| 7 | (new) §3.3.4 | Reviewer mode F0 contains (action = editorial_decision=accept) |
| 8 | (new) §3.3.4 | writer_full F0 contains (action = writer_decision=accept) |
| 9 | (new) §3.3.4 | evaluator_full F0 contains (action = evaluator_decision=accept) |
| 10 | (new) §3.3.5 | Reviewer mode requires panel_size |
| 11 | (new) this section | writer_full requires pre_commitment_artifacts |
| 12 | (new) this section | evaluator_full requires disagreement_handling |
Order is logically grouped (existing first; then reviewer-conditional including panel_size; then generator-family triplets for action enum and F0 contains; then mode-specific top-level requirements) but JSON Schema does not enforce allOf order semantics.
The branch count is high because two existing constraints (action enum, F0 contains) had to be lifted out of base properties and re-expressed three times for the three mode families, and two further fields (measurement_procedure, panel_size) had to be lifted from base required into reviewer-conditional gates. This is the trade-off for keeping reviewer validation byte-identical (zero-touch promise) while extending the schema to writer/evaluator modes.
A Schema 13 reviewer contract that was valid before v3.6.6 must remain valid under Schema 13.1. The proof is structural:
| Schema 13 reviewer contract field | Schema 13.1 fate | Validation impact |
|---|---|---|
mode: reviewer_* |
Still in enum | Pass |
measurement_procedure (object, all three subfields) |
Removed from base required; reviewer-conditional gate (allOf branch 3, §3.3.1) re-requires it for reviewer modes |
Pass — same effective constraint |
failure_conditions[].cross_reviewer_quantifier |
Same conditional as Schema 13 (allOf branch 1) | Pass |
failure_conditions[].action (editorial_decision=*) |
Base enum lifted; reviewer-conditional gate (allOf branch 4, §3.3.3) re-pins it to the same five editorial_decision=* values for reviewer modes |
Pass — same effective enum |
failure_conditions contains F0 (editorial_decision=accept) |
Inline contains lifted; reviewer-conditional gate (allOf branch 7, §3.3.4) re-pins F0 to editorial_decision=accept for reviewer modes |
Pass — structurally identical contains clause |
panel_size (integer ≥ 1) |
Removed from base required; reviewer-conditional gate (allOf branch 10, §3.3.5) re-requires it for reviewer modes; base property definition unchanged |
Pass — same effective constraint |
All other fields (contract_id, stage, baseline_version, acceptance_dimensions, override_ladder, agent_amendments, generated_at) |
Unchanged | Pass |
The pattern across the lifted constraints (measurement_procedure, action enum, F0 contains, panel_size): each was originally enforced unconditionally in the base; each is now enforced via a reviewer-mode allOf branch that re-applies the original constraint. Validation is byte-equivalent for every reviewer contract; the lift only opens room for writer/evaluator modes to satisfy different but mode-appropriate constraints in branches 5/6 and 8/9, and to omit panel_size entirely (a single-agent generator has no panel).
pre_commitment_artifacts and disagreement_handling are absent from reviewer contracts — both are optional at top level and gated only for writer/evaluator modes (allOf branches 11/12), so reviewer contract validation does not require them.
scripts/test_check_sprint_contract.py will gain a test_existing_reviewer_contracts_still_valid_under_13_1 regression test that loads both shipped reviewer templates (shared/contracts/reviewer/full.json + methodology_focus.json) against the Schema 13.1 file and asserts validation success without modification. A second test, test_byte_equivalent_validation_for_reviewer_contracts, will diff the validation result tree (errors + warnings) against Schema 13's result for the same contract and assert empty diff.
These are out of scope for §3 and live in later sections:
§4 specifies the two new shipped templates introduced in v3.6.6:
shared/contracts/writer/full.json — writer_full mode, gated to draft_writer_agent Phase 4shared/contracts/evaluator/full.json — evaluator_full mode, gated to peer_reviewer_agent Phase 6Both templates are concrete instances of Schema 13.1 (§3) and demonstrate every new conditional gate (pre_commitment_artifacts for writer; disagreement_handling for evaluator; mode-specific failure_conditions[].action enums; mode-appropriate F0 accept). Each template's acceptance dimensions are extracted from the existing draft_writer_agent.md / peer_reviewer_agent.md Quality Gates and rubric, preserving the v3.6.2 design philosophy that "contract = pre-existing agent behavior turned into a hard gate", not "contract = new behavior bolted on".
Spec-branch notice. While
spec/v3.6.6-generator-evaluator-contractis the active branch, the JSON files atshared/contracts/writer/full.jsonandshared/contracts/evaluator/full.jsonare design-time artefacts: they describe what good will look like at v3.6.6 ship time, but Schema 13.1 has not yet landed inshared/sprint_contract.schema.json. The current Schema 13 validator rejects these files by design (see expected errors below). Do not load them as live contracts on this branch. Once the v3.6.6 implementation PR merges tomain, both files become live shipped templates atomically with the schema upgrade.
The shipped JSON files land on the spec branch as part of this design doc rather than as a follow-up template PR for one reason: §3 and §4 are co-load-bearing. Schema 13.1's eleven new allOf branches are only reviewable in the abstract until concrete templates exercise them; the writer's pre_commitment_artifacts shape (§3.4.1) is only reviewable once an actual writer_full template binds it to acceptance dimensions and acceptance_criteria_paraphrase.minimum_dimensions; F-condition severity ordering (§4.2 / §4.3) is only reviewable once mode-specific action enums are populated. Splitting the templates out into a follow-up patch would mean reviewing the schema and the templates against different mental models on different days — the exact drift mode that v3.6.5.2 codex review caught (and fixed in the §3 panel_size patch on this branch).
Expected validator behaviour on the spec branch:
$ python scripts/check_sprint_contract.py shared/contracts/writer/full.json --ars-version v3.6.6
ERROR: ['mode']: 'writer_full' is not one of [...reviewer modes only...]
ERROR: []: 'panel_size' is a required property
ERROR: []: 'measurement_procedure' is a required property
ERROR: []: Additional properties are not allowed ('pre_commitment_artifacts' was unexpected)
ERROR: ['failure_conditions', N, 'action']: 'writer_decision=*' / 'evaluator_decision=*' rejected
Each of these errors is closed by exactly one Schema 13.1 change in §3:
| Validator error | Closed by |
|---|---|
writer_full / evaluator_full not in mode enum |
§3.2 mode enum extension |
panel_size required |
§3.3.5 reviewer-conditional gate |
measurement_procedure required |
§3.3.1 reviewer-conditional gate |
pre_commitment_artifacts not allowed |
§3.4.1 new top-level optional field |
disagreement_handling not allowed |
§3.4.2 new top-level optional field |
| Writer / evaluator action enum values rejected | §3.3.3 mode-conditional action enum |
| F0 contains action mismatch | §3.3.4 mode-conditional F0 contains |
The implementation PR ships the schema diff and these templates as one atomic commit so the validator transitions from "rejects v3.6.6 templates and accepts reviewer templates" to "accepts both" in a single step, preserving the §3.6 byte-equivalence promise for reviewer contracts throughout the rollout. Staging the templates under a non-live path during the spec phase (e.g. docs/design/v3.6.6-templates/) was considered and rejected: it would require a copy or move step at implementation time that risks introducing drift between the spec-branch artefact and the shipped artefact, defeating the co-load-bearing review benefit above. shared/contracts/README.md carries a parallel spec-branch warning so anyone reaching the contracts directory through the README has the same context as readers of this section.
Note for cross-model review: codex review against the cumulative spec-branch diff (origin/main baseline) repeatedly surfaces this concern because the reviewer reads the JSON files in isolation and cannot see the §4.0 / README treatment unless those documents are themselves in the diff. The treatment above is the project's sustained answer; the design call has been settled twice on the spec branch (§4 patch round-2 and again on the cumulative-diff round-4) and is not pivot-pending.
The two templates follow the v3.6.2 shared/contracts/reviewer/*.json conventions:
contract_id matches ^[a-z_]+/[a-z_]+/v\d+$ (schema rule). Writer: academic_paper/writer_full/v1. Evaluator: academic_paper/evaluator_full/v1. The first segment names the consumer skill (academic_paper) rather than the role (reviewer), keeping the convention that the path identifies the artefact's home directory under shared/contracts/.baseline_version is v3.6.6 for both templates (the ARS suite version that introduces them). The baseline_version regex ^v\d+\.\d+\.\d+$ accepts v3.6.6 directly.panel_size is omitted from both templates (per §3.3.5 — single-agent generators do not have a panel; the field is no longer top-level required).acceptance_dimensions[].description is one sentence, ≤ 1.5 lines wrapped, mirroring reviewer template prose density.failure_conditions[].expression references dimension priority and score band ('pass' / 'warn' / 'block') in natural language, matching the synthesizer-recognised vocabulary in academic-paper-reviewer/references/sprint_contract_protocol.md §4. Generator-mode templates extend that vocabulary with mode-specific verbs (writer: "self-repair", "escalate"; evaluator: "request revision", "dissent note", "reviewer stage") that §7's lint surface and sprint_contract_protocol.md extension covers.failure_conditions[].severity is an ordinal anchor (not absolute), aligned to v3.6.2 reviewer ranges so the synthesizer's severity-precedence resolution behaves consistently across modes: 95 = round-2-exhausted escalation (generator-only; must outrank mandatory-block to break in-pair revision loop — see §4.3 F5 rationale), 90 = mandatory hard-block, 75 = mandatory warn (writer-only F4 — see §4.2), 70 = mandatory warn / aggregation (evaluator F2 / reviewer F2), 60 = high block, 55 = normal-priority block (evaluator-only F6 — see §4.3), 50 = evaluator dissent note (mid-tier non-mandatory warn), 40 = high warn, 10 = F0 accept.override_ladder and agent_amendments are omitted by default (both schema-optional). The shipped reviewer templates also omit them; v3.6.6 preserves that posture.cross_reviewer_quantifier is omitted from every failure_conditions[] entry (per §3.3.2 — required only for reviewer modes). Generator-mode runtime treats every condition as evaluated by a single agent (the writer for writer_full, the evaluator for evaluator_full), so no panel aggregation is involved.shared/contracts/writer/full.jsonPurpose. Gates draft_writer_agent Phase 4 with a paper-blind Phase 4a pre-commitment (acceptance criteria paraphrase, per §3.4.1 — paper-specific section outlines / CER chains / word counts are read from upstream runtime artefacts at Phase 4a / 4b runtime, not committed in the frozen contract baseline) before paper-visible drafting in Phase 4b. Encodes the seven Quality Gates from academic-paper/agents/draft_writer_agent.md as acceptance_dimensions, with four writer-specific failure conditions plus F0 accept.
acceptance_dimensions — seven dimensions extracted from draft_writer Quality Gates:
| id | name | description | priority |
|---|---|---|---|
| D1 | section_completeness | Every section in the Structure Architect's approved outline is written; no [MATERIAL GAP] placeholders remain unresolved at handoff to citation_compliance. |
mandatory |
| D2 | citation_density | Every factual claim carries at least one citation; uncited claims either gain a source or are rewritten with hedging language. | mandatory |
| D3 | argument_blueprint_fidelity | Each section follows the CER chains delivered by argument_builder_agent (the Argument Blueprint runtime artefact); topic sentences match the claim, evidence cites assigned sources, and explanation links back to the section purpose. The Argument Blueprint, not the contract, is the source of CER chains. | mandatory |
| D4 | total_word_count | Final draft total stays within ±10% of the target declared in the Paper Configuration Record. | high |
| D5 | per_section_word_count | Each section stays within ±15% of the per-section allocation declared in the Paper Configuration Record. | high |
| D6 | paragraph_structure | At least 80% of body paragraphs follow TEEL (Topic / Evidence / Explanation / Link). Introduction-first and conclusion-last paragraphs are exempt. | normal |
| D7 | register_consistency | Academic register holds throughout; no colloquial mixing; discipline-appropriate phrasing per the Paper Configuration Record. | normal |
revision_response is intentionally not a v3.6.6 dimension. It is a revision-mode behavior; v3.6.6 covers academic-paper/full mode only (§1 in-scope item 1, §2.1 Q1). Revision-mode dimensions and contract are deferred to v3.6.7+ / v3.7.x. transition_completeness is folded into D6 description rather than a separate dimension to keep dimension granularity at the same level as the reviewer template (5 dimensions in reviewer/full.json); transitions are a sub-property of paragraph structure, not an independent quality axis.
pre_commitment_artifacts. Writer Phase 4a paper-blind output, structured per §3.4.1:
acceptance_criteria_paraphrase.minimum_dimensions: "all" — writer must paraphrase all seven dimensions before reading the paper context, mirroring the reviewer template's paraphrase_minimum_dimensions: "all".structure_plan.section_outline, structure_plan.word_count_per_section, and argument_plan.claim_evidence_chains — were intentionally not included in pre_commitment_artifacts. They are paper-specific runtime values that vary across papers (different paper types, different word-count targets, different argument structures), and embedding them in the frozen contract baseline would either force orchestrators to mutate the frozen contract before Phase 4a (violating v3.6.2's frozen-baseline invariant) or hard-code IMRaD assumptions onto non-IMRaD paper types (literature reviews, theoretical, case study, policy brief — all supported by writer_full per draft_writer_agent.md's Paper Type Adjustments table). Source-of-truth for each:structure_architect_agent output, the Paper Outline runtime artefact). D1 section_completeness is evaluated against that outline.intake_agent output). D4 / D5 phrase their gates as "stays within ±N% of the PCR target".argument_builder_agent output). D3 argument_blueprint_fidelity is evaluated against the Blueprint's CER chains, not against a contract-embedded copy.
The §3.4.1 schema reflects this: pre_commitment_artifacts.required lists only acceptance_criteria_paraphrase. The pre_commitment_artifacts object shape is preserved (rather than promoting acceptance_criteria_paraphrase to top level) so v3.7.x can extend the structured object with new invariant, paper-agnostic policy without an architecture refactor. See §8.2 R2 for the eviction pattern as a forward-looking authoring rule.failure_conditions:
| id | severity | expression | action |
|---|---|---|---|
| F1 | 90 | any mandatory dimension scores 'block' | writer_decision=escalate_to_evaluator |
| F4 | 75 | any mandatory dimension scores 'warn' | writer_decision=revise_in_phase_4b |
| F2 | 60 | any high-priority dimension scores 'block' | writer_decision=revise_in_phase_4b |
| F3 | 40 | any high-priority dimension scores 'warn' | writer_decision=revise_in_phase_4b |
| F0 | 10 | every mandatory dimension scores 'pass' | writer_decision=accept |
Severity ordering rationale:
escalate_to_evaluator (not revise_in_phase_4b) because mandatory dimension blocks generally signal an upstream gap the writer cannot close in-phase: a missing source for D1 / D2, or a CER chain that the argument_builder did not deliver for D3. The writer hands off to the evaluator, whose disagreement_handling (§4.3) decides whether to request_revision (loops back to writer with concrete additions), accept_with_dissent_note (proceeds, logs the gap), or flag_for_reviewer_stage (routes to the external academic-paper-reviewer skill).revise_in_phase_4b (not escalate_to_evaluator) because mandatory-warn cases are recoverable in-phase: D1 section_completeness 'warn' typically means a section is short but written (writer expands); D2 citation_density 'warn' typically means uncited factual claims that can be either cited from the assigned source list or rewritten with hedging language; D3 argument_blueprint_fidelity 'warn' means partial CER chain coverage that the writer can complete within Phase 4b. F4 sits between F1 (90) and F2 (60) at severity 75 to mark mandatory dimensions as more severe than high-priority dimensions even at the warn band, while still routing to local self-repair rather than upstream escalation. Without F4, a paper with mandatory-dim 'warn' but no 'block' on any dimension would fall through every other condition and leave the writer phase without a terminal action — a state the synthesizer cannot resolve.revise_in_phase_4b because high-priority dimensions in this template (total_word_count, per_section_word_count) are local-state failures the writer can resolve by trimming or expanding within the same phase without upstream replanning. The Paper Configuration Record carries the per-section word-count target; F2 ('block') corresponds to deviations beyond ±20% of the PCR target (writer must restructure section evidence budget); F3 ('warn') corresponds to ±15-20% deviations (writer adjusts paragraph length or transition density).The five-condition shape (F1 / F4 / F2 / F3 / F0) extends reviewer/full.json's four-condition F1 / F2 / F3 / F0 ordinal density with one additional condition (F4) that closes the mandatory-warn gap surfaced during cross-template codex review. There is still no F-condition for normal-priority dimensions (D6 / D7) — by v3.6.2 convention, normal-priority dimensions inform the synthesizer's narrative but do not on their own fire a failure condition. Synthesizer behavior for normal-priority dimensions is documented in sprint_contract_protocol.md §4 and unchanged by v3.6.6.
shared/contracts/evaluator/full.jsonPurpose. Gates peer_reviewer_agent Phase 6 with a paper-blind Phase 6a pre-commitment (contract paraphrase + scoring plan + disagreement-handling plan) before paper-visible evaluation in Phase 6b. Encodes the five-dimension peer-review rubric from academic-paper/agents/peer_reviewer_agent.md as acceptance_dimensions, with six evaluator-specific failure conditions plus F0 accept.
acceptance_dimensions — five dimensions mirroring the existing peer-review rubric:
| id | name | description | priority |
|---|---|---|---|
| D1 | originality | Novel contribution to the field; new framework, new evidence, or unique perspective; not a restatement of existing work. | high |
| D2 | methodological_rigor | Research design matches the question; method is appropriate, replicable, and limitations are disclosed. | mandatory |
| D3 | evidence_sufficiency | Every claim is supported by adequate citations or data; no unsupported assertions; no cherry-picking. | mandatory |
| D4 | argument_coherence | Logic flows across sections; transitions are smooth; thesis-to-conclusion alignment holds; no self-contradiction. | high |
| D5 | writing_quality | Language is clear and precise; grammar, formatting, and citation style are consistent; readability fits the venue. | normal |
Priority assignment rationale: D2 + D3 are mandatory, mirroring the reviewer template's methodology_rigor + domain_accuracy mandatory pair (rigor and evidence are the dual hard floors of a peer-review-grade paper). D1 originality is high rather than mandatory because an evidence-rigorous paper with weak originality is salvageable via accept_with_dissent_note (F4 below); a hard mandatory block on originality would conflict with the existing peer_reviewer rubric's behavior of allowing minor revisions on weak-originality papers. D4 mirrors the reviewer template's cross_disciplinary_relevance high; D5 mirrors writing_and_structure normal.
The five dimensions correspond directly to the existing five-dimension rubric in peer_reviewer_agent.md Section 4 and the percentage weights table; v3.6.6 does not alter the weights or rubric scoring scale (1-10 per dimension, weighted overall). The contract layer expresses dimension priorities (mandatory / high / normal) for synthesizer F-condition resolution, while the runtime rubric continues to drive the peer-review scoring narrative.
disagreement_handling. Evaluator Phase 6a paper-blind output, structured per §3.4.2:
paraphrase_minimum_dimensions: "all" — evaluator Phase 6a paper-blind output must paraphrase all five acceptance_dimensions before reading the paper draft. This mirrors reviewer template measurement_procedure.paraphrase_minimum_dimensions: "all" and writer template pre_commitment_artifacts.acceptance_criteria_paraphrase.minimum_dimensions: "all". Schema 13.1 §3.4.2 makes this field hard-required on disagreement_handling so the Phase-6a anti-drift gate cannot be skipped by an otherwise valid template / orchestrator output — without the schema-level requirement, an evaluator could validate-pass and still go straight to paper-visible scoring without paraphrasing the contract.scoring_plan.per_dimension_criteria — five entries (one per dimension), each with dimension_id, what_to_look_for, what_triggers_block, what_triggers_warn. Block / warn anchors are extracted from the existing peer_reviewer rubric: block corresponds to the rubric's 3-4 'Below Average' band description (e.g., D2 block: "Method does not match RQ; significant design flaws; limitations not discussed"); warn corresponds to the 5-6 'Acceptable' band description (e.g., D2 warn: "Fundamentally sound method but insufficiently detailed; some choices lack justification"). Full per-dimension criteria are in the shipped JSON; this spec section names the source rather than reproducing all five entries.pre_commitment_check_protocol.check_writer_artifact: "pre_commitment_artifacts" — pinned to the writer's pre-commitment artifact name per §3.4.2 const. Evaluator Phase 6a verifies the writer's Phase 4a output before evaluating the Phase 4b draft.disagreement_resolution.on_dimension_disagreement: "evaluator_decision=request_revision" — strict default. The "disagreement" is between the evaluator's pre-committed scoring_plan.per_dimension_criteria (Phase 6a paper-blind output) and the evaluator's post-paper observation (Phase 6b), specifically when a dimension hits its pre-committed what_triggers_block or what_triggers_warn anchor. It is not anchored on numeric writer pre-commitments (the writer does not emit per-dimension numeric scores; pre_commitment_artifacts carries only acceptance_criteria_paraphrase per §3.4.1). Per the design call (§2.3), the contract baseline treats any mandatory-dimension trigger as a revision round; runtime evaluator may downgrade non-mandatory triggers to accept_with_dissent_note (this is exactly the F4 path in §4.3).disagreement_resolution.on_structural_drift: "evaluator_decision=request_revision" — strict default. Structural drift means the actual draft deviates from the Structure Architect's approved outline (the upstream runtime artefact from structure_architect_agent that the writer follows in Phase 4b — sections added, removed, or merged in a way the approved outline did not call for). Word-count drift is not part of on_structural_drift: per-section word counts live in the Paper Configuration Record (per §4.2), so the writer's own D4 / D5 dimensions plus F2 / F3 cover word-count deviation; what reaches the evaluator's on_structural_drift is section-shape drift only.failure_conditions:
| id | severity | expression | action |
|---|---|---|---|
| F1 | 90 | any mandatory dimension scores 'block' | evaluator_decision=request_revision |
| F2 | 70 | one or more mandatory dimensions score 'warn' | evaluator_decision=request_revision |
| F3 | 60 | any high-priority dimension scores 'block' | evaluator_decision=request_revision |
| F6 | 55 | any normal-priority dimension scores 'block' | evaluator_decision=request_revision |
| F4 | 50 | the evaluator's own scoring plan triggers 'warn' on a high or normal-priority dimension AND no mandatory dimension scores 'block' or 'warn' | evaluator_decision=accept_with_dissent_note |
| F5 | 95 | round 2 still shows mandatory-dimension block after a writer revise_in_phase_4b attempt |
evaluator_decision=flag_for_reviewer_stage |
| F0 | 10 | every mandatory dimension scores 'pass' AND no dimension scores 'block' | evaluator_decision=accept |
Severity ordering rationale:
request_revision rather than the reviewer's editorial_decision=* because the evaluator role is one notch upstream of editorial decision: it loops back to the writer for a revision round, not to the editor for a publication call. F2's expression diverges from the reviewer template's wording: reviewer F2 reads "two or more mandatory dimensions score 'warn' or worse" because reviewer panels aggregate across multiple reviewers and "two or more" is a cross-reviewer count threshold. The evaluator template runs a single agent (no panel), so "two or more" has no panel-aggregation anchor; v3.6.6 evaluator F2 reads "one or more mandatory dimensions score 'warn'" to close the single-mandatory-warn gap that would otherwise leave the evaluator without a terminal action when exactly one mandatory dimension warns and no dimension blocks.writing_quality scoring 'block' (the rubric's 1-2 'Poor' band — "Difficult to understand; numerous grammar errors; colloquial register; citation format inconsistent across the manuscript") is a peer-review-grade reject signal that must request revision rather than accept. The evaluator's normal-priority dimension is not symmetric with the writer's normal-priority dimensions (D6 paragraph_structure / D7 register_consistency in writer/full.json). Writer normal-dim conventions follow v3.6.2's "narrative-only, no F-condition" pattern because writer normal-dim is the writer's self-quality indicator, not a verdict. Evaluator normal-dim is a verdict the evaluator emits about the paper; a normal-dim block must propagate to the synthesizer rather than be silenced. The asymmetry is documented in sprint_contract_protocol.md §4 (extended in v3.6.6) and is one place where evaluator F-condition shape diverges from writer F-condition shape on purpose.scoring_plan.per_dimension_criteria.what_triggers_warn for each dimension) rather than on numeric writer pre-commitments — the writer's pre_commitment_artifacts carries only acceptance_criteria_paraphrase per §3.4.1, not per-dimension numeric score predictions. F4's "no mandatory dimension scores 'block' or 'warn'" guard ensures F4 fires only when nothing higher-severity also matches, so the synthesizer's severity-precedence resolution does not need to disambiguate F4 against F1 / F2.revise_in_phase_4b once in round 2 and still shows mandatory-dimension block. Mechanical severity-precedence resolution would otherwise pick F1 (also matching mandatory block at 90), emit request_revision, and trap the loop in a third revision round that the contract does not authorise. Putting F5 at 95 forces the synthesizer to escalate to flag_for_reviewer_stage (the external academic-paper-reviewer skill) the moment round-2 mandatory block recurs. F5's expression is round-aware (round 2 still shows mandatory-dimension block after a writer revise_in_phase_4b attempt), so on round 1 it does not match and F1 fires normally; the 95 severity only takes effect once F5's round-2 precondition is met. This is the only F-condition in the v3.6.6 contracts that exceeds the reviewer-template 90 severity ceiling, and it does so because v3.6.6 introduces in-pair revision rounds that the reviewer template did not contemplate.shared/contracts/reviewer/full.json and shared/contracts/reviewer/methodology_focus.json are not modified by v3.6.6. The §3.6 backward-compatibility proof (every reviewer-only constraint preserved via reviewer-conditional allOf branches) means both files validate byte-equivalent under Schema 13.1 without change. This is the v3.6.2 zero-touch promise that v3.6.6 inherits.
The reserved reviewer modes without shipped templates (reviewer_re_review, reviewer_calibration, reviewer_guided, per shared/contracts/README.md) remain reserved enum values without templates. v3.6.6 does not add them; their addition is a post-v3.6.6 patch when each mode's pattern is concrete.
shared/contracts/README.md already carries a spec-branch warning section (added on the spec branch alongside this design doc) that lists the writer/ and evaluator/ directories as design-time artefacts and points readers at §4.0 for the sequencing rationale. The v3.6.6 implementation PR will close out the README update by:
main).reviewer_* / writer_* / evaluator_* are the three current mode families, with one shipped template family member each at v3.6.6.The "How to add a new template" subsection stays verbatim — the procedure (add the file, run the validator, update protocol-doc and synthesizer pattern list) is unchanged for new templates regardless of mode family.
Four boundaries between §4 and downstream sections:
draft_writer_agent Phase 4a / 4b and peer_reviewer_agent Phase 6a / 6b as separate calls is in §5 (the L2 open question).argument_plan because the field is removed from pre_commitment_artifacts entirely (§3.4.1 / §4.2); §7 still covers the new mode-conditional schema branches and the writer/evaluator template additions.§5 specifies how academic-paper Phase 4 (drafting) and Phase 6 (peer review) are split into paper-blind / paper-visible call pairs gated by the writer_full and evaluator_full contracts defined in §3 + §4. The split mechanism mirrors academic-paper-reviewer/references/sprint_contract_protocol.md (the v3.6.2 reviewer pattern) but adapts it for single-agent generator modes that have no panel and (for the writer) no scoring_plan.
§5 covers what academic-paper orchestration must add to support contract-gated phase splits — call sequencing, lint protocols, retry/abort semantics, delimiter naming, cross-phase artefact handoff, backward compatibility surfaces, and the academic-pipeline integration boundary. §5 does not specify validator code paths, test fixtures, A/B evidence, or implementation PR patches; those are §6 / §7 / implementation-PR scope per §5.6.
Five orchestration audit targets were inspected against branch head bd93d5e to establish what §5 must add and what §5 must leave alone:
| Target | File | Conclusion |
|---|---|---|
academic-paper skill orchestration |
academic-paper/SKILL.md |
Phase 4 (drafting) and Phase 6 (peer review) are described as single-call phases. No native paper-blind / paper-visible split exists. v3.6.6 must add the split. |
| Writer agent | academic-paper/agents/draft_writer_agent.md |
Single Phase 4 invocation. All upstream artefacts (Paper Configuration Record, Paper Outline, Argument Blueprint, Annotated Bibliography, optional Style Profile, optional Knowledge Isolation Directive) are made available simultaneously to the agent. No phase split exists. |
Evaluator agent (in-pair, lives in academic-paper) |
academic-paper/agents/peer_reviewer_agent.md |
Single Phase 6 invocation. Five-dimension rubric (Originality / Methodological Rigor / Evidence Sufficiency / Argument Coherence / Writing Quality) applied in one call against the full draft. This is the in-pipeline academic-paper Phase 6 evaluator — the in-pair self-quality gate before handoff out of academic-paper. It is not the v3.6.2 sprint contract reviewer (different skill). |
| v3.6.2 sprint contract reviewer (separate skill) | academic-paper-reviewer/references/sprint_contract_protocol.md, academic-paper-reviewer/agents/*.md |
Two-phase split exists: paper-content-blind Phase 1 → paper-visible Phase 2 with <phase1_output>...</phase1_output> data delimiter, Phase 1 lint (5 structural checks), Phase 2 lint (6 structural checks), multi-dissent retry from Phase 1 once, panel cardinality invariant [PANEL-SHRUNK] abort, synthesizer three-step protocol with severity precedence. v3.6.6 evaluator borrows the structural pattern but drops panel + multi-dissent — see §5.2.3. |
academic-pipeline orchestrator |
academic-pipeline/SKILL.md |
Stage 2 (WRITE) dispatches academic-paper in plan or full mode; when full, all of Phase 0–7 runs including in-pair Phase 6 evaluator. Stage 3 (REVIEW) dispatches the separate academic-paper-reviewer skill (5-panel external editorial review). Phase 6 evaluator and Stage 3 reviewer are different reviews at different layers, not duplicate reviews — see audit conclusion 2 below. |
Three audit conclusions follow:
Audit conclusion 1: Writer / evaluator have no native two-phase split. v3.6.6 must add the orchestration glue inside academic-paper. This closes §8.1 L2 (the open question "does existing academic-paper orchestration support separate paper-blind and paper-visible agent calls?"). The answer is no; v3.6.6 must add the glue. The glue lives inside academic-paper/SKILL.md plus the two agent files, not inside academic-pipeline.
Audit conclusion 2: Phase 6 evaluator (in academic-paper, in-pair self-quality gate) and Stage 3 reviewer (in academic-paper-reviewer, external editorial 5-panel) are different review layers, not duplicate reviews. Phase 6 evaluator is the writer's self-quality floor before handoff out of academic-paper (single-agent, max two in-pair revision loops). Stage 3 reviewer is the external editorial review (five-panel double-blind). v3.6.5 already runs both in pipeline path when Stage 2 dispatches academic-paper full: Stage 2 runs Phase 1–7 (including Phase 6 evaluator); Stage 3 dispatches academic-paper-reviewer separately. (Stage 2 dispatches academic-paper in plan or full mode; the Phase 6 evaluator only runs under full. See §5.5.4 for the v3.6.6 contract gate's mode-conditional applicability.) v3.6.6 does not change the Stage 2 / Stage 3 boundary — it upgrades the in-pair Phase 6 evaluator (pre-existing prose-only behaviour) into a contract-gated hard rule. Escalation hierarchy is encoded in §4.3 evaluator F5 flag_for_reviewer_stage.
There are exactly two paths into Stage 3 reviewer dispatch, settled by §4.3 evaluator F-condition resolution:
evaluator_decision=accept or F4 evaluator_decision=accept_with_dissent_note. The draft has cleared the in-pair self-quality floor; Stage 3 reviewer receives an evaluator-passed (or evaluator-passed-with-dissent-note) draft. Standard editorial review proceeds.evaluator_decision=flag_for_reviewer_stage because the in-pair revision loop has exhausted at round 2 with mandatory-dimension block recurring. The draft has not cleared the self-quality floor; Stage 3 reviewer receives a flagged non-passed draft with the escalation context (the F5 trigger, the round-2 evaluator output, the writer's failed revise_in_phase_4b attempt). The external editorial review then operates with the additional knowledge that the in-pair gate could not resolve the issue.Stage 3 reviewer's 5-panel sprint contract is invariant across both paths; only the Editorial Decision Letter's framing differs (standard editorial vs failure-of-in-pair-gate context). [GENERATOR-PHASE-ABORTED] (§5.2.4) does not constitute a valid Phase 6b emission and therefore cannot enter either Stage 3 path; it is a phase-abort signal, not an evaluator decision.
Audit conclusion 3: §5 home turf is academic-paper (SKILL plus the two agent files). academic-pipeline boundaries are unchanged. The phase split is internal to academic-paper. Per the v3.6.5.2 home-turf-vs-edge-client lesson (feedback_dont_lose_sight_of_purpose.md), generator-evaluator orchestration glue belongs inside academic-paper, not pushed into academic-pipeline because pipeline is "the orchestrator" — pipeline orchestrates between skills; what is inside academic-paper Phase 4 / Phase 6 is academic-paper's job. The v3.6.2 sprint contract pattern lives in academic-paper-reviewer; v3.6.6 is the first time academic-paper itself gains contract-gated phase splits.
The split adds four new call boundaries inside academic-paper, each a separate model invocation with its own system prompt and user content:
System prompt vs user content discipline mirrors sprint_contract_protocol.md §2 reviewer pattern verbatim: system prompt carries invariant policy text only (the phase sub-section instructions from the agent file's ## v3.6.6 Generator-Evaluator Contract Protocol block, the lint description, the phase-boundary tag conventions). User content carries the contract JSON (re-injected per call) plus the runtime inputs allowed at that phase (paper metadata, <phase4a_output> / <phase6a_output> delimiter blocks, upstream drafting artefacts, the paper draft). All dynamic LLM output (Phase Na runtime emissions, paper content) lives in user content via data delimiters, never in the system prompt — this prevents accidental elevation of dynamic per-paper content into the invariant policy surface.
Per-call placement:
### Phase 4a — Writer paper-blind pre-commitment sub-section of the writer agent's ## v3.6.6 Generator-Evaluator Contract Protocol block. User content: writer_full contract JSON + paper metadata only (title, field, word_count). Output: ## Acceptance Criteria Paraphrase section + terminal [PRE-COMMITMENT-ACKNOWLEDGED] tag, lint-checked per §5.2.2.### Phase 4b — Writer paper-visible drafting + self-scoring sub-section of the same block. User content: writer_full contract JSON (re-injected) + Phase 4a output wrapped in <phase4a_output>...</phase4a_output> data delimiter + upstream drafting artefacts (Paper Configuration Record, Paper Outline, Argument Blueprint, Annotated Bibliography, optional Style Profile, optional Knowledge Isolation Directive). Output: ## Draft Body → ## Dimension Scores → ## Failure Condition Checks → ## Writer Decision, lint-checked per §5.2.3.### Phase 6a — Evaluator paper-blind pre-commitment sub-section of the evaluator agent's ## v3.6.6 Generator-Evaluator Contract Protocol block. User content: evaluator_full contract JSON + paper metadata + the writer's most recent <phase4a_output> (the writer artefact the evaluator must verify per §3.4.2 pre_commitment_check_protocol.check_writer_artifact). Output: ## Contract Paraphrase + ## Scoring Plan (per-dimension dimension_id / what_to_look_for / what_triggers_block / what_triggers_warn) + terminal [PRE-COMMITMENT-ACKNOWLEDGED] tag, lint-checked per §5.2.2.### Phase 6b — Evaluator paper-visible scoring + decision sub-section of the same block. User content: evaluator_full contract JSON (re-injected) + Phase 6a output wrapped in <phase6a_output>...</phase6a_output> + the writer's <phase4a_output> (per §3.4.2 pre_commitment_check_protocol, required for evaluator_full mode per disagreement_handling.required) + the writer Phase 4b draft (the artefact under review). Output: ## Dimension Scores → ## Failure Condition Checks → ## Review Body → ## Evaluator Decision, lint-checked per §5.2.3.The four-call structure mirrors sprint_contract_protocol.md §2 reviewer Phase 1 → Phase 2 transition verbatim, applied per agent (writer for Phase 4a/4b, evaluator for Phase 6a/6b). Reviewer-side surfaces stay zero-touch: the validator (scripts/check_sprint_contract.py), the reviewer contract templates (shared/contracts/reviewer/full.json, methodology_focus.json), and the reviewer agent prompts are byte-equivalent across v3.6.5 → v3.6.6. The implementation PR will add a one-line cross-reference in sprint_contract_protocol.md pointing to this §5 (covered under §5.6 implementation-PR scope).
Mode-specific structural check counts, per sprint_contract_protocol.md §4 enumeration convention:
## Contract Paraphrase, ## Scoring Plan, terminal [CONTRACT-ACKNOWLEDGED]); paraphrase paragraph count ≥ measurement_procedure.paraphrase_minimum_dimensions; one ### <Dn>: <name> subsection per acceptance dimension; each scoring_plan subsection contains measurement_procedure.scoring_plan_schema.required lines; Phase 1 content references contract JSON + paper metadata only (no full draft / paper content).## Acceptance Criteria Paraphrase, terminal [PRE-COMMITMENT-ACKNOWLEDGED]); paraphrase paragraph count ≥ pre_commitment_artifacts.acceptance_criteria_paraphrase.minimum_dimensions; Phase 4a content references contract JSON + paper metadata only. No ## Scoring Plan section — writer_full carries no scoring_plan (§4.2). Therefore no scoring_plan subsection lint, no scoring_plan_schema lint.## Contract Paraphrase, ## Scoring Plan, terminal [PRE-COMMITMENT-ACKNOWLEDGED]); paraphrase paragraph count ≥ disagreement_handling.paraphrase_minimum_dimensions; one ### <Dn>: <name> subsection per acceptance dimension; each scoring_plan subsection contains disagreement_handling.scoring_plan.per_dimension_criteria four-field shape (dimension_id, what_to_look_for, what_triggers_block, what_triggers_warn); Phase 6a content references contract JSON + paper metadata + the writer's <phase4a_output> only (no full draft / paper content).The asymmetry between writer (3) / evaluator (5) / reviewer (5) is deliberate. It tracks the schema: writer has no scoring_plan (3 minimal checks); evaluator has scoring_plan but is single-agent (5 mid-tier checks); reviewer has scoring_plan + panel (5 checks at Phase 1 since panel-multi-dissent is a Phase 2 concern). This is "design-symmetric, not surface-symmetric" — the lint count is the schema's downstream expression.
Retry semantics mirror reviewer Phase 1: lint failure on the first attempt → retry once with the specific lint gap hinted in the system prompt; second failure → mark this role unusable and emit [PROTOCOL-VIOLATION: phaseNa_lint_failed=true]. The retry-once-from-Phase-Na mechanism is preserved for all three modes (writer / evaluator / reviewer) as the standard recovery path. Abort semantics differ by mode: writer / evaluator second-failure mark-unusable triggers [GENERATOR-PHASE-ABORTED] (§5.2.4) — phase-level abort with no panel cardinality invariant. Reviewer second-failure mark-unusable does not trigger [GENERATOR-PHASE-ABORTED]; instead the reviewer is dropped from the panel and the v3.6.2 panel cardinality invariant applies ([PANEL-SHRUNK] if len(usable_phase2_outputs) < panel_size after all reviewers complete, per sprint_contract_protocol.md §6). [GENERATOR-PHASE-ABORTED] is generator-mode only.
Schema field name vs runtime emission distinction is enforced in §5 prose strictly: pre_commitment_artifacts (snake_case, backticks) is the schema field name in shared/sprint_contract.schema.json — a configuration declaration in the frozen contract baseline. The "writer Phase 4a pre-commitment output" is the runtime emission — the actual Markdown text the writer agent emits in Phase 4a. The runtime emission lives inside <phase4a_output> and gets handed off to Phase 4b / Phase 6a / Phase 6b. Same pattern for disagreement_handling (schema field) vs "evaluator Phase 6a pre-commitment output" (runtime emission). Mixing the two leads to confusion between contract baseline configuration and LLM-generated content, and the strict separation closes the Phase 4a / 6a authoring ambiguity that the round-1 codex review surfaced.
Mode-specific structural check counts:
## Dimension Scores, ## Failure Condition Checks, ## Review Body, ## Editorial Decision); Dimension Scores one-to-one per dimension; Failure Condition Checks one-to-one per failure_conditions[]; multi-dissent retry from Phase 1 once (reviewer-only); consistency check (Phase 2 score substring-matches Phase 1 trigger tokens, vacuous triggers bypassed); editorial decision derivable from F-condition precedence.## Draft Body, ## Dimension Scores, ## Failure Condition Checks, ## Writer Decision; (2) Dimension Scores one-to-one across the seven writer dimensions D1–D7 (§4.2); (3) Failure Condition Checks one-to-one across F1 / F4 / F2 / F3 / F0 (§4.2); (4) Writer Decision derivable from F-condition severity precedence. No multi-dissent retry — writer has no scoring_plan, no dissent anchor; Phase 4b lint failure marks the writer unusable directly per §5.2.4 (no retry). No consistency check — writer Phase 4a emits no scoring_plan trigger tokens; substring-match has no anchor; the lint code path that always vacuously bypasses is dead code, omitted on purpose.## Dimension Scores, ## Failure Condition Checks, ## Review Body, ## Evaluator Decision (mirror reviewer Phase 2 ordering, except ## Scoring Plan Dissent is dropped); (2) Dimension Scores one-to-one across the five evaluator dimensions D1–D5 (§4.3); (3) Failure Condition Checks one-to-one across F1 / F2 / F3 / F6 / F4 / F5 / F0 (§4.3); (4) consistency check (Phase 6b score substring-matches Phase 6a disagreement_handling.scoring_plan.per_dimension_criteria trigger tokens); (5) Evaluator Decision derivable from F-condition severity precedence. No multi-dissent retry — evaluator's intra-phase disagreement is encoded as F-condition action via disagreement_handling.disagreement_resolution.on_dimension_disagreement and on_structural_drift (§3.4.2 / §4.3) rather than as a retry trigger.Multi-dissent retry is reviewer-only, not because the generator templates "drop a feature" but because the underlying mechanism has no anchor in generator modes. Reviewer panels have both ## Scoring Plan Dissent Phase-1 section + panel_size > 1 cross-reviewer aggregation context. Writer has neither (no scoring_plan, no panel). Evaluator has scoring_plan but is single-agent (no panel). §5.2.3 prose explicitly states: "Phase 4b / 6b lint failure → mark unusable, no retry. Multi-dissent retry is a reviewer-only mechanism specific to scoring_plan dissent in a panel context. Generator modes do not have an analog."
Writer Phase 4b output ordering — ## Draft Body → ## Dimension Scores → ## Failure Condition Checks → ## Writer Decision — is structurally different from reviewer / evaluator Phase 2 ordering. Reviewer Phase 2 reads paper → scores → writes review body explaining scores → emits decision (review body is explanation of scores). Writer Phase 4b writes the paper from scratch → scores its own draft → emits decision (draft is the input artefact for self-scoring). The writer's draft must precede the writer's self-scoring; otherwise the score has nothing concrete to score. Evaluator Phase 6b mirrors reviewer ordering (read paper → score → write review body → emit decision) since evaluator is structurally a reviewer of the writer's draft.
Lint count summary (the schema's downstream expression):
| Phase | Reviewer | Writer | Evaluator |
|---|---|---|---|
| Phase 1 / 4a / 6a | 5 | 3 | 5 |
| Phase 2 / 4b / 6b | 6 | 4 | 5 |
Counting convention matches sprint_contract_protocol.md §4 / §5 enumeration: every distinct structural rule the orchestrator enforces is counted as one check, including "decision derivable from F-condition precedence" which the reviewer protocol enumerates as Phase 2 lint check #6.
When a writer or evaluator phase becomes unusable (Phase Na lint twice fail OR Phase Nb lint fail), academic-paper emits a phase-level abort tag and routes to user intervention:
[GENERATOR-PHASE-ABORTED: role=writer, contract=<id>, reason=<lint_failure_kind>] → abort academic-paper Phase 4 → user intervention decides retry / fallback / regression to Phase 3 (Argument Blueprint).[GENERATOR-PHASE-ABORTED: role=evaluator, contract=<id>, reason=<lint_failure_kind>] → abort academic-paper Phase 6 → user intervention decides retry / fallback / regression to Phase 5 (Drafting completion). No automatic Stage 3 escalation from this path: [GENERATOR-PHASE-ABORTED] does not constitute a valid Phase 6b emission and therefore cannot enter the two contract-gated Stage 3 paths defined in §5.1 (standard F0 / F4, exceptional F5). A user may manually dispatch academic-paper-reviewer on the partial draft as an out-of-band override after the abort, but that is a user-driven action outside the contract gate's two-path semantics, not a third orchestrated entry path.academic-paper carries no panel cardinality invariant for writer / evaluator (no panel_size field — §3.3.5 settled this as reviewer-conditional). Therefore there is no [PANEL-SHRUNK] analogue at the generator side; [GENERATOR-PHASE-ABORTED] is phase-level abort (writer Phase 4 OR evaluator Phase 6), distinct from the reviewer's round-level [PANEL-SHRUNK] per sprint_contract_protocol.md §6. Naming alignment with v3.6.2 §11 failure-mode diagnostic table is preserved.
Operational monitor with explicit denominator definition: track [GENERATOR-PHASE-ABORTED] rate over the first three months of v3.6.6 deployment. The denominator is per academic-paper full run — one user-perceived top-level invocation of the academic-paper skill in full mode (whether standalone or dispatched by academic-pipeline Stage 2). Within a single run, writer Phase 4 may abort once and evaluator Phase 6 may abort once, so the abort count per run is 0, 1, or 2. The 5% threshold is (runs_with_any_abort) / (total_runs) — a run is "aborted" if any phase aborts, not abort-count-weighted. The unit corresponds to v3.6.2 reviewer's per editorial round (one panel run) at the user-perceived top-level invocation level, even though the internal mechanics differ (reviewer panel vs single-agent generator).
If the monitored rate exceeds 5%, v3.6.7 introduces graceful-degradation fallback (Phase Na lint failure → degrade Phase 4 / 6 to v3.6.5 single-call mode, log the degradation, do not abort). v3.6.6 ships with abort-only behaviour; the implementation PR will add a forward note to academic-paper/SKILL.md (in a new "Known limitations" section the implementation PR introduces) but no fallback code path is implemented in v3.6.6.
The 5% threshold and three-month observation window cross-reference v3.6.2 reviewer [PANEL-SHRUNK] operational monitor in sprint_contract_protocol.md §6 — same rationale, same numerical anchors; unit semantics differ as noted above.
v3.6.6 enables the contract gate by default for academic-paper full mode. There is no opt-in env flag at the skill boundary — the contract gate is the new pre-existing behaviour, mirroring v3.6.2 reviewer ship pattern. Users who need the v3.6.5 single-call behaviour can either pin to v3.6.5 or use one of the nine non-full modes that remain unchanged in v3.6.6.
The nine non-full modes (plan / outline-only / revision / revision-coach / abstract-only / lit-review / format-convert / citation-check / disclosure) are byte-equivalent across v3.6.5 → v3.6.6. None of them invoke Phase 4 drafting + Phase 6 peer review in the contract-gated form; their phase coverage is narrower or different by mode, and adding the contract gate to them is deferred to v3.6.7+.
Token cost transparency (cross-reference §5.5.5): the incremental cost vs v3.6.5 is the two new paraphrase calls — Phase 4a (writer pre-commitment) and Phase 6a (evaluator pre-commitment). Phase 4b drafting and Phase 6b evaluation reuse the v3.6.5 baseline cost (Phase 4b's draft + self-scoring is one call replacing v3.6.5's Phase 4 single-call drafting; Phase 6b's scoring + review body + decision is one call replacing v3.6.5's Phase 6 single-call peer review). academic-pipeline Stage 2 budget transparency must include the two paraphrase calls in its estimate.
<phase4a_output> / <phase6a_output> delimiter designPhase-numbered delimiters (<phase4a_output> / <phase6a_output>) are chosen over role-named delimiters (e.g., <writer_pre_commitment> / <evaluator_pre_commitment>) for three reasons:
<phase1_output> is already phase-numbered (v3.6.2). Using the same naming convention for v3.6.6 keeps the cross-skill delimiter family consistent with no patch to sprint_contract_protocol.md delimiter naming text.<phase4a_output> and the evaluator's own <phase6a_output>. Phase-numbered delimiters distinguish them by source phase; role-named delimiters would require a "from writer" / "from evaluator" prefix to avoid ambiguity.All dynamic phase outputs live in user content / data delimiters only; system prompt carries only invariant policy. This is a hard rule mirroring sprint_contract_protocol.md §2 step 4 explicit posture: never elevate dynamic LLM output into the system prompt surface.
Per-call placement:
<phase4a_output>...</phase4a_output> (the writer's own Phase 4a runtime emission) plus upstream drafting artefacts.<phase4a_output>...</phase4a_output> (the writer's Phase 4a runtime emission) only — the evaluator has not yet emitted <phase6a_output> at this point.<phase4a_output> (writer's, per §3.4.2 pre_commitment_check_protocol.check_writer_artifact) and <phase6a_output> (evaluator's own, emitted in Phase 6a and handed forward into Phase 6b). Phase 6b also carries the writer Phase 4b draft (the artefact under review). The <phase4a_output> handoff into Phase 6b is unconditional, not optional: §3.4.2 pre_commitment_check_protocol is required for evaluator_full mode per disagreement_handling.required.System prompt across all four phase-call boundaries carries only the invariant phase sub-section instructions + lint description + phase-boundary tag conventions — no per-paper data, and (per §5.2.1) no contract JSON. Contract JSON lives in user content, re-injected per call alongside the runtime inputs allowed at that phase.
| Delimiter | Phase emitting it | Phases consuming it | Semantic gloss |
|---|---|---|---|
<phase1_output> |
Reviewer Phase 1 | Reviewer Phase 2 | Reviewer pre-commitment runtime emission satisfying measurement_procedure (v3.6.2, unchanged). |
<phase4a_output> |
Writer Phase 4a | Writer Phase 4b, Evaluator Phase 6a, Evaluator Phase 6b | Writer pre-commitment runtime emission satisfying pre_commitment_artifacts. |
<phase6a_output> |
Evaluator Phase 6a | Evaluator Phase 6b | Evaluator pre-commitment runtime emission satisfying disagreement_handling. |
§5 prose enforces strict separation:
pre_commitment_artifacts and disagreement_handling. These are configuration declarations in the frozen contract baseline (shared/sprint_contract.schema.json Schema 13.1) that specify what kind of pre-commitment the writer / evaluator must emit (e.g., acceptance_criteria_paraphrase.minimum_dimensions: "all").## Acceptance Criteria Paraphrase / ## Contract Paraphrase + ## Scoring Plan + terminal [PRE-COMMITMENT-ACKNOWLEDGED] tag) that satisfies the schema field's requirements.<phase4a_output>, <phase6a_output>) carry the runtime emission, not the schema field.The contract field reference name "pre_commitment_artifacts" (the string value of pre_commitment_check_protocol.check_writer_artifact) is what the evaluator's pre_commitment_check_protocol schema names as the artefact-class to verify; the runtime carries content satisfying that named contract field. Mixing the two names leads to confusion between contract baseline configuration and LLM-generated content; §5 prose strictly distinguishes them at every mention.
academic-paper SKILL sessionThe writer's Phase 4a runtime emission lives in the academic-paper SKILL conversation context, carried forward via the <phase4a_output> delimiter into:
academic-paper SKILL session — no cross-skill boundary crossed).<phase6a_output>): writer agent's Phase 4a output handed forward into the evaluator agent's Phase 6b call alongside the evaluator's own Phase 6a output.The same delimiter shape works both same-agent-cross-phase and cross-agent-within-skill because both spans live inside the academic-paper SKILL conversation context. Zero Schema 9 (Material Passport) change — the handoff is in-conversation, not via the Material Passport ledger.
A manual session split mid generator-evaluator round (e.g., user closes the Claude Code session between Phase 4a and Phase 4b, or between Phase 6a and Phase 6b) is not supported in v3.6.6. The only supported recovery is to restart academic-paper full mode from Phase 0.
The reason is structural: v3.6.3 reset_boundary[] (Schema 9) operates at pipeline-Stage level, not at academic-paper internal phase level. Stage boundaries fire after academic-paper full mode completes Phase 1–7 in academic-pipeline Stage 2. The internal phases 4a / 4b / 6a / 6b are not boundary points — they are mid-skill transient states without ledger entries.
The pending_decision schema field on reset_boundary[] entries requires ≥2 routed options (e.g., Stage 3 reject / restructure / abort) and is a Stage-level mechanism for deferring MANDATORY branch choices across reset. It is not applicable to phase-level transient state because phase boundaries have no routed options to defer — they are linear progression points within a single skill invocation.
For users on academic-pipeline systematic-review mode with ARS_PASSPORT_RESET=1 enabled (per academic-pipeline/references/passport_as_reset_boundary.md), the v3.6.6 phase-split semantics do not change behaviour. Pipeline Stage 2 boundary fires after academic-paper full mode completes Phase 1–7 (which now includes the in-pair Phase 4 generator-evaluator and Phase 6 evaluator passes). Standard pipeline flow does not fire mid-round boundaries regardless of mode.
The only edge case is manual session split mid-round, and §5.4.2's unsupported declaration covers it — ARS_PASSPORT_RESET=1 does not extend boundary semantics to internal phase transitions; the v3.6.3 reset boundary protocol (passport_as_reset_boundary.md) operates at pipeline-Stage level only, consistent with §5.4.2.
A pre_commitment_history[] Schema 9 extension (carrying writer Phase 4a / evaluator Phase 6a runtime emissions across reset boundaries) is a candidate for v3.6.7+ depending on operational monitoring data and user feedback. The extension would let mid-round session split survive a reset by replaying pre-commitments from the ledger. v3.6.6 carries no such extension; the implementation PR will add a forward note to academic-paper/SKILL.md (in the same new "Known limitations" section noted in §5.2.4) but no schema or code path is implemented in v3.6.6.
academic-paper full modev3.6.6 enables the contract gate by default for academic-paper full mode. This mirrors the v3.6.2 reviewer ship pattern: the contract turns pre-existing behaviour into a hard gate without changing the user-facing flow. Users who run academic-paper full after v3.6.6 ships will see the new visible sections (## Acceptance Criteria Paraphrase from writer Phase 4a, ## Contract Paraphrase + ## Scoring Plan from evaluator Phase 6a) but the overall draft + evaluation output remains structurally similar to v3.6.5 outputs.
The nine non-full modes (plan, outline-only, revision, revision-coach, abstract-only, lit-review, format-convert, citation-check, disclosure) are byte-equivalent across v3.6.5 → v3.6.6. v3.6.6 contract templates and Schema 13.1 changes touch only writer_full and evaluator_full modes; non-full modes do not invoke the new contract gate. This preserves the v3.6.2 first-test-case discipline (mirror reviewer-only first-mode rollout) by limiting v3.6.6 scope to one mode pair on the writer / evaluator side.
academic-paper revision mode in v3.6.6 is byte-equivalent to v3.6.5 — no new contract, no new lint, no phase-split. A revision_response dimension is not a v3.6.6 dimension; the revision-mode contract is deferred to v3.6.7+ as part of the broader generator-mode rollout sequencing (§9 decision log). v3.6.6 ships writer + evaluator full only.
academic-pipeline integration unchanged: Phase 6 evaluator vs Stage 3 reviewer hierarchyacademic-pipeline Stage 2 (WRITE) dispatches academic-paper in either plan or full mode (per academic-pipeline/SKILL.md Stage 2 row). When Stage 2 dispatches academic-paper full, Phase 1–7 includes the in-pair Phase 4 generator-evaluator + Phase 6 evaluator under v3.6.6 contract gates. When Stage 2 dispatches academic-paper plan, the v3.6.6 contract gate does not apply (plan mode is byte-equivalent to v3.6.5 per §5.5.2). Stage 3 (REVIEW) dispatches the separate academic-paper-reviewer skill (5-panel external editorial review under v3.6.2 sprint contract).
Two reviews, two layers (per §5.1 audit conclusion 2). v3.6.6 contract gate operates inside academic-paper Phase 4 / Phase 6 only; the pipeline orchestrator does not learn about the contract gate — pipeline still sees academic-paper as a black-box skill invocation that produces a draft.
Two Stage 3 entry paths (re-stated from §5.1 for orchestration completeness):
accept or F4 accept_with_dissent_note → evaluator-passed draft → Stage 3.flag_for_reviewer_stage → flagged non-passed draft with escalation context → Stage 3.F5 makes the distinction observable; Stage 3 reviewer's 5-panel sprint contract is invariant across paths but the Editorial Decision Letter's framing differs (standard editorial vs failure-of-in-pair-gate context). [GENERATOR-PHASE-ABORTED] (§5.2.4) is not a Stage 3 entry path — it is a phase-abort signal that aborts academic-paper before any Phase 6b emission, and any post-abort academic-paper-reviewer dispatch is a user-driven out-of-band override.
academic-pipeline v3.2 budget transparency must update to include the two new paraphrase calls per academic-paper full invocation:
<phase4a_output> + scoring_plan emission. Mid-size, smaller than Phase 6b.Phase 4b drafting and Phase 6b evaluation reuse the v3.6.5 baseline cost — Phase 4b is one call producing ## Draft Body + self-scoring (replaces v3.6.5 single-call drafting); Phase 6b is one call producing scoring + review body + decision (replaces v3.6.5 single-call peer review). The incremental cost is approximately Phase 4a + Phase 6a, both small relative to Phase 4b / Phase 6b.
v3.6.6 release notes must call out the visible-output change in academic-paper full mode: two new sections appear in v3.6.6 outputs that did not appear in v3.6.5 — ## Acceptance Criteria Paraphrase (writer Phase 4a) and ## Contract Paraphrase + ## Scoring Plan (evaluator Phase 6a). This is a non-breaking visible change (existing downstream consumers parsing the draft body or evaluator review body are unaffected) but warrants explicit disclosure under release-notes-level visibility.
Mirroring §3.7 / §4.6 boundary statements, §5 explicitly defers the following to other sections / artefacts:
academic-paper/SKILL.md text patch (including a new "Known limitations" section carrying the §5.2.4 graceful-degradation forward note + the §5.4.4 pre_commitment_history[] forward note) and the two academic-paper/agents/*.md patches encoding the four-phase call sequence + the per-phase ## v3.6.6 Generator-Evaluator Contract Protocol blocks → implementation PR (the spec branch does not modify those files). On the spec branch, docs/design/2026-04-27-ars-v3.6.6-generator-evaluator-contract-design.md (this file) carries the Schema 13.1 spec text in §3 + §4 + §5; shared/contracts/writer/full.json and shared/contracts/evaluator/full.json are present on the branch as design-time artefacts per §4.0's spec-branch notice (the current shared/sprint_contract.schema.json still titles itself Schema 13 and rejects them by design until the implementation PR lands the schema upgrade atomically).sprint_contract_protocol.md one-line cross-reference pointing to this §5 (per §5.2.1 reviewer zero-touch scope) → implementation PR.§5 closes §8.1 L2 (the open question on whether existing academic-paper orchestration supports separate paper-blind and paper-visible agent calls). The audit findings answer no; v3.6.6 must add the orchestration glue inside academic-paper, and §5.1–§5.5 specify the glue at protocol-level depth. §8.1 L2 row updates from "open" to "closed by §5" once §5 ships.
§6 specifies the A/B evidence package that demonstrates the v3.6.6 generator-evaluator contract gate produces measurable improvement over the v3.6.5 single-call baseline. The A/B fixture, hypotheses, metric computation, ship gate threshold, reproducibility artefacts, and boundary statement are scoped per the v3.6.2 evidence-discipline pattern: process-metric-only, delta-based, directional rather than absolute (§1 design goal 5 / non-goal 1).
§6 covers what the A/B comparison must measure, how the fixture must be composed, how H1–H3 are computed, what threshold gates the ship decision, and what artefacts must land git-tracked at which commit. §6 does not specify the manifest CI lint code path, the treatment runs themselves, the codex judge prompt template / Semantic Scholar API client code, threshold tuning beyond the values frozen here, production telemetry, or the post-fail decision flow; those are §7 / implementation-PR / v3.6.7+ scope per §6.6.
The A/B comparison contrasts two academic-paper full mode call configurations against the same upstream input artefacts:
## Draft Body against the full upstream artefact set; evaluator Phase 6 runs as one call producing ## Dimension Scores + ## Failure Condition Checks + ## Review Body + ## Evaluator Decision against the writer's draft. No paper-blind pre-commitment phase, no <phase4a_output> / <phase6a_output> delimiter handoff. This is the configuration in main at branch base (spec/v3.6.6-generator-evaluator-contract's parent commit on main), exercised against branch-frozen upstream inputs.writer_full and evaluator_full contracts (§3 + §4). This is the configuration the implementation PR ships.The comparison runs both configurations against the same upstream input artefacts — frozen Paper Configuration Record, Paper Outline (or Argument Blueprint / Structure Architect output for non-IMRaD papers), Annotated Bibliography, optional Style Profile, optional Knowledge Isolation Directive — to isolate the contract-gate variable. Same model, same temperature, same prompt invariants except for the v3.6.6 generator-evaluator contract block additions per §5.2.
The A/B comparison is process-metric-only and delta-based per §1 design goal 5: §6 does not claim that v3.6.6 papers are objectively better in any absolute sense, only that the contract gate produces measurable directional improvement on three pre-declared process metrics (H1–H3, §6.3). This inherits the v3.6.2 sprint contract evidence-discipline pattern verbatim.
The A/B fixture lives at tests/fixtures/v3.6.6-ab/ with a manifest.yaml declaring exactly seven papers covering two roles:
Role A — paper-A (six papers, same-input repro for H1–H3 averaging): three paper-type families × two repetitions, drawn from existing deep-research synthesis reports already produced by the suite. The three families span complementary writing modalities and are selected to surface variance the contract gate can plausibly affect, independent of draft_writer_agent's Paper Type Adjustments table (which enumerates adjustments to the implicit IMRaD baseline, not coverage of the production paper-type space):
draft_writer_agent's default writing mode (the table's adjustments are framed against this default).Two repetitions per paper-type family provides minimal within-family variance signal without inflating fixture cost. Six papers is a toy fixture: §6.3 explicitly does not claim statistical significance, only directional signal sufficient for ship-gate adjudication (§6.4).
Role C — paper-C (one paper, v3.6.5 known-fail sanity check): one paper carried over from the existing session log of a v3.6.5 academic-paper full run where the in-pair Phase 6 evaluator's self-scoring passed but the Stage 3 reviewer (external academic-paper-reviewer skill) caught a hallucinated citation that the in-pair evaluator missed. Paper-C carries known_failure_mode: "v3.6.5_in_pair_evaluator_missed_hallucinated_citation_caught_by_stage_3" and failure_evidence pointing to the Stage 3 reviewer output that surfaced the hallucination. Paper-C exists to test whether the v3.6.6 contract gate (writer Phase 4b self-score / evaluator Phase 6a–6b scoring_plan / F-condition checks) catches the hallucination earlier than v3.6.5 did. Paper-C is categorical, not part of H1–H3 averaging (§6.3 / §6.4).
The manifest schema lives at tests/fixtures/v3.6.6-ab/manifest.yaml with the following top-level shape:
fixture_version: semver string (see §6.5 invariant 4 for bump rules).manifest_lint_mode: one of spec_branch | implementation_pr. The mode declares which population stage the manifest expects to be in; CI lint applies different required-vs-optional path checks under each mode (per §6.5 invariant 5). The spec branch lands the manifest under spec_branch mode; the implementation PR upgrades the value to implementation_pr in the same atomic merge state that lands treatment / judge / metrics paths.documentation_paths[]: array of relative paths under tests/fixtures/v3.6.6-ab/ to operator-facing documentation files (e.g., README.md). These are git-tracked operator notes, not fixture data. Listed here so that §6.5 invariant 1 reverse-scan recognises them as known documentation rather than fixture orphans. Required to contain at least the spec-PR-shipped README.md.papers[]: array of seven entries, each carrying:paper_id (slug, required).role (paper-A | paper-C, required).paper_type (free-text family label such as imrad_empirical / literature_review / case_study for paper-A entries; free-text descriptor for paper-C — these labels describe the fixture composition per §6.2 paper-A family selection and are not constrained to mirror draft_writer_agent's Paper Type Adjustments table, required).topic_label (short human label for traceability, required).input_artefacts (object with optional sub-fields paper_configuration_record, paper_outline, argument_blueprint, annotated_bibliography, style_profile, knowledge_isolation_directive; each value is a relative path under inputs/{paper_id}/; at least one sub-field required per paper).baseline_output (object with two required sub-fields, both relative paths under baseline/{paper_id}/v3.6.5/):writer_draft: the v3.6.5 single-call writer Phase 4 output (## Draft Body only — v3.6.5 writer does not self-score).evaluator_review: the v3.6.5 single-call evaluator Phase 6 output (## Dimension Scores + ## Failure Condition Checks + ## Review Body + ## Evaluator Decision in one file).baseline_output is the v3.6.5 academic-paper full-run output that produced the missed-hallucination case (the Stage 3 reviewer excerpt that surfaced the miss is a separate artefact, listed in failure_evidence below).judge_output_baseline (relative path under codex-judge/, required for paper-A entries under both lint modes; optional and expected absent for paper-C since paper-C is categorical and not subject to H1 / H3 codex judging).treatment_output (object with four sub-fields, each a relative path under treatment/{paper_id}/v3.6.6/, mirroring §5.2.1 phase boundaries; all four sub-fields optional under spec_branch mode and required under implementation_pr mode):phase4a_output: writer paper-blind pre-commitment output (## Acceptance Criteria Paraphrase + [PRE-COMMITMENT-ACKNOWLEDGED]).phase4b_output: writer paper-visible drafting + self-scoring output (## Draft Body + ## Dimension Scores + ## Failure Condition Checks + ## Writer Decision).phase6a_output: evaluator paper-blind pre-commitment output (## Contract Paraphrase + ## Scoring Plan + [PRE-COMMITMENT-ACKNOWLEDGED]).phase6b_output: evaluator paper-visible scoring + decision output (## Dimension Scores + ## Failure Condition Checks + ## Review Body + ## Evaluator Decision).judge_output_treatment (relative path under codex-judge/, required for paper-A entries under implementation_pr mode; optional and expected absent for paper-C under both modes).metrics_output (relative path under metrics/, optional and expected absent for paper-C entries under both lint modes since paper-C is categorical and its pass/fail signal is recorded in the top-level summary_output file rather than in a per-paper machine-readable record; for paper-A entries: optional under spec_branch mode, required under implementation_pr mode).known_failure_mode (categorical descriptor string, required) and failure_evidence (relative path under inputs/{paper_id}/ to the Stage 3 reviewer excerpt that surfaced the v3.6.5 miss, required).summary_output (top-level): relative path to the human-readable aggregate summary file under metrics/. This file carries the H1 / H2 / H3 paper-A aggregates plus the paper-C categorical pass/fail entry, so paper-C signal lives in this file even though paper-C carries no per-paper metrics_output. Optional under spec_branch mode, required under implementation_pr mode.§7 will add a manifest CI lint enforcing: fixture_version matches semver regex; manifest_lint_mode is one of the two enum values; len(papers) == 7; len([p for p in papers if p.role == "paper-A"]) == 6; len([p for p in papers if p.role == "paper-C"]) == 1; each of the three paper-A paper_type family labels appears exactly twice (paper-C paper_type excluded from this count); every path declared in any papers[].* sub-field, in top-level summary_output, or in any entry of documentation_paths[] exists git-tracked at the commit under check; every paper-C entry carries non-empty known_failure_mode + failure_evidence; paper-C entries do not carry judge_output_baseline, judge_output_treatment, or metrics_output; mode-conditional path requirements per the spec_branch / implementation_pr semantics above. The lint is §6.5 invariant 5's enforcement mechanism.
Three pre-declared directional hypotheses, computed against the seven-paper fixture per §6.2:
(paper_id, configuration) pair; full output committed per §6.5. Direction: H1 must increase under treatment.deep-research/references/semantic_scholar_api_protocol.md), divided by the total citation count in the output. No LLM judge — this metric is fully programmatic. A Tier 0 verification failure does not by itself constitute a hallucinated/fabricated citation: per the Semantic Scholar protocol, S2_NOT_FOUND only means the citation did not match Tier 0 lookup and the protocol's later tiers (DOI lookup, WebSearch, manual review) are required to classify a Tier 0 miss as fabricated vs. legitimately-missed-by-S2 (e.g., recent preprint, niche venue, indexing lag). H2 measures the Tier 0 verification-fail surface as a quality signal, treating any improvement in Tier 0 pass rate under v3.6.6 as evidence the contract gate caught more sourcing problems before output, without claiming the failed citations are necessarily fabricated. The H2 aggregate is the arithmetic mean of the six paper-A per-paper rates. Reporting form: when H2_baseline > 0, report relative percentage delta (H2_treatment - H2_baseline) / H2_baseline * 100; when H2_baseline == 0 (baseline already clean at Tier 0 across all six paper-A), the relative-delta form is undefined and §6.4's soft gate degenerates to the absolute condition H2_treatment ≤ 0 (treatment must also produce zero Tier 0 verification failures). Direction: H2 must decrease or hold under treatment. Later-tier classification (which Tier 0 failures are actually fabricated) is implementation-PR concern per §6.6.mean(H3_treatment_per_paper) - mean(H3_baseline_per_paper). Direction: H3 must hold or increase under treatment (more or equal self-detected issues = better self-quality awareness, not "more bugs").Paper-C categorical signal (not part of H1–H3 aggregation):
Paper-C is scored categorically under treatment only — does the v3.6.6 contract gate catch the v3.6.5-missed hallucinated citation at any of three observable surfaces? Pass = caught at writer Phase 4b self-score (## Failure Condition Checks flags it) OR evaluator Phase 6b score (Dimension Scores or Failure Condition Checks flags it) OR an F-condition trigger fires (writer F1/F2/F4 or evaluator F1/F2/F6) on the affected dimension. Fail = v3.6.6 mirrors v3.6.5 and misses the hallucination at all three surfaces; the Stage 3 reviewer would still be the only line of catching it.
The codex gpt-5.5 + xhigh judge is invoked twice per H1 / H3 metric ((paper, configuration) matrix per §6.2), in verdict-only mode (no cross-call memory; each invocation receives only the contract excerpt, the paper output, and the per-dimension verdict prompt). H2 invokes Semantic Scholar API directly with no LLM in the loop. The judge prompt templates and the API client implementation are out of scope per §6.6.
The ship gate is a four-condition rubric evaluated after the implementation PR's treatment runs land (per §6.5):
H1_treatment - H1_baseline ≥ +10 percentage points. Failure to meet the threshold blocks ship pending root-cause investigation (not a soft signal).H2_relative_delta ≤ 0% when H2_baseline > 0; H2_treatment ≤ 0 (absolute) when H2_baseline == 0 (per §6.3 H2 reporting form for the baseline-clean degenerate case). Treatment Tier 0 unverified citation rate must not regress relative to baseline. Soft = magnitude reported in metrics/summary.md but not weighted into H1's hard threshold; soft regression triggers P1 review surface, not auto-block.H3_aggregate_delta ≥ 0 (mean self-detected issue count must not regress). Same soft semantics as H2.Ship decision rubric:
H1 +10pp buffer rationale: the +10pp value is buffered down from the original §1 item 11 / §8.1 L3 initial estimate of +20pp after fixture sizing. Worst-case combined variance from two dominant sources contributes approximately ±7–8pp to the observed H1 delta, with a third (small) qualitative source noted but not summed into the dominant total:
## Acceptance Criteria Paraphrase etc.) is orthogonal to per-dimension acceptance verdicts. The qualitative bound is recorded but is not summed into the dominant ±7–8pp because it would double-count judge-non-determinism: the same LLM judge that exhibits ±2–3pp re-run variance is the one absorbing the framing-drift effect.Net worst-case noise floor: ±7–8pp dominant + ≤±1pp qualitative residual = approximately ±8–9pp. A +10pp threshold therefore leaves approximately ±1–2pp signal margin above the worst-case noise floor. This is directional sufficiency, not statistical significance — §6.3 explicitly does not claim significance from a seven-paper toy fixture; the buffer rationale is the toy-fixture analog of a confidence-interval discipline.
H2 / H3 soft semantics: directional only. H2 + H3 magnitudes report in metrics/summary.md but do not aggregate into H1's hard gate. Soft regression (H2 > 0 or H3 < 0) does not auto-block ship; instead it raises a P1 design-question surface for user-driven decision. This avoids over-fitting the ship gate to a noisy multi-dimensional threshold from a seven-paper fixture.
Paper-C categorical fail handling: a fail does not auto-block ship; it surfaces as a P1 design question — the v3.6.6 contract gate inherited the v3.6.5 blind spot at this dimension and the rubric / F-condition severity / writer self-scoring path failed to catch the hallucination. The fix is rubric-level or F-condition-severity-level redesign, not threshold tuning.
§6.5 lists which artefacts must land git-tracked at which PR-merge boundary (the merged-to-main state when the spec PR or the implementation PR lands, not any individual in-progress commit on the working branch). The boundary discipline mirrors §5.6's shared/contracts/writer/full.json design-time-artefact convention: the spec PR ships the structure + baseline + baseline judge under manifest_lint_mode: spec_branch; the implementation PR ships the treatment + treatment judge + metrics atomically under manifest_lint_mode: implementation_pr. Within either PR's working branch, fixture data may be populated across multiple commits (a manifest-schema stub commit followed by data-population commits); the §6.5 invariants apply to the PR's merge-time state, not to every intermediate commit.
Spec PR ship (the merged-to-main state when spec/v3.6.6-generator-evaluator-contract lands; the spec PR contains the §6 spec body commit 2e4d303, follow-up manifest-schema commits, and follow-up fixture data-population commits) must contain:
tests/fixtures/v3.6.6-ab/manifest.yaml — the §6.2 manifest schema instantiated with seven paper entries (six paper-A + one paper-C), fixture_version semver, manifest_lint_mode: spec_branch, documentation_paths[] listing the spec-PR-shipped README.md, and all baseline-side sub-fields declared per §6.2.tests/fixtures/v3.6.6-ab/README.md — operator-facing notes: which commit landed the spec body, which commit(s) populated fixture data, the manifest_lint_mode lifecycle expectation across spec PR vs implementation PR, and where the data originated (deep-research synthesis report references for paper-A; v3.6.5 session log reference for paper-C). Listed under the manifest's documentation_paths[] to satisfy invariant 1 source-of-truth coverage.tests/fixtures/v3.6.6-ab/inputs/{paper_id}/ — frozen upstream artefact snapshots per paper (Paper Configuration Record / Paper Outline / Argument Blueprint / Annotated Bibliography / optional Style Profile / optional Knowledge Isolation Directive, as applicable to the paper type). Paper-C additionally carries the Stage 3 reviewer excerpt referenced by the manifest's failure_evidence field; this excerpt is evidence of the v3.6.5 miss and is distinct from the v3.6.5 academic-paper full-run output (which lives under baseline_output per the next bullet).tests/fixtures/v3.6.6-ab/baseline/{paper_id}/v3.6.5/ — the v3.6.5 single-call configuration outputs for all seven papers (six paper-A and one paper-C). Per the §6.2 baseline_output schema, each paper's directory contains writer_draft.md (Phase 4 single-call output) and evaluator_review.md (Phase 6 single-call output with scores + decision in one file). Paper-C's baseline_output is the v3.6.5 academic-paper full-run output that produced the missed-hallucination case; the Stage 3 reviewer excerpt evidencing the miss lives separately under inputs/{paper_id}/ per the bullet above.tests/fixtures/v3.6.6-ab/codex-judge/{paper_id}-v3.6.5.txt — full text of the codex gpt-5.5 + xhigh verdict-only output for H1 / H3 against each paper-A's baseline (six files, ~150–400 KB each). Paper-C is categorical and not subject to codex H1 / H3 judging, so paper-C carries no judge_output_baseline per §6.2 schema.Implementation PR ship (the merged-to-main state when the PR landing Schema 13.1 + agent file rewrites + treatment runs lands; not on the spec branch) must additionally contain, as the atomic merge state that includes (in any commit topology within that PR):
manifest_lint_mode upgraded from spec_branch to implementation_pr (alongside populating the previously-optional treatment_output.{phase4a,phase4b,phase6a,phase6b}_output sub-fields, paper-A judge_output_treatment, paper-A metrics_output, and top-level summary_output paths).tests/fixtures/v3.6.6-ab/treatment/{paper_id}/v3.6.6/ — the v3.6.6 contract-gated outputs per paper, organised into the four sub-files per §6.2 schema: phase4a_output.md (writer paper-blind pre-commitment), phase4b_output.md (writer paper-visible drafting + self-scoring), phase6a_output.md (evaluator paper-blind pre-commitment), phase6b_output.md (evaluator paper-visible scoring + decision). The three new visible sections per §5.5.6 (## Acceptance Criteria Paraphrase from writer Phase 4a, ## Contract Paraphrase + ## Scoring Plan from evaluator Phase 6a) appear in phase4a_output.md / phase6a_output.md respectively.tests/fixtures/v3.6.6-ab/codex-judge/{paper_id}-v3.6.6.txt — full text of the codex gpt-5.5 + xhigh verdict-only output for H1 / H3 against each paper-A's treatment (six files). Paper-C does not receive codex H1 / H3 judging.tests/fixtures/v3.6.6-ab/metrics/{paper_id}.json — machine-readable per-paper record for paper-A only (six files), each carrying H1 (per-dimension pass/fail), H2 (Tier 0 Semantic Scholar verification result per citation), H3 (deduped issue count) for both baseline and treatment. Paper-C carries no per-paper metrics_output; paper-C's categorical pass/fail signal is recorded in the top-level summary_output file.tests/fixtures/v3.6.6-ab/metrics/summary.md — human-readable aggregate: H1 arithmetic mean delta, H2 relative percentage delta (or absolute degenerate value per §6.3 baseline-zero clause), H3 mean count delta, paper-C categorical pass/fail entry, and ship gate rubric verdict per §6.4.Git-tracked invariants (apply to PR merge-time state per the §6.5 preamble):
tests/fixtures/v3.6.6-ab/ (excluding manifest.yaml itself) must be referenced from the manifest — either via a papers[].* sub-field, via top-level summary_output, or via a documentation_paths[] entry. Any file not so referenced is a fixture-orphan and fails CI lint (§7 reverse-scan check). Conversely, every path declared in the manifest must exist git-tracked at the commit under check, subject to the mode-conditional rules in invariant 5 below.manifest_lint_mode: spec_branch); the treatment outputs, treatment judge, and metrics must land as part of the implementation PR's atomic merge state alongside the manifest_lint_mode flip from spec_branch to implementation_pr. Treatment artefacts must not split across multiple PRs (a half-shipped treatment fixture is a worse reproducibility state than no treatment fixture). Within the implementation PR's working branch, the flip and the path population may live in different commits as long as both are present in the merge state.fixture_version semver bump rules: bump patch for typo fixes, replacing a paper within the same paper_type family, or adding additional codex-judge re-runs without changing the metric definition; bump minor for adding a new paper_type family, changing H1 / H2 / H3 semantic definition, or adjusting the ship gate threshold beyond the values frozen in §6.4 (a minor bump implies §6 spec amendment per §6.6 boundary); bump major when the manifest schema shape itself changes (renaming top-level fields, adding required sub-fields to papers[], extending the manifest_lint_mode enum, or — concretely — the spec_branch → implementation_pr mode flip itself, since that flip is the schema-state transition that the implementation PR ships and the new required paths constitute a schema-shape change in their required-vs-optional posture). The mode flip is therefore not a patch-bump candidate; it is bundled into the implementation PR's major bump for the v3.6.6 schema 13.1 release.spec_branch-mode path declared in the manifest exists git-tracked", which under the §6.2 schema means: documentation_paths[] entries; per-paper input_artefacts.* sub-fields with at least one populated; per-paper (both paper-A and paper-C) baseline_output.writer_draft + baseline_output.evaluator_review; paper-A judge_output_baseline; paper-C known_failure_mode + failure_evidence. A fresh clone of the implementation PR's merge state must satisfy the spec-PR requirements plus, on all seven papers (six paper-A and one paper-C, since paper-C also runs through the v3.6.6 contract gate to test whether it catches the v3.6.5-missed hallucination per §6.3 paper-C categorical signal): treatment_output.phase4a_output / phase4b_output / phase6a_output / phase6b_output. Implementation-PR-mode also requires paper-A-only fields (judge_output_treatment, metrics_output) plus the top-level summary_output; paper-C remains exempt from those paper-A-only requirements per §6.2 schema (paper-C is categorical, no codex H1/H3 judge, no per-paper machine-readable metrics). CI lint (§7) reads manifest_lint_mode and applies the corresponding required-path set.Mirroring §3.7 / §4.6 / §5.6 boundary statements, §6 explicitly defers the following to other sections / artefacts:
manifest_lint_mode from spec_branch to implementation_pr). The spec PR's merge state ships the baseline + baseline judge only under manifest_lint_mode: spec_branch.deep-research/references/semantic_scholar_api_protocol.md) are runtime concerns → implementation PR.fixture_version minor bump per §6.5 invariant 4); it is not an implementation-PR degree of freedom.[GENERATOR-PHASE-ABORTED] rate in production — §5.2.4 already specifies the 5% / three-month observation window for production telemetry. §6 fixtures are design-time A/B evidence; they do not substitute for the §5.2.4 production monitor.§6 closes §8.1 L3 (the open question on final ship thresholds for H1–H3 A/B hypotheses, initial estimates +20pp / 15% / +2 findings tuned during fixture sizing). Final values frozen in §6.4: H1 hard ≥ +10pp (buffered from +20pp initial estimate, rationale §6.4), H2 soft ≤ 0 (relative percentage delta must not regress), H3 soft ≥ 0 (mean count delta must not regress). §8.1 L3 row is updated to Closed by §6 in this commit.
§7 specifies the lint / CI surface that enforces v3.6.6 contracts at PR-time. Most rules are cross-references to spec already settled in §3 (schema), §4 (templates), §5 (per-phase output lint), and §6 (A/B fixture manifest); §7 organises those references into the executing-script and CI-workflow surfaces and notes the additions vs the v3.6.2 baseline.
§7 covers what each lint script must enforce, what test fixtures must exist, and how the lint scripts hook into CI. §7 does not specify the executing Python class hierarchy, file naming for new lint scripts, or workflow YAML structure beyond the protocol-level wiring; those are implementation discretion at the implementation PR per §7.6.
scripts/check_sprint_contract.py already validates contracts against shared/sprint_contract.schema.json via jsonschema.Draft202012Validator plus a small set of Python-side structural invariants (uniqueness of dimension id / dimension name / condition_id per §4.1). The v3.6.6 division of labour is unchanged:
allOf branches per §3.5 table) are enforced by the schema itself when validators iterate errors. The validator does not re-implement these rules in Python — it relies on the schema's mode-conditional gates to fire on writer / evaluator / reviewer contracts. The implementation PR's only schema-related code change is loading the upgraded shared/sprint_contract.schema.json (Schema 13.1) instead of Schema 13; the validator's iteration call site is identical.acceptance_dimensions[].id, duplicate acceptance_dimensions[].name, duplicate failure_conditions[].condition_id. These remain mode-agnostic — the same uniqueness checks apply across reviewer / writer / evaluator contracts. v3.6.6 adds no new Python-side structural invariants beyond what Schema 13 already enforced; the new constraints are all schema-expressible.The validator's external interface (CLI arguments, exit codes, --ars-version flag) is byte-equivalent across v3.6.5 → v3.6.6, preserving the §3.6 zero-touch promise for reviewer-side users.
§8.2 R1 cross-mode field leakage decision (the spec amendment §8.2 R1 deferred to §7): v3.6.6 does not add an advisory warning in scripts/check_sprint_contract.py for writer / evaluator contracts that copy-paste reviewer-only fields. Reasoning: §4 shipped templates omit those fields by default; §4 authoring guidance restates the omission; the §3.5 require-conditional gates already produce a hard validation failure when a generator-mode contract is missing its required field (writer missing pre_commitment_artifacts; evaluator missing disagreement_handling), so the path most likely to bite an author is already covered. Adding an advisory warning for the leakage direction (carrying a reviewer-only field that the generator-mode validator does not require) would surface noise on every author who copy-pastes from reviewer/full.json as a starting point and then trims down — a common authoring path. The §8.2 R1 v3.7.x deferred hardening (not clause to forbid reviewer-only fields in generator modes) remains the long-term fix. §8.2 R1 row updates from "decision deferred to §7" to "no advisory warning in v3.6.6; §7.1 records the decision; v3.7.x not-clause hardening remains the long-term fix" once §7 ships.
Existing reviewer-only warnings must be mode-gated (an implementation requirement — not a deferral): the v3.6.5 validator emits SC-4 / SC-5 / similar warn_suspicious() advisory warnings whose semantics are reviewer-mode-specific (e.g., SC-5 fires when measurement_procedure.reviewer_must_output_before_paper does not contain both contract_paraphrase and scoring_plan — a reviewer-mode quality concern that does not apply to writer / evaluator contracts, which intentionally omit measurement_procedure per §3.3.1). Under Schema 13.1, writer / evaluator contracts pass schema validation cleanly, but the unchanged warning code would still fire reviewer-only warnings on every clean generator template, producing CLI noise on every implementation-PR validation run. The implementation PR must mode-gate every reviewer-mode-specific warning so it fires only when mode starts with reviewer_; warnings whose semantics are mode-agnostic (e.g., SC-4 dimension-reference warning) continue to fire universally. The implementation PR's specific mode-gating list — which existing warning codes are reviewer-only vs mode-agnostic — is a small audit on top of scripts/check_sprint_contract.py's warn_suspicious() body; the audit + gating both land in the same implementation PR commit that lands Schema 13.1.
Lint counts and per-mode rules are settled in §5.2.2 (Phase 4a / 6a) and §5.2.3 (Phase 4b / 6b). §7.2 re-states the count summary table for cross-reference and notes which lint surfaces are new vs unchanged from v3.6.2:
| Phase | Reviewer | Writer | Evaluator | v3.6.6 status |
|---|---|---|---|---|
| Phase 1 / 4a / 6a | 5 | 3 | 5 | Reviewer unchanged; writer + evaluator new |
| Phase 2 / 4b / 6b | 6 | 4 | 5 | Reviewer unchanged; writer + evaluator new |
The writer (3+4) and evaluator (5+5) lint surfaces are new in v3.6.6 and must be added to whichever lint code path the implementation PR chooses (the v3.6.2 baseline did not need to lint generator-mode phase outputs because no generator-mode contract gate existed). The reviewer (5+6) lint surface is byte-equivalent and zero-touch per §3.6.
Per §5.2.2 retry semantics, lint failure on Phase Na first attempt → retry once with the specific lint gap hinted in the system prompt; second failure → mark this role unusable. Abort behaviour differs by mode per §5.2.4 (writer / evaluator → [GENERATOR-PHASE-ABORTED]; reviewer → [PANEL-SHRUNK] panel cardinality), but the lint code path itself is symmetric: detect the structural violation, emit a [PROTOCOL-VIOLATION: phaseNa_lint_failed=true] tag, hand control back to the orchestrator. The mode-specific abort routing lives in academic-paper (writer / evaluator) or academic-paper-reviewer (reviewer), not in the lint code.
scripts/test_check_sprint_contract.py (the same module CI invokes via python3 -m unittest scripts.test_check_sprint_contract per the spec-consistency.yml workflow — see §7.4) gains writer / evaluator coverage extending the existing reviewer test surface. Two fixture families must land in the implementation PR:
shared/contracts/writer/full.json and shared/contracts/evaluator/full.json (already present on the spec branch as design-time artefacts per §4.0; the implementation PR ships them under live schema validation when Schema 13.1 lands). Each must validate cleanly under the upgraded validator.pre_commitment_artifacts (exercises §3.5 allOf branch 11).disagreement_handling (exercises §3.5 allOf branch 12).failure_conditions[].action to an editorial_decision=* value (exercises §3.5 allOf branch 5 — writer mode action enum mis-pinning to reviewer enum).failure_conditions[].action to a writer_decision=* value (exercises §3.5 allOf branch 6 — evaluator mode action enum mis-pinning to writer enum).failure_conditions[].action to a writer_decision=* or evaluator_decision=* value (exercises §3.5 allOf branch 4 — reviewer mode action enum mis-pinning to generator enum, completing the cross-mode triplet coverage).measurement_procedure / panel_size / cross_reviewer_quantifier fields is allowed to validate cleanly under v3.6.6 (the §3.5 require-conditional gates only require those fields for reviewer modes; they do not forbid them in generator modes). The v3.7.x not-clause hardening (§8.2 R1 deferred-hardening column) is the long-term fix and the test for it lives in v3.7.x, not v3.6.6.Reviewer-side fixtures (shared/contracts/reviewer/full.json and shared/contracts/reviewer/methodology_focus.json) remain byte-equivalent and re-run unchanged; their continued passing is the §3.6 zero-touch verification. §3.6 already commits to two specific reviewer regression tests landing in the implementation PR: test_existing_reviewer_contracts_still_valid_under_13_1 (loads both shipped reviewer templates against Schema 13.1 and asserts validation success without modification) and test_byte_equivalent_validation_for_reviewer_contracts (diffs the validation result tree against Schema 13's result for the same contract and asserts empty diff). §7.3 inherits those two test names from §3.6; the implementation PR ships them together with the schema upgrade.
Per-phase output lint test fixtures are out of scope for scripts/test_check_sprint_contract.py (which tests contract validation, not phase-output lint). The implementation PR may add a separate test module for per-phase output lint fixtures (writer Phase 4a / 4b / evaluator Phase 6a / 6b structural pass + structural fail cases) at the implementation PR's discretion; the test module naming and file structure are deferred per §7.6.
Three existing workflow files anchor v3.6.6's lint surface, with distinct roles:
.github/workflows/spec-consistency.yml — runs scripts/check_spec_consistency.py, runs python3 -m unittest scripts.test_check_sprint_contract, and validates each shipped contract template via scripts/check_sprint_contract.py in a per-template loop (currently iterates shared/contracts/reviewer/*.json only). This is the home for sprint-contract-related CI, not pytest.yml. The workflow trigger is unfiltered push / pull_request (no paths: key), so v3.6.6 does not need a trigger-key change. v3.6.6 extension: the implementation PR adds writer and evaluator validation loops alongside the existing reviewer loop (e.g., for f in shared/contracts/writer/*.json; do python3 scripts/check_sprint_contract.py "$f" --ars-version "v${ARS_VERSION}"; done, same for evaluator). The two new loops live in the same workflow step or as additional steps; the choice is implementation discretion per §7.6..github/workflows/pytest.yml — runs pytest scripts/adapters/tests/ -v for adapter-related tests (literature corpus adapters from v3.6.4); path-filtered to scripts/adapters/** and related adapter docs. v3.6.6 does not modify this workflow (sprint contract tests do not live here, despite the workflow being named pytest.yml)..github/workflows/freshness-check.yml — runs the PRISMA-trAIce snapshot freshness check from v3.4. v3.6.6 does not modify this workflow.v3.6.6 also adds:
- The implementation PR may extend scripts/check_spec_consistency.py itself to add §6 / §7 cross-reference invariants (e.g., the §5.2.2-§5.2.3 lint count summary table cell values must match the tables the implementation PR adds in academic-paper-reviewer/references/sprint_contract_protocol.md cross-reference + the new SKILL.md blocks per §5). The cross-reference invariant set itself is deferred to the implementation PR per §7.6.
- A new manifest CI lint job (per §7.5) is added — either as an additional step in spec-consistency.yml's existing job, in a new dedicated workflow file, or as a separate job within spec-consistency.yml. The choice is implementation discretion per §7.6. (Adding to pytest.yml is not an option: that workflow's role is adapter tests, not sprint-contract or A/B-fixture lint.)
The CI lint coverage that must be green at the spec PR merge: the existing Spec Consistency job (which runs spec consistency checks, the sprint contract test module, the reviewer template validation loop, the passport reset contract validation step from v3.6.3, and other compliance / data-access / version-consistency / collaboration-depth checks as steps within the single spec-consistency job), the freshness-check workflow (PRISMA-trAIce), and the pytest.yml adapter workflow. The implementation PR extends this coverage with the writer + evaluator validation loops (within the existing Spec Consistency job per the bullet above) and the new manifest CI lint coverage (placement per §7.6 — additional step inside spec-consistency.yml's job, additional job in the same workflow, or new dedicated workflow file). The aggregate CI surface gains lint coverage rather than a discrete fifth top-level job; the exact job topology is implementation discretion. Release-readiness sweep CI signals (CHANGELOG entry, version bump, documentation cross-references) are out of scope for §7 wiring per §7.6 — they are checked manually at implementation PR review or by a scripts/check_version_consistency.py-class extension whose extension is itself part of the release-readiness sweep deferral.
A new lint script enforces the §6.2 manifest schema and §6.5 git-tracked invariants on tests/fixtures/v3.6.6-ab/manifest.yaml. The script's naming and file structure are deferred per §7.6; the rules below specify what it must enforce.
Schema-shape checks (per §6.2 manifest schema definition):
fixture_version (string), manifest_lint_mode (string enum), documentation_paths (array), papers (array). Top-level summary_output (string) required iff manifest_lint_mode == implementation_pr (otherwise optional).fixture_version matches semver regex (e.g., ^\d+\.\d+\.\d+$).manifest_lint_mode is one of the enum values spec_branch | implementation_pr.documentation_paths[] is a non-empty array (must contain at least the spec-PR-shipped README.md per §6.2); each entry is a string (relative path).papers[] length is exactly 7.papers[] entry must carry, with declared types): paper_id (string slug), role (string enum paper-A | paper-C), paper_type (string), topic_label (string), input_artefacts (object with at least one populated sub-field from paper_configuration_record / paper_outline / argument_blueprint / annotated_bibliography / style_profile / knowledge_isolation_directive), baseline_output (object with required sub-fields writer_draft (string) + evaluator_review (string)). Paper-A entries must carry judge_output_baseline (string); paper-C entries must carry known_failure_mode (string) + failure_evidence (string).len([p for p in papers if p.role == "paper-A"]) == 6; len([p for p in papers if p.role == "paper-C"]) == 1. paper_id values across papers[] are unique.paper_type family labels each appear exactly twice (paper-C paper_type excluded from this count per §6.2).judge_output_baseline, judge_output_treatment, or metrics_output (per §6.2 paper-C exemptions). A paper-C entry carrying any of these fails lint.treatment_output shape when present: when populated (any paper, any mode), treatment_output must be an object with the four required sub-fields phase4a_output (string) + phase4b_output (string) + phase6a_output (string) + phase6b_output (string).Path-existence checks (mode-conditional + populated-optional, per §6.5 invariant 5 + §6.5 invariant 1):
manifest_lint_mode: spec_branch: every path declared in documentation_paths[], every path declared in any populated papers[].input_artefacts.* sub-field (at least one per paper required), every papers[].baseline_output.writer_draft + papers[].baseline_output.evaluator_review, every paper-A papers[].judge_output_baseline, and every paper-C papers[].failure_evidence must exist git-tracked at the commit under check.manifest_lint_mode: implementation_pr: all of the above plus every paper's papers[].treatment_output.{phase4a_output, phase4b_output, phase6a_output, phase6b_output}, every paper-A papers[].judge_output_treatment, every paper-A papers[].metrics_output, and the top-level summary_output must exist git-tracked.spec_branch mode, if the author has chosen to populate treatment_output.* / judge_output_treatment / metrics_output / top-level summary_output ahead of the implementation PR, those paths are not required by mode but are required to exist if declared. The lint script must check both required-paths-exist and declared-paths-exist as separate violation classes.Reverse-scan check (per §6.5 invariant 1):
tests/fixtures/v3.6.6-ab/ (excluding manifest.yaml itself) must be referenced from the manifest — either via a papers[].* sub-field path, via top-level summary_output, or via a documentation_paths[] entry. Any unreferenced file is a fixture-orphan and fails lint.Behaviour on malformed input: if manifest.yaml cannot be parsed as YAML (or the parsed value is not a top-level mapping), the lint script exits 1 with a parse-error message identifying the file. This mirrors the existing scripts/check_sprint_contract.py exit-1-on-file-error convention; the exact error message format is implementation discretion per §7.6.
The lint script's exit code is 0 on pass, 1 on any rule violation. Violation messages list the offending rule (schema-shape / required-path / optional-but-populated-path / reverse-scan / parse-error class) and the offending path or value. CI integrates the script per §7.4.
Mirroring §3.7 / §4.6 / §5.6 / §6.6 boundary statements, §7 explicitly defers the following to other sections / artefacts:
spec-consistency.yml or in a new dedicated workflow file; matrix / cache / parallelism choices; trigger filter additions if any are needed for performance reasons) → implementation PR.test_check_sprint_contract.py; fixture file naming convention) → implementation PR.spec-consistency.yml cross-reference invariant additions for §5 / §6 / §7 — listed at protocol level (lint count table cell-value matching across spec doc, sprint_contract_protocol.md, and SKILL.md blocks) → implementation PR.check_sprint_contract.py handles this for contracts; the new manifest lint script must mirror the same exit-1-on-parse-error convention but the exact error message format is implementation discretion.§7 does not close any open questions in §8.1 (L1 closed by §3, L2 closed by §5, L3 closed by §6, L4 closed by §3+§4 — all closures predate §7).
These were raised during brainstorm (Q1-Q7) and codex round-1 sanity check (D1-D4). They cannot be answered without the schema or protocol detail in §3-§6, so they are tracked here until those sections close them.
| # | Open question | Will be settled in |
|---|---|---|
| L1 | What is the exact pre_commitment_artifacts field shape (subfields, validation rules)? |
§3 (schema spec) |
| L2 | Does existing academic-paper orchestration support separate paper-blind and paper-visible agent calls, or does v3.6.6 need to add the orchestration glue? |
Closed by §5 (audit conclusion 1): no native two-phase split exists; v3.6.6 adds the orchestration glue inside academic-paper (SKILL plus the two agent files); §5.1–§5.5 specify the glue at protocol-level depth. |
| L3 | Final ship thresholds for H1-H3 A/B hypotheses (initial estimates: +20 pp / 15% / +2 findings). | Closed by §6: H1 hard ≥ +10 pp (buffered from +20 pp initial estimate after fixture-sizing variance accounting per §6.4), H2 soft ≤ 0% relative delta (must not regress), H3 soft ≥ 0 mean count delta (must not regress). Paper-C categorical signal added as separate sanity-check axis (§6.3 / §6.4); fail = P1 review surface, not auto-block ship. |
| L4 | Does writer Phase 4a output need both acceptance_criteria_paraphrase AND pre_commitment_artifacts, or are they the same artifact under different names? |
§3 + §4 (schema + contract template) — codex P3 #33 raised the parity question; needs concrete shape to answer |
| # | Risk | Mitigation in v3.6.6 | Deferred hardening |
|---|---|---|---|
| R1 | Cross-mode field leakage: Schema 13.1 require-conditional gates (measurement_procedure in §3.3.1, panel_size in §3.3.5, cross_reviewer_quantifier in §3.3.2) only require fields for reviewer modes; they do not forbid those fields from appearing in generator-mode contracts. A writer_full or evaluator_full contract that copy-pastes reviewer-only fields will pass schema validation but carry semantically empty data (e.g., cross_reviewer_quantifier: any with no panel to aggregate over). |
Shipped writer/full.json and evaluator/full.json templates omit reviewer-only fields by default; §4 authoring guidance restates the omission. §7.1 settled: no advisory warning in v3.6.6 (the §3.5 require-conditional gates already hard-fail the more common missing-field author path, and the leakage-direction warning would noise every author copy-pasting from reviewer template as a trimming start point). |
v3.7.x schema strictness pass: declare reviewer-only properties false for generator modes (or add a top-level not clause). Out of scope for v3.6.6 because the same posture is inherited from Schema 13's existing reviewer-conditional design — fixing it would expand the strictness surface beyond the generator-evaluator contract scope of this release. |
| R2 | Frozen-baseline / paper-specific data eviction: early pre_commitment_artifacts drafts nested three paper-specific subcomponents inside the frozen contract baseline — word_count_per_section (IMRaD numeric default), argument_plan.claim_evidence_chains (placeholder + lint), and structure_plan.section_outline (IMRaD section list). Each violated the v3.6.2 frozen-baseline invariant: every paper would either force the orchestrator to mutate the frozen contract before Phase 4a or hard-code IMRaD assumptions onto valid non-IMRaD paper types (literature reviews, theoretical, case study, policy brief — all supported by writer_full per draft_writer_agent's Paper Type Adjustments table). Codex review surfaced the three violations across separate iterations rather than as a single audit pass. |
All three subcomponents evicted to upstream runtime artefacts before v3.6.6 ship: word counts read from Paper Configuration Record (intake_agent); CER chains read from Argument Blueprint (argument_builder_agent); section outline read from Structure Architect output (structure_architect_agent). Writer dimensions D1 / D3 / D4 / D5 phrase their gates as "stays within / follows / per the upstream artefact" rather than committing concrete numbers, chains, or section names in the contract. pre_commitment_artifacts retains only acceptance_criteria_paraphrase — the single invariant policy that is paper-agnostic. |
No deferred hardening required; the eviction is the fix. The pattern is recorded as a forward-looking authoring rule for future contract templates: any contract sub-field whose value would differ paper-to-paper belongs in an upstream agent's runtime artefact, not in the frozen baseline. The v3.6.6 implementation PR's contract-authoring guidance (in the README "How to add a new template" section) is extended to state this invariant explicitly so the lesson does not need to be re-learned via codex review on the next mode family's templates. |
§9 records the chronological provenance of binding decisions in the v3.6.6 spec: brainstorm settlements (Round A through Round D below), codex review-driven amendments (catalogued in §9.2), and the ROADMAP §3.6.4 description correction (§9.3). Each entry cross-references the spec sub-section that ships the decision so future readers tracing "why does v3.6.6 do X" can follow the chain backwards. §9 introduces no new decisions; it inventories decisions already recorded in §1-§8.
Four brainstorm rounds shaped v3.6.6 between 2026-04-27 (initial scope clarification) and 2026-05-03 (§6 Section 1-6 design):
Round A — §1 scope clarification (Q1-Q7, user-driven, recorded in §2.1):
| # | Question | Decision | Lands in |
|---|---|---|---|
| Q1 | v3.6.6 scope: full mode only vs include revision + pipeline | Narrow: academic-paper/full only |
§1 in-scope items 1, 6, 7; §1 out-of-scope items 1, 2 |
| Q2 | Two-reviewer overlap (internal Phase 6 evaluator vs external academic-paper-reviewer skill) |
Document as known tech debt; do not merge | §1 known limitations; §5.1 audit conclusion 2 |
| Q3 | Model selection mechanism for evaluator | Mirror existing ARS_CROSS_MODEL env pattern unchanged |
§1 in-scope item 9; §1 design goal 4 |
| Q4 | A/B evidence shape for open-ended task with no ground-truth quality score | Process-metric only, delta-based, three pre-declared directional hypotheses | §1 in-scope item 11; §6 entire section |
| Q5 | A/B fixture source: real retracted vs AI-generated synthetic vs human-written | Originally AI-generated synthetic + defect injection + expected_findings.yaml; evolved during Round D §6.2 brainstorm to re-using existing suite outputs (six paper-A from deep-research synthesis reports + one paper-C from v3.6.5 session log) |
§2.1 Q5 row records evolution; §6.2 ships final composition |
| Q6 | Schema strategy: 13.1 backward-compat add vs Schema 14 vs base+extension refactor | Schema 13.1 backward-compatible add | §3 entire section; §3.6 backward-compat proof |
| Q7 | Correct ROADMAP §3.6.4 outdated text or only describe what v3.6.6 does | Correct ROADMAP + decision-log entry | §1 in-scope item 12; §9.3 below |
Round B — §1 codex sanity check (D1-D4, codex-driven, recorded in §2.2):
| # | Codex finding | Decision | Lands in |
|---|---|---|---|
| D1 | Schema 13 has reviewer-required fields that bare mode enum addition would break |
Apply allOf / if mode startsWith reviewer_ conditional pattern to all reviewer-specific fields |
§3.3 (four reviewer-conditional gates); §3.5 allOf branch table |
| D2 | disagreement_handling proposed for both writer and evaluator, but solo writer has no peer to disagree with |
Split into pre_commitment_artifacts (writer) + disagreement_handling (evaluator); two new fields |
§3.4.1 (writer); §3.4.2 (evaluator); §3.5 allOf branches 11+12 |
| D3 | Original "cross-model by default" framing in Design Goal 4 contradicted Q3-decided actual behaviour | Reword to "Cross-model compatible via existing env pattern" | §1 design goal 4 |
| D4 | agreement_audit proposed as schema field but it is observed-during-run data, violating v3.6.2's "contract = pre-registered acceptance criteria" principle |
Remove agreement_audit from schema entirely; keep protocol doc + runtime example separately |
§1 in-scope item 1, item 10; §3 explicit non-addition note |
Round C — §5 two-phase orchestration brainstorm (Q1-Q10 user-driven brainstorm during §5 brief preflight, 2026-04-28):
| # | Question | Decision | Lands in |
|---|---|---|---|
| Q1 | Phase split mechanism (where the new instructions live) | B — academic-paper/SKILL.md adds ## v3.6.6 Generator-Evaluator Contract Protocol block, verbatim mirror of v3.6.2 reviewer two-phase pattern |
§5.2.1 phase boundaries; §5 implementation-PR scope at §5.6 |
| Q2 | Delimiter naming convention | B — phase-numbered <phase4a_output> / <phase6a_output>; reviewer <phase1_output> unchanged |
§5.3 delimiter design |
| Q3 | Phase 4a/6a lint count | A — verbatim mirror of reviewer Phase 1 lint; writer 3 / evaluator 5 / reviewer 5 checks | §5.2.2 lint count summary |
| Q4 | Phase 4b/6b lint + multi-dissent | A — drop multi-dissent retry from generator modes; drop writer consistency check; writer 4 / evaluator 5 / reviewer 6 checks | §5.2.3 lint count summary |
| Q5 | Single-agent generator unusable handling | C — [GENERATOR-PHASE-ABORTED] abort + 5%-per-academic-paper-full-run threshold + v3.6.7 graceful degradation forward note |
§5.2.4 |
| Q6 | Cross-phase artefact handoff | A (revised from r1 C) — mid-round cross-session resume unsupported; in-conversation <phase4a_output> delimiter only; Schema 9 zero change; v3.6.7+ pre_commitment_history[] forward note |
§5.4; §8.1 L2 closure |
| Q7 | Backward compatibility scope | B+D — default ON for academic-paper full mode + pipeline boundary unchanged + Phase 6 evaluator (in-pair self-quality gate) vs Stage 3 reviewer (external editorial) two-layer distinction |
§5.5 (full ship); §5.1 audit conclusion 2 |
| Q8 | Pipeline Stage 2 vs Phase 6 boundary (raised by §5 brief r1 P1 #3) | D — pipeline boundary unchanged; Phase 6 evaluator vs Stage 3 reviewer two-layer review documented in §5.1 audit findings | §5.1 audit conclusion 2 |
| Q9 | Mid-round session split (raised by §5 brief r1 P1 #2) | A — explicitly declared unsupported; Schema 9 zero change | §5.4 |
| Q10 | Generator-mode multi-dissent (raised by §5 brief r1 P1 #4) | A — drop entirely; lint count converges to writer 3+4 / evaluator 5+5 / reviewer 5+6 | §5.2.2 + §5.2.3 |
Q8/Q9/Q10 are revisions surfaced by the §5 brief preflight codex r1 (per §9.2 §5 brief 22-finding cascade); the original §5 brief draft did not enumerate them. They were added to Round C after r1 surfaced the gap, then ratified user-side and shipped through §5 spec body.
Round D — §6 A/B evidence design (Q1-Q10 user-driven brainstorm + Section 1-6 user-approved structure, 2026-04-29 + 2026-05-03):
| # | Question | Decision | Lands in |
|---|---|---|---|
| Q1 | A/B comparison frame | Baseline = v3.6.5 single-call; treatment = v3.6.6 contract-gated 4a/4b/6a/6b | §6.1 |
| Q2 | What H1-H3 measure | H1 acceptance dimension pass rate / H2 Tier 0 unverified citation rate (renamed during §6 r2 from "hallucinated") / H3 deduped self-detected issues per paper | §6.3 |
| Q3 | Fixture size | 6-paper toy fixture (later expanded to 7 with paper-C in Q4) | §6.2 |
| Q4 | Paper source | 6 paper-A same-input repro + 1 paper-C v3.6.5 known-fail sanity case | §6.2 |
| Q5 | Paper-C sanity check type | v3.6.5 self-scoring passed but Stage 3 caught hallucinated citation | §6.2 paper-C; §6.3 paper-C categorical signal |
| Q6 | Threshold semantics | Tiered: H1 hard (initial +20pp, tuned to +10pp during Section 4); H2/H3 soft must-not-regress | §6.4 |
| Q7 | Metric computation | Codex gpt-5.5 + xhigh verdict-only judge for H1/H3; Semantic Scholar API Tier 0 for H2 | §6.3 |
| Q8 | Baseline timing | Spec branch ships v3.6.5 baseline output | §6.5 spec-PR ship list |
| Q9 | Fixture spec granularity | Outline + manifest schema (§6 spec defines protocol; tests/fixtures/v3.6.6-ab/manifest.yaml instantiates) |
§6.2 manifest schema; §6.5 invariants |
| Q10 | Codex judge output handling | Commit codex judge full output into fixture (12 paper-A files for both baseline + treatment) | §6.5 spec-PR + impl-PR ship lists |
§6 Section 1-6 user-approved structure additionally locked: comparison frame (§6.1) / fixture composition + manifest (§6.2) / H1-H3 hypotheses + metric computation (§6.3) / ship gate threshold + buffer rationale (§6.4) / reproducibility artefacts (§6.5) / boundary statement (§6.6).
Four spec sub-sections (§3, §5, §6, §7) ran multi-round codex iterative reviews per feedback_codex_iterative_spec_review_to_zero.md. Cumulative trajectory:
| Section | Rounds | Findings per round | Total |
|---|---|---|---|
| §3 schema | 3 | r1 5 / r2 1 / r3 0 | 6 |
| §5 brief preflight | 6 | r1 10 / r2 4 / r3 4 / r4 1 / r5 2 / r6 1 | 22 |
| §5 spec body | 3 | r1 5 / r2 3 / r3 0 | 8 |
| §6 spec body | 3 | r1 5 / r2 7 / r3 0 | 12 |
| §7 spec body | 4 | r1 5 / r2 2 / r3 1 / r4 0 | 8 |
| Cumulative | 19 rounds across 5 review streams | — | 56 |
Recurring lessons (recorded in the maintainer's private memory during the spec sprint):
feedback_brief_itself_needs_codex_review.md).feedback_architectural_inflection_after_repeated_p1.md).feedback_complex_spec_review_inventory_pattern.md).manifest_lint_mode enum + mode-conditional optional fields. r2 surfaced 7 cascade findings (orphan / asymmetry / vocab) that all flowed from the r1 schema shape. r3 hit 0 — the cascade discharged in one fix round (per feedback_cross_model_review_cascade_inconsistency.md).draft_writer_agent.md:379 to verify the Paper Type Adjustments table contents and caught that author's "IMRaD empirical / lit-review / case study" claim did not match the table (Theoretical / Case study / Policy brief / Chinese paper). §6 P2 #7 codex grepped semantic_scholar_api_protocol.md:75 to catch that "hallucinated citation" overclaimed Tier 0 semantics. §7 r1 P2 #1 grepped pytest.yml to catch the workflow attribution error (sprint contract test lives in spec-consistency.yml). §7 r2 P2 #2 grepped check_sprint_contract.py:138 to catch existing reviewer-only warn_suspicious() SC-* warnings that would noise on clean generator templates. Reading authoritative protocol documents before claiming a fact about them is a discipline the author repeatedly skipped and codex repeatedly caught.git diff -G between settle-decision sub-sections and dependent test/fixture sub-sections in the same commit to catch this class of contradiction (added to lessons in project_ars_v3_6_6_spec_wip.md § §7 session lessons).grep the entire spec doc for any remaining occurrences of the old wording before commit.ROADMAP entry §3.6.4 originally described v3.6.6 as "split single agent into writer + evaluator". This description was incorrect for the v3.6.5 baseline state: draft_writer_agent and peer_reviewer_agent were already two separate agents inside academic-paper, not one agent to be split. The actual v3.6.6 work is to extend the v3.6.2 sprint contract pattern (originally introduced for the academic-paper-reviewer skill's reviewer panels) to the existing in-pair writer/evaluator pair inside academic-paper, adding contract-gated phase splits (Phase 4 → Phase 4a + Phase 4b; Phase 6 → Phase 6a + Phase 6b) governed by Schema 13.1 conditional gates.
The implementation PR's ROADMAP edit replaces the §3.6.4 description with: "Extend v3.6.2 sprint contract pattern to the existing academic-paper writer/evaluator pair via contract-gated phase splits and Schema 13.1 conditional gates. No new agent files; existing draft_writer_agent and peer_reviewer_agent gain per-phase sub-section instructions." This is in-scope item 12 (§1 line 60) and lives in the implementation PR per §3.7 / §4.6 / §5.6 / §6.6 / §7.6 release-readiness sweep deferral.
The corrective entry exists because Q7 (§9.1 Round A) chose truthfulness over change-minimisation: ROADMAP is source of truth for future contributors and stale text would mislead anyone reading §3.6.4 before reaching this v3.6.6 design doc. Recording the correction here closes the audit loop.