Date: 2026-05-18
Issue: #133 — Phase 2+ agents can silently inflate scope and skip independent Phase 3/5 checkpoints
Severity: HIGH (silent integrity failure; recurrence-risk pattern)
Target release: v3.9.2
Design phase: v3 review complete (codex review --base main broken-corner + inline opus subagent). Opus surfaced 2 HIGH + 5 MED + 3 LOW findings. v4 (this doc) addresses both HIGH; MED/LOW deferred to impl phase per user decision.
Related: #134 (v3.10 active conductor architecture — long-term fix; hook + multi-phase schema + provenance now live there)
Doc revision history:
- v1: initial — single-track self-review. 9 issues identified.
- v2: incorporated codex r2 + inline opus + self triple-track review. Resolved 3 conflicts; v2 committed to hook + verifier in v3.9.2.
- v3: incorporated v2 dual-track review (codex r2 retry + inline opus). 14 findings showed scalar ars_phase schema is wrong (multi-phase agents exist), L1 routes to orchestrator that can't handle it, hook lookup mechanism is hand-waved. Scope narrowed: hot-fix ships only L1 routing + Fix 1 prompt fence + advisory verifier. Hook (Fix 2), multi-phase schema, deterministic provenance all moved to v3.10 conductor.
- v4 (this doc): incorporated v3 review (codex broken corner + inline opus). HIGH-1 (3-bucket model insufficient → 4-bucket A/B/C/D with out-of-scope inflation column) + HIGH-2 (multi-phase lite reminder is placebo → cut entirely, honest framing) addressed. MED/LOW carry to impl phase. Effort revised 18-22h → 18-23h with Phase 0.2 bumped to 2-3h.
User provided pre-existing abstract (Phase 4 artifact) + pre-collected literature (Phase 2 artifact) with no explicit invocation. ARS routing logic auto-dispatched a single phase agent without engaging pipeline_orchestrator_agent. This is a routing discipline failure: ambiguous input + LLM helpfulness = shortcut to single-phase dispatch.
Once dispatched without orchestrator oversight, the phase agent treated visible end-goal + write permissions as license to "be helpful" and produced Phase 3/4/5/6 output autonomously. Mandatory independent crosschecks (DA2 / DA3 / editor-in-chief / ethics_review) were silently skipped.
v3.10 will replace passive routing + autonomous subagent with active conductor + structured task envelopes. v3.9.2 is the hot-fix that buys time for v3.10 to be designed correctly.
v2 design committed to deterministic hook + verifier in v3.9.2. Round-2 dual-track review (codex retry + inline opus) surfaced critical findings that revealed the deterministic gate cannot be built correctly in hot-fix scope. Key blockers:
ars_phase schema is wrongMulti-phase agents exist in actual ARS:
- devils_advocate_agent runs at Phases 1, 3, 5 + Socratic Layer 2, 4 (per deep-research/SKILL.md agent table)
- report_compiler_agent runs at Phases 4, 6
- socratic_mentor_agent runs at Socratic layers, not numbered phase
- compliance_agent runs at any phase (Stage 2.5 / 4.5 integrity gates, cross-skill)
- academic-paper has sub-phases (5a/5b, generator-evaluator contract)
Single scalar ars_phase: <N> cannot represent any of these. v2's {1..6, orchestrator, meta} enum is also insufficient (e.g., what's DA's value? 1? 1,3,5? meta?).
Correct schema is multi-phase (ars_phase_writes: [list], ars_phase_reads: [list]) or pattern-based (ars_phase: "1|3|5"). Designing that properly + auditing all 38 agents to assign correct values is multi-day work, not hot-fix scope.
phase{N-1}_*/ reads breaks legitimate handoffsv2 Phase 1 prompt fence said:
You MUST NOT: Read or write files in
phase{N+1}_*/,phase{N-1}_*/, ...
Codex P1 finding: Phase 3 synthesis must read Phase 2 bibliography; Phase 4 drafting must read Phase 3 synthesis; etc. The "no regress" rule was over-reach. v3 corrects: only writes are scope-bound, reads from upstream phases are always allowed.
Current pipeline_orchestrator_agent.md keyword-matches user phrasing (Stage 1 vs Stage 2 etc.). It does NOT have logic for "user dropped cross-phase artifacts, no passport, no explicit invocation — figure out where in the pipeline to resume from." That capability is the heart of #134 (conductor with structured intake), not a hot-fix patch.
v2's L1 strategy (route ambiguous input to orchestrator) was thus broken — destination was non-functional. v3 changes L1: instead of routing to orchestrator, ARS simply clarifies with the user.
agent_type → ars_phase lookup mechanism that doesn't exist (where it lives, regeneration, stale-safe).Both are infrastructure work that v3.10 conductor will design correctly with task envelopes. Trying to bolt them on in hot-fix produces fragile half-solutions.
Hot-fix scope: - L1: routing clarification gate (not orchestrator detour — destination is broken) - Fix 1: prompt hard fence per agent (writes only, reads allowed) — for single-phase agents only in v3.9.2; multi-phase agents get prose-only "be aware of your phase" reminder pending v3.10 schema - Fix 3: SKILL.md phase-by-phase invocation contract - Fix 4 lite: advisory-only verifier (heuristic, no provenance guarantee)
Cut from v3.9.2:
- ~~Fix 2 PreToolUse hook~~ → v3.10 (needs multi-phase schema + lookup mechanism + Phase 0.1's agent_type finding is preserved for v3.10)
- ~~Multi-phase ars_phase schema~~ → v3.10
- ~~Orchestrator cross-phase intake capability~~ → v3.10 (this is conductor's core feature anyway)
- ~~Deterministic verifier with provenance~~ → v3.10
What remains is a prompt-discipline-plus-routing-clarification hot-fix. It won't fully prevent #133 (capable models can still rationalize past prompts), but it raises the bar significantly + buys time for v3.10.
agent_type field equaling agent frontmatter name. Not used in v3.9.2 (hook deferred to v3.10). Recorded for v3.10 conductor design.bibliography_agent = P2 only, research_question_agent = P1 only) → gets v3.9.2 hard boundary promptdevils_advocate_agent = P1+P3+P5, report_compiler_agent = P4+P6) → NO boundary added in v3.9.2 (per HIGH-2 honest framing — placebo prose would create illusion of enforcement). Recurrence acknowledged; v3.10 envelope is the fix.socratic_mentor_agent operates on Socratic Layers 1-5 not numbered pipeline phases; compliance_agent runs at Stage 2.5 / 4.5 integrity gates cross-skill) → no boundary added. Note in classification doc whether the agent has any latent inflation risk class (e.g., socratic_mentor inflating into direct conclusions); v3.9.2 explicitly does not cover those classes.pipeline_orchestrator_agent, field_analyst_agent, intake_agent) → no boundary added (legitimate cross-phase by design role)Audit-flag (NOT a bucket): draft_writer_agent, literature_strategist_agent already carry partial boundary language. Assigned to whichever real bucket they belong (likely both Bucket A — Phase 4 and Phase 2-equivalent respectively); audit existing language for v3.9.2 consistency. Do NOT classify these as a separate bucket.
Output table columns (per opus HIGH-1):
| Agent | Bucket | Phase(s) declared in | v3.9.2 fence coverage | Out-of-scope inflation risk | Source citation |
|---|---|---|---|---|---|
| bibliography_agent | A | Phase 2 | hard-fence | (none specific to v3.9.2) | deep-research/SKILL.md row 3 |
| devils_advocate_agent | B | Phase 1, 3, 5, Socratic L2/L4 | none-unmitigated | scope inflation across invocations; awaits v3.10 envelope | deep-research/SKILL.md row 8 |
| socratic_mentor_agent | C | Socratic Layers 1-5 (phase-orthogonal) | not-applicable | latent: direct-conclusion inflation (different class from #133, not covered) | deep-research/SKILL.md row 10 |
| compliance_agent | C | Stage 2.5 / 4.5 cross-skill | not-applicable | latent: high-write-authority misclassification (low probability) | shared/agents/compliance_agent.md |
| ... | ... | ... | ... | ... | ... |
Output: docs/design/2026-05-18-ars-v3.9.2-agent-phase-classification.md (38-row table with the columns above).
Estimate revision: 2-3h (not 1-1.5h per opus MED-5 — 38 agents × careful cross-check against SKILL.md tables + frontmatter + actual usage is non-trivial).
Goal: When user provides ambiguous cross-phase input, ARS must clarify before dispatching ANY agent. v2's "route to orchestrator" approach dropped because orchestrator can't handle this case (deferred to v3.10).
.claude/CLAUDE.md (single source of truth)Insert before existing "Routing Rules":
## Routing Discipline (v3.9.2)
**Routing precedence:** This section runs BEFORE Routing Rules 1-5. Once this section settles on a destination, Rules 1-5 apply within that destination's skill family.
Before invoking any ARS skill or agent, classify the user's input:
1. **Explicit clear intent** (user names a specific skill via `/ars-*` command or unambiguous trigger keyword like "lit-review this", "review my paper"):
→ Route directly; no clarification.
2. **Cross-phase materials detected** (user provides artifacts spanning ≥ 2 pipeline phases — e.g., pre-written abstract + pre-collected literature):
→ **Clarify**. Do NOT auto-route to a single-phase agent. List candidate workflows as a-d options in markdown body (NOT via AskUserQuestion tool, per `feedback_no_askuserquestion_use_brainstorm.md`).
→ Reason: clarification is the safest action when materials don't unambiguously identify intent. (v3.10 conductor will handle this via structured intake; for now, ask.)
3. **No materials, ambiguous request:**
→ Clarify per `shared/references/intent_clarification_protocol.md`.
**Escape hatch:** Prefix `[direct-mode]` at start of first message → all clarification bypassed; ARS dispatches whatever single agent matches the literal trigger.
**Anti-pattern (caused #133):** Receiving ambiguous cross-phase materials and silently auto-routing to a single-phase agent based on which phase the materials "look closest to."
**Forward note (v3.10):** Active conductor will reframe this gate as structured intake with task envelope dispatch. v3.9.2 ships clarification-only as interim.
shared/references/intent_clarification_protocol.md (NEW file)Carries:
- Trigger condition table (precise rules for the 3 routing classes)
- Clarification message template (a-d options format, multi-select discipline per feedback_telegram_conversation_style.md)
- [direct-mode] escape hatch behavior spec: must be byte-0 of first message (case-insensitive); literal [direct-mode] or [Direct-Mode] accepted; consumed by main session before routing, stripped from passed-through message
- 3-5 worked examples of inputs → correct routing outcome
- Forward note referencing #134 / v3.10 active conductor
> **Routing discipline:** see `.claude/CLAUDE.md` "Routing Discipline (v3.9.2)" + `shared/references/intent_clarification_protocol.md`. This skill assumes routing has already settled.
No rule duplication; source-of-truth lives in CLAUDE.md + protocol doc.
8 fixture cases covering:
- Cross-phase materials (abstract + lit) → asserts clarification triggers (NOT silent dispatch)
- Single-phase material (literature only) → asserts ARS proceeds with Phase 2 work (no clarification needed for single-phase)
- No materials + ambiguous request → asserts clarification with a-d options
- Explicit /ars-lit-review invocation → asserts skip clarification
- [direct-mode] run bibliography_agent → asserts skip clarification
- [direct-mode] mid-message (not byte-0) → asserts clarification still fires (escape hatch not honored)
- [Direct-Mode] case variant → asserts escape hatch honored (case-insensitive)
- Full draft + abstract + lit → assertion: clarification fires (cross-phase, no clear skill named)
Acceptance criterion: 100% pass on Opus 4.7; ≥ 75% pass on Sonnet 4.6 + GPT-5.5 (degradation flagged but non-blocking ship). Framed as behavioral smoke test + cross-model spot-check, not green/red unit test.
Scope (per Phase 0.2 4-bucket classification, revised per v3 review):
- Bucket A — Single-phase agents (~25 of 38, finalized in Phase 0.2): full hard boundary block as below
- Bucket B — Multi-phase agents (~8 of 38): NO boundary added (placebo prose creates false-enforcement illusion; HIGH-2 honest framing)
- Bucket C — Phase-orthogonal agents (~3 of 38): no boundary added; classification doc notes any latent inflation risk
- Bucket D — Cross-phase / meta agents (~2 of 38): no boundary added (legitimate cross-phase role)
- Audit-flag (within Bucket A): draft_writer_agent, literature_strategist_agent carry existing partial language — harmonize with v3.9.2 hard fence shape
Net effect: only Bucket A agents receive new prompt fence. Bucket B-D are documentation-only changes (added to classification doc, no agent .md modification).
Insert after persona, before role description:
## Phase Boundary (v3.9.2)
You are a single-phase agent assigned to Phase N (<phase name>). Your sole deliverable is <X>.
You MUST NOT:
- WRITE files in `phase{M}_*/` directories where M ≠ N (no inflate)
- Produce content classified as a downstream-phase deliverable type (synthesis / draft / review / revision / etc.) even if you can see the end-goal
- Invoke or simulate any other agent persona's output
- "Helpfully" continue past your assigned deliverable
You MAY READ files in `phase{1}_*/` through `phase{N}_*/` (upstream + own phase) for legitimate context. Reads from downstream phases (`phase{N+1}_*/`+) are not needed for your work.
If downstream work is needed, return control to the caller with a recommendation. Do not execute.
**Enforcement (v3.9.2):** prompt-level only. Advisory verifier (`scripts/check_pipeline_integrity.py`) can detect violations post-hoc. Deterministic PreToolUse hook deferred to v3.10 active conductor (#134).
v3 dual-track review (inline opus HIGH-2) surfaced that any prose-level "lite reminder" for multi-phase agents is essentially placebo — #133's caller prompt already contained tight phase-scope language ("Phase 2 only, no fabrication, return under 300 words") and the agent still inflated. Adding a similar reminder at the agent persona layer delegates enforcement back to the same prompt mechanism that already failed.
Decision: multi-phase agents (devils_advocate, report_compiler, etc., per Phase 0.2 classification) receive NO additional prompt fence in v3.9.2. Their existing role descriptions stand. The multi-phase inflation class is explicitly unmitigated in v3.9.2; deterministic enforcement via task envelope is the v3.10 conductor (#134) fix.
This is documented in:
- The Phase 0.2 classification table (per-agent column: "v3.9.2 fence coverage" with values hard-fence | none-unmitigated | not-applicable)
- §5 Failure modes — explicit "multi-phase inflation remains possible; recurrence expected for ~N agents until v3.10"
- README migration note — if user sees DA / RC / similar multi-phase agent producing out-of-scope content, the remediation is "report to issue #134, do not retry, switch to Mode A orchestrator-driven invocation"
Rationale for not adding placebo prose: users (and codex review) reading the multi-phase agent .md should not see boundary language that creates an illusion of enforcement. Either the fence works (single-phase case) or there's no fence (multi-phase case, explicitly acknowledged). This is more honest than splitting hairs about "lite reminder" semantics.
ars_phase field in v3.9.2v2's scalar ars_phase is intentionally NOT added in v3.9.2. Multi-phase schema is v3.10 work. Adding scalar now and changing to multi-list later creates schema migration friction.
Add to all 4 SKILL.md files:
## Phase-by-phase Invocation Contract (v3.9.2)
ARS pipeline runs in 6 phases. Two invocation modes:
**Mode A — orchestrator-driven (default):** `pipeline_orchestrator_agent` runs all phases end-to-end with state tracking.
**Mode B — phase-by-phase (Material Passport resume):** User invokes one agent per phase across sessions (long-running projects).
In Mode B, **each single-phase agent stays strictly within its assigned phase for writes**. Reads from upstream phases are allowed. Multi-phase agents (e.g., `devils_advocate_agent`) do exactly the work specified by the caller's invocation prompt, no more.
Routing into Mode B requires explicit user signal — `/ars-<mode>` slash command or `[direct-mode]` prefix. Ambiguous cross-phase input triggers clarification (per `.claude/CLAUDE.md` Routing Discipline).
**Enforcement:** prompt-level + advisory verifier in v3.9.2; deterministic gate in v3.10 active conductor (#134).
File: scripts/check_pipeline_integrity.py (joins existing 28 check_*.py scripts).
Logic (v3.9.2 lite — no provenance required):
- Scan working directory for phase{1-6}_*/ dirs
- For each directory pair (phaseN_, phaseM_) where N+1 < M (skip-ahead detected):
- Flag (INTEGRITY-ADVISORY) if files in both directories share creation timestamp within 5-minute window (suggests single-dispatch generation)
- For Phase 5 specifically:
- Flag (INTEGRITY-ADVISORY) if phase5_/ exists but no separate review reports from DA / EIC / Ethics (heuristic: filenames not matching *devils*, *editor*, *ethics*)
- Output:* human-readable advisory list, NOT machine-enforceable. User reviews + decides.
Acceptance criterion: verifier flags the #133 reported failure case (bibliography_agent producing phase3_/phase4_/phase5_/phase6_/ in single dispatch — they'll all have timestamps within ~minutes of each other).
Limitations explicit (v3.9.2): - NO author provenance check (deferred to v3.10 conductor + envelope schema) - 5-minute window is heuristic, not deterministic - Multi-phase agents producing legitimate work in 2+ phases will trigger false-positive (acknowledged limitation) - Advisory-only — does not block, only surfaces signals for user review
Order corrected from v2: lint compat audit runs before prompt fence sweep, not after. Reason: if existing 28 check_*.py scripts have strict frontmatter validation, adding boundary block headings (or worse, frontmatter fields) could break CI mid-sweep.
Audit method:
- Grep all 28 scripts/check_*.py for frontmatter, yaml, parse_yaml, agent .md path patterns
- Identify which scripts read agent frontmatter and what fields they expect
- Run each on a single test agent .md with proposed v3.9.2 prompt fence body (no new frontmatter field — just new section in body) — confirm no break
- Document findings in Phase 0.2 classification doc
Expected outcome: no breaks (we're adding markdown sections, not frontmatter fields). But verify before sweep.
Test additions: - L1.4 behavioral smoke tests (8 fixtures, cross-model spot-check) - Phase 4 verifier tests (positive: #133 case detected; negative: legitimate multi-day pipeline run not flagged) - Lint compat verified (Phase 5)
Review:
- codex review --base main (success corner config: small diff, free-form output)
- Inline opus subagent independent review (parallel)
- Skip Gemini per memory sample 12 (broken on ARS context)
Expected rounds: 2-3 (v3 already incorporates 2 rounds of dual-track findings; remaining iterations cover implementation-level issues codex catches in the actual code).
feedback_public_flip_readiness_checklist.md 12-item audit (ARS is public repo)feedback_ars_public_repo_boundary.md — grep for hei-platform / Springer / HEEACT leakage in new filesdocs/design/2026-05-18-ars-v3.9.2-phase-boundary-spec.md (final)[Unreleased]feedback_* memory updates as warrantedThe original v1 design doc carried a 14-row inventory of silent drift candidates beyond #133. Per v2 opus subagent review, that material is roadmap-class, not hot-fix material, and is in #134 issue body.
[direct-mode] prefix edge cases (per opus M1): specified in protocol doc — byte-0, case-insensitive, stripped before pass-through.codex review --base main with small diff (success corner); skip xhigh codex exec; inline opus subagent as parallel insurance.Once Phases 0-8 land, produce proof:
tests/fixtures/issue_133/.check_*.py pass on agent files with new boundary block.| Phase | Description | Hours |
|---|---|---|
| 0 | Pre-flight + 38-agent 4-bucket phase classification (per opus HIGH-1) | 2-3 |
| 5 | Lint compat audit (MOVED before Phase 1) | 1.5-2 |
| L1 | Routing rule + protocol doc + 4 SKILL.md backpointer + 8 smoke tests | 3-4 |
| 1 | Prompt fence × ~25 Bucket A agents only (Bucket B/C/D no agent .md changes) | 3-4 |
| 3 | SKILL.md phase contract × 4 | 1 |
| 4 | Advisory verifier (lite, no provenance) | 2 |
| 6 | TDD red tests + cross-model review (2-3 rounds) | 3-4 |
| 7 | Security + PII + public-flip audit | 1 |
| 8 | Spec doc + CHANGELOG + README migration | 1.5 |
Total v3.9.2: 18-23h, ~3-4 working days.
(v2 estimate was 22-32h with hook + provenance; v3 cut both. v4 adjustments — Phase 0.2 bump for 4-bucket audit + Phase 1 shrink because only Bucket A gets fence — net to 18-23h.)
Items deferred from v3.9.2 to v3.10 conductor:
| Item | Reason for deferral |
|---|---|
| PreToolUse hook (Fix 2) | Requires multi-phase schema + lookup mechanism; v3.10 conductor's task envelope is the proper substrate |
Multi-phase ars_phase_writes + ars_phase_reads schema |
Cannot be retrofitted scalar → list cleanly; v3.10 will design correctly with envelope |
| Deterministic verifier with author provenance | Requires tamper-resistant author record on phase files; v3.10 envelope provides natively |
| Orchestrator cross-phase intake capability | Heart of v3.10 conductor architecture — passive routing → active intent classification |
agent_type → ars_phase lookup mechanism |
Will be obsoleted by envelope-carried phase info |
| Phase-5 reviewer cross-phase READ exception | Becomes envelope-declared read permissions |
Phase 0.1 finding (CC PreToolUse payload has agent_type field equaling agent frontmatter name) preserved as design input for v3.10 conductor hook implementation.
| Decision | v3 Choice | Source |
|---|---|---|
| Hook (Fix 2) | DEFERRED to v3.10 | v2 dual-track review — Codex P1 (regress read break) + Opus H1 (multi-phase schema) + Opus H2 (lookup hand-wave) |
Multi-phase ars_phase schema |
DEFERRED to v3.10 | Codex P1 (multi-phase agents exist) + Opus H1 |
| L1 routing default | Clarify, not orchestrator detour | Opus H3 — orchestrator doesn't handle this case yet |
| Deterministic verifier with provenance | DEFERRED to v3.10; v3.9.2 ships advisory-only | Codex P2 — provenance infrastructure missing |
| Prompt fence READ scope | Allow upstream reads, only block writes | Codex P1 — v2 over-reached |
| Single-phase vs multi-phase agent treatment | Different prompts (hard fence vs lite reminder) | Codex P1 (multi-phase exist) |
| Effort estimate | 18-22h (down from v2's 22-32h) | Scope narrowed |
| Lint compat audit order | Before Phase 1 sweep | Opus H5 |
[direct-mode] prefix spec |
Byte-0, case-insensitive, stripped before routing | Opus M1 |
| Q1 platform scope | Option B (preserved from v1/v2) | unchanged |
| Inventory (14 silent drift) | Lives in #134 | v2 Opus |
| Codex 0.130 routing | codex review --base main only; skip xhigh codex exec |
Memory sample 12 |
| v4: 4-bucket classification model | A single-phase / B multi-phase / C phase-orthogonal / D cross-phase-meta; "already-has-language" is audit-flag within Bucket A, not separate bucket | v3 review opus HIGH-1 |
| v4: Multi-phase agents get NO fence in v3.9.2 | Honest framing — placebo prose would create false-enforcement illusion. ~8 Bucket B agents explicitly unmitigated; v3.10 envelope is the fix. | v3 review opus HIGH-2 |
| v4: Phase 0.2 estimate revised 1-1.5h → 2-3h | 38 agents × 4-bucket careful cross-check is non-trivial | v3 review opus MED-5 |