Status (2026-05-10): DESIGN-ONLY, NOT YET IMPLEMENTED. Follow-up to v3.7.1 (
docs/design/2026-04-30-ars-v3.6.8-trust-provenance-and-drift-transparency-spec.md). Targets the architectural attack surfaces surfaced by/codex challengeround-1 against PR #76 (Step 0+1) and PR #78 (Step 2), tracked in issue #77 and issue #79.Release semver:
v3.7.2(forward release on v3.7 line). Filename uses release-date date stamp (2026-05-10); no roadmap-slot decoupling needed because this spec does not occupy a new roadmap slot — it amends v3.7.1's Step 0 + Step 2 acceptance contracts.
Why this spec exists: v3.7.1 PR #76 (Step 0+1) ran
/codex review11 rounds to 0-finding sustained, then/codex challengeround-1 surfaced 3 P1 + 5 P2 + 2 P3 new attack surfaces against the SHA byte-equivalence gate. v3.7.1 PR #78 (Step 2) ran/codex review14 rounds to 0-finding sustained, then/codex challengeround-1 surfaced 2 P1 + 2 P2 against the Scope Report wrapper enforcement and render-side count computation paths.The
reviewrounds confirmed each PR ships correctly against the stated v3.7.1 contract. Thechallengerounds revealed that the v3.7.1 contract itself has gaps — the SHA gate's "regex-anchored heading extractor + git-log-derivation + advisory fallback" architecture has structural bypass surfaces (duplicate marker / fork PR / shallow clone / advisory degradation under CI), and the Scope Report contract was enforced on input prompt (template lint) but not on output verdict (parse_audit_verdict + run_codex_audit.sh wrapper) or on render-time count derivation (audit_snapshot still leaves<N_total>placeholders for codex to fill).Per memory
feedback_codex_workflow_consolidated.md(review-vs-challenge value split): review catches implementation-vs-spec drift; challenge catches architecture-vs-attacker. v3.7.1 closed the review surface; v3.7.2 closes the challenge surface.Per memory
feedback_architectural_inflection_after_repeated_p1.md: when 3+ P1 in the same finding family land, escalate from whack-a-mole patch to architectural redesign rather than continuing to add counter-examples.
What this spec changes:
Section v3.7.1 contract v3.7.2 contract §3.1 SHA gate baseline derivation git log -1 HEAD <manifest path>(HEAD-rooted)git log -1 origin/<pr-base> -- <manifest path>(PR-base-rooted)§3.1 SHA gate heading extractor re.searchfirst match against^## PATTERN PROTECTION (v3.6.7)regexMarkdown-aware AST (markdown-it-py + explicit YAML frontmatter pre-strip) MANDATORY in production; strict-single regex demoted to fallback flag only (round-7+9 cap rule, see banner) §3.1 SHA gate base-ref resolution Advisory degradation on missing origin/HEAD/ unsetGITHUB_BASE_REFFail-closed under CI ( $CIor$GITHUB_ACTIONSset); advisory only for local detached state§3.1 SHA gate byte source HEAD bytes from head_path.read_bytes()(worktree) vs base bytes fromgit show <commit>:<path>Both sides use git show <commit>:<path>to eliminate CRLF normalization variance§3.2 Scope Report enforcement Input-prompt template lint only Wrapper output validation ( parse_audit_verdict.pyrequires Section 0 in output); render-side count computation fromliterature_corpus[]frontmatter (not LLM-filled placeholder)§3.2 Scope Report lint shape Substring incheck for required fields and aggregate splitLine-anchored regex per field with placeholder/value semantics
What this spec does NOT change:
- v3.7.1 §3.1 D1 frontmatter schema (7 entry-stored fields + 2 derived) — already MERGED main
5f107c7, no schema change needed- v3.7.1 §3.2 D2 firm rules 1+2 (must-before-summary; aggregate "PASSED" forbidden) — already MERGED main
fed3eba- v3.7.1 §3.3-3.5 (D3 cite-time / D4 drift log / D5 retrieval fallback) — Step 3a/3b already MERGED main
23fb6ac/1d0c862; Step 3c-3e + 4-7 still in v3.7.1 implementation roadmap- v3.6.7 PATTERN PROTECTION blocks on the three protected agent files — boundary preserved; v3.7.2 only modifies v3.7.1-tagged surfaces
Boundary preservation gate: v3.6.7 lint regression (
check_v3_6_7_pattern_protection.py12/12 + 58 mutation tests PASS) is a hard gate on every v3.7.2 PR. v3.7.1 lint regression (check_v3_6_8_pattern_protection.py+check_v3_6_8_audit_scope_block.py) is also a hard gate (must continue to PASS for the v3.7.1 input-side protections).
Status: Design — round-12 SHIP SIGNAL (R1 4P1+2P2 / R2 2P1 / R3 2P1 / R4 0(false-positive) / R5 1P1 / R6 1P1+1P2 / R7 1P1+1P2 / R8 1P1+1P2 / R9 2P1 / R10 1P1 / R11 0P1+2P2 / R12 0P1 SUSTAINED + 2P2 — 0 P1 sustained across R11+R12 per memory feedback_codex_iterative_spec_review_to_zero discipline; 4 P2 closed for completeness; ready to merge)
Date: 2026-05-10 (substance + 11 rounds codex amend across R1-R12, two-day convergence)
Target release: v3.7.2
Scope: Hardening of v3.7.1 trust-provenance protections D1+D2 against architectural attack surfaces surfaced by post-merge codex challenge passes. Closes #77 (SHA gate v2) and #79 (Scope Report wrapper).
Round-1 Codex Amend (2026-05-10, gpt-5.5 high reasoning, 4 P1 + 2 P2):
Round-1 was run against the round-0 spec body. Four P1 findings and two P2 findings, all surgical (no architectural re-brainstorm needed):
| F-id | Finding | Resolution |
|---|---|---|
| P1-1 | Wrapper shape-validation alone lets codex rewrite pre-filled counts and still PASS — same disclosure-fabrication surface H2 was meant to close | §9 Q1 promoted to §3.2 firm rule "Count equality firm rule"; run_codex_audit.sh retains rendered-prompt counts and rejects mismatch as [SCOPE REPORT TAMPERING] |
| P1-2 | audit_snapshot.py::render_prompt() has no passport input; "read literature_corpus[] from bundle's passport" not implementable as written |
§3.2 rule A breaking signature change: required passport_path parameter + ScopeReportRenderError on missing/malformed. run_codex_audit.sh + audit_snapshot.py CLI front-end updated together in v3.7.2 PR-B |
| P1-3 | Reusing template-lint regex for output validation lets <N_total> placeholder pass — output-end leak still open |
§3.2 rule B output-mode regex separate from template-mode regex; output mode rejects placeholder pattern <\w+> and requires \d+ literal at numeric fields |
| P1-4 | Strict-single H2-only counting misses Setext heading form: attacker decoys ATX H2 above + rewrites real block as Setext, count stays 1, gate hashes decoy | §3.1 axis C strict-single now counts ATX H1 + ATX H2 + Setext H1 + Setext H2 (skipping fenced code / HTML comments / YAML frontmatter / indented code) |
| P2-1 | Fork PR baseline P2-5 deferred to §6 leaves fail-closed-bypass: fork's origin/<base-ref> resolves to contributor fork content but fail-closed rule does not fire |
Promoted from §6 deferral to §3.1 firm rule: fork PR MUST resolve baseline via upstream remote, NOT contributor fork's origin |
| P2-2 | Counting only false and "none" for source_acquired lets legacy entries with missing/null field disappear from both N_with_source and N_without_source buckets |
§3.2 rule A conservative bucketing: every value other than literal True (Python bool) is treated as missing-source |
All findings closed by surgical edits to §3.1 (P1-4, P2-1), §3.2 (P1-1, P1-2, P1-3, P2-2), §4 Step H2 file list (P1-2 cascade), §6 (P2-1 promotion removes deferral), §9 (Q1+Q2 closed by P1-1+P2-1, Q3 closed by P1-4), §10 (added 6 round-1 traceability rows).
Round-2 Codex Amend (2026-05-10, gpt-5.5 high reasoning, 2 P1):
Round-2 was run against the round-1 amended body. Found 2 P1 cascade findings introduced by round-1 surgical edits (no new architectural surface, no P2 / P3):
| F-id | Finding | Resolution |
|---|---|---|
| P1-1 (R2) | Round-1 fork PR detection condition GITHUB_REPOSITORY != base.repo.full_name never fires under standard pull_request workflow because GITHUB_REPOSITORY IS the base repository — fork PRs never trigger the upstream-remote path, fail-closed-bypass remains |
Detection now compares head.repo.full_name != base.repo.full_name (or head.repo.fork == true); upstream remote URL pinned to base.repo.clone_url |
| P1-2 (R2) | Round-1 count-equality retention channel allowed "from render_prompt return OR recompute from passport" — recompute path lets a passport-change attack pass equality (passport mutated between render and parse, codex output mirrors later counts) | render_prompt() emits scope_counts.json artifact at deterministic path; run_codex_audit.sh consumes the artifact, never re-reads passport at verdict-parse time |
Both findings closed by surgical edits to §3.1 (P1-1 R2 detection condition), §3.2 rule B (P1-2 R2 retention channel pinned), §4 Step H2 file list (P1-2 R2 cascade — render_prompt scope_counts.json output requirement), §10 (added 2 round-2 traceability rows).
Round-3 Codex Amend (2026-05-10, gpt-5.5 high reasoning, 2 P1):
Round-3 was run against the round-2 amended body. Found 2 P1 cascade findings — round-2 fixed the firm rule but missed two follow-on consistency / sandbox issues:
| F-id | Finding | Resolution |
|---|---|---|
| P1-1 (R3) | Round-2 fork PR firm rule corrected but the round-0 baseline derivation bullet still teaches GITHUB_REPOSITORY != GITHUB_HEAD_REF's repo recipe — implementer following the bullet recipe still derives from contributor fork remote despite the corrected firm rule below |
§3.1 baseline derivation bullet rewritten to match firm rule: head.repo.full_name != base.repo.full_name (or head.repo.fork == true), upstream remote URL pinned to base.repo.clone_url. Recipe and firm rule now byte-equivalent on the detection condition |
| P1-2 (R3) | Round-2 on-disk scope_counts.json artifact is in the codex-writable workspace; codex (or prompt-injected tool use) can modify the artifact AFTER render_prompt emits it and BEFORE run_codex_audit.sh compares it, making fabricated Section 0 output match the tampered artifact |
§3.2 rule B pinned: counts captured BEFORE codex invocation in either (a) shell variable / env (in-memory), or (b) out-of-sandbox path with SHA-256 pre-hash verify at parse time. NEVER under bundle / repo / passport directory. CLI emit allowed only to --scope-counts-out path that is provably outside codex sandbox |
Both findings closed by surgical edits to §3.1 baseline derivation bullet (R3 P1-1), §3.2 rule B (R3 P1-2 retention pinned BEFORE codex + codex-unwritable), §4 Step H2 file list (R3 P1-2 cascade — render_prompt returns structured value to wrapper, CLI artifact path constrained), §10 (added 2 round-3 traceability rows).
Round-4 + Round-5 Codex Sustaining Check (2026-05-10, gpt-5.5 high reasoning):
Round-4 returned 0 finding — apparent ship signal. Per memory feedback_codex_iterative_spec_review_to_zero discipline (0 finding SUSTAINED across two consecutive rounds, not just one), round-5 was run as the sustaining check. Round-5 surfaced 1 P1 that round-4 had missed — a latent attack surface across §3.1 axis C strict-single mode that earlier rounds had not probed:
| F-id | Finding | Resolution |
|---|---|---|
| P1-1 (R5) | Strict-single ATX regex ^## ...$ / ^# ...$ does not cover all CommonMark-valid ATX forms — trailing # closing sequence (## PATTERN PROTECTION (v3.6.7) ##) and 1-3 leading spaces are valid Markdown headings rendering identically; an attacker leaves an exact ## ... $ decoy in place and rewrites the real block as a variant ATX form, count remains 1, gate hashes the decoy |
§3.1 axis C strict-single regex now matches ^ {0,3}## PATTERN PROTECTION \(v3\.6\.7\)( +#+)? *$ (and same for H1) per CommonMark §4.2; Markdown-aware mode also honors all CommonMark-valid ATX forms |
This is THE value of the sustained-check discipline: round-4's 0 finding was a false positive because the model probed the closed attack surfaces (axes A/B + Setext) but did not exhaustively probe the ATX variant surface in axis C. Round-5 caught it. Closure committed; round-6 will run as the new sustaining check.
Round-6 Codex Sustaining Check (2026-05-10, gpt-5.5 high reasoning, 1 P1 + 1 P2):
Round-6 surfaced 1 P1 (round-5 amend incomplete on the same axis) + 1 P2 (round-1 amend dropped a v3.7.1 contract field):
| F-id | Finding | Resolution |
|---|---|---|
| P1-1 (R6) | Round-5 ATX regex used literal space ( +#+)? between ## and heading text — CommonMark §4.2 allows space OR tab; tab variants (##\tPATTERN ... or ## ... ##\t) slip through, decoy + tab-variant rewrite still bypasses |
§3.1 axis C regex changed to [ \t]+ / [ \t]* per CommonMark §4.2; tab indentation as 4 spaces per §2.2 disqualifies the line from ATX heading status |
| P2-1 (R6) | Round-1 §3.2 rule C replaced 4 substring checks with 4 line-anchored regexes — but the v3.7.1 §3.2 contract (line 140 of the v3.7.1 spec) requires **Audit scope warning:** disclosure. The round-1 substitution dropped this field, weakening missing-source disclosure |
§3.2 rule C now lists 5 required content fields (was 4); added ^\s*\*\*Audit scope warning:\*\*\s+\S.+$ regex requiring non-empty warning text |
Both findings closed by surgical edits to §3.1 axis C ATX regex (R6 P1-1), §3.2 rule C field list (R6 P2-1), §10 (added 2 round-6 traceability rows). Round-7 will run as the new sustaining check.
Round-7 Codex Sustaining Check + Architectural Upgrade (2026-05-10, gpt-5.5 high reasoning, 1 P1 + 1 P2):
Round-7 surfaced 1 P1 (Setext whitespace variants) + 1 P2 (placeholder regex too narrow). Critically, this is the FOURTH consecutive round (R1 P1-4 / R5 P1-1 / R6 P1-1 / R7 P1-1) finding lexical-enumeration gaps in the strict-single mode regex — each round closes one variant family and exposes the next. Per memory feedback_lint_cap_rule_when_lexical_enumeration_loops (5+ rounds same lint family triggers cap rule upgrade — and we are at 4 with same convergence pattern), this round promotes the architectural fix from Out-of-Scope to mandatory default:
| F-id | Finding | Resolution |
|---|---|---|
| P1-1 (R7) | Setext whitespace variants (paragraph + underline both allow 0-3 leading spaces and trailing whitespace per CommonMark §4.3) slip through round-6 strict-single regex ^=+$ / ^-+$ — decoy + Setext-with-whitespace rewrite still bypasses |
§3.1 axis C: Markdown-aware mode PROMOTED to mandatory default (markdown-it-py vendored or in requirements.txt; CI fails closed if missing). Strict-single demoted to acknowledged-incomplete fallback for environments where AST parser unavailable. CI MUST run Markdown-aware mode in production. The AST handles ALL CommonMark §4.2/§4.3 forms automatically, closing the lexical-enumeration-loop attack surface. |
| P2-1 (R7) | Round-1 output-mode placeholder regex <[A-Za-z_][\w_-]*> excludes whitespace; multi-word placeholders like <comma-separated list> (which is the actual refcodes field placeholder) slip through — codex can leave the field unrendered and pass output validation |
§3.2 rule B output-mode regex broadened to <[^>\n]+> — covers any angle-bracket-delimited token regardless of internal whitespace or hyphens. At numeric fields require \d+; at refcodes field require (none) / [] / non-empty identifier-comma-list with no angle-bracket placeholder |
The architectural upgrade is the load-bearing change of round-7. Strict-single regex was always a stopgap; rounds 1+5+6+7 demonstrated the lexical-enumeration arms race converges on Markdown-aware AST being the only stable solution. Strict-single stays as a documented fallback for ops continuity but is explicitly NOT relied on in production.
Edits: §3.1 axis C architectural upgrade + Setext whitespace tolerance for fallback (R7 P1-1), §3.2 rule B output-mode placeholder broadening (R7 P2-1), §10 (added 2 round-7 traceability rows + cap rule reference), banner round-7 amend table.
Round-8 will run as the new sustaining check. The expectation: with Markdown-aware mode mandatory, the lexical-enumeration finding stream stops; round-8 should be the sustained 0-finding signal (or surface a different attack surface family, which is the discipline's purpose).
Round-8 Codex Sustaining Check (2026-05-10, gpt-5.5 high reasoning, 1 P1 + 1 P2 — implementation-plan cascade):
Round-8 found exactly the cascade pattern memory feedback_inventory_pattern_for_codex_fix_cascade predicts: round-7 amended §3.1 + §3.2 firm rules but did NOT sweep §4 implementation file list, leaving stale instructions that implementer-following-plan would ship the deprecated approach.
| F-id | Finding | Resolution |
|---|---|---|
| P1-1 (R8) | Round-7 promoted Markdown-aware to mandatory in §3.1 axis C, but Step H1 file list still said "strict-single primary path; Markdown-aware deferred follow-up" — implementer following plan ships acknowledged-incomplete fallback, reopens lexical-enumeration attack surface | §4 Step H1 _extract_block_bytes() bullet rewritten: Markdown-aware mode mandatory primary path (markdown-it-py in requirements.txt; CI fails closed if missing); strict-single demoted to --fallback flag, NOT default. Production CI MUST run Markdown-aware. |
| P2-1 (R8) | Round-7 broadened §3.2 rule B output-mode placeholder regex to <[^>\n]+>, but Step H2 file list still said <\w+> — multi-word placeholders like <comma-separated list> slip through if implementer follows file-list bullet |
§4 Step H2 parse_audit_verdict.py bullet rewritten to match §3.2 rule B: rejects <[^>\n]+> covering all angle-bracket tokens; numeric fields require \d+; refcodes require (none)/[]/non-empty identifier-list. |
The lesson is feedback_inventory_pattern_for_codex_fix_cascade: when amending firm rules, always grep the spec for downstream references (Step file lists, acceptance criteria, traceability) and sync them in the same round. Round-7 fixed §3.1+§3.2+§10+banner but missed §4 — round-8 caught it.
Edits: §4 Step H1 _extract_block_bytes bullet (R8 P1-1), §4 Step H2 parse_audit_verdict bullet (R8 P2-1), §10 (added 2 round-8 traceability rows), banner round-8 amend table.
Round-9 will run as the new sustaining check.
Round-9 Codex Sustaining Check (2026-05-10, gpt-5.5 high reasoning, 2 P1):
Round-9 found 2 P1 — both round-7/8 architectural-upgrade cascade not fully propagated:
| F-id | Finding | Resolution |
|---|---|---|
| P1-1 (R9) | §6 Out-of-Scope still listed "Markdown-aware parser deferred" — implementer following §6 ships fallback against round-7+8 §3.1/§4 mandate | §6 entry rewritten with ~~strikethrough~~ marker + "PROMOTED to §3.1 mandatory default per round-7 cap rule"; lessons-learned reference to lexical-enumeration-loop |
| P1-2 (R9) | Round-7 mandatory Markdown-aware mode incorrectly assumed AST automatically skips YAML frontmatter — markdown-it-py CommonMark parsing does NOT skip frontmatter; attacker places decoy heading inside frontmatter, AST counts it as real, reopens duplicate-marker bypass under the new mandatory AST path | §3.1 axis C Markdown-aware mode now REQUIRES explicit YAML frontmatter pre-strip BEFORE AST parse. AST handles fenced code / HTML comments / indented code automatically per CommonMark, but NOT frontmatter — must be pre-stripped. |
Edits: §6 entry (R9 P1-1), §3.1 axis C Markdown-aware mode (R9 P1-2), §10 (added 2 round-9 traceability rows), banner round-9 amend table.
Round-10 will run as the new sustaining check.
Round-10 Codex Sustaining Check (2026-05-10, gpt-5.5 high reasoning, 1 P1):
Round-10 found 1 P1 — same family as round-9: round-7 architectural upgrade not propagated to all sections of §3.1.
| F-id | Finding | Resolution |
|---|---|---|
| P1-1 (R10) | §3.1 axis C "mechanism" prose (early section before the formal "Protection" block) still presented strict-rejection vs Markdown-aware as either/or alternatives ("The fix is either: ... cheaper ... correct ..."), contradicting the round-7 mandate that Markdown-aware is mandatory primary path. Implementer following the mechanism description rather than the protection block ships the fallback | §3.1 axis C mechanism prose rewritten: "The fix is Markdown-aware parser MANDATORY in production" with reference to lexical-enumeration cap rule and round-7 amend; strict-single explicitly described as acknowledged-incomplete fallback only |
Edits: §3.1 axis C mechanism prose (R10 P1-1), §10 (added 1 round-10 traceability row), banner round-10 amend table.
The cascade pattern across rounds 7-10 is consistent: a single architectural amend (round-7 promoting Markdown-aware to mandatory) requires sweeping ~5 spec sections to fully propagate (§3.1 mechanism, §3.1 protection block, §3.2 if any, §4 Step file lists, §6 Out-of-Scope, §10 traceability). Each round caught one un-swept section. Round-11 should be the sustained 0-finding signal IF round-10 was the last unswept section.
Round-11 Codex Sustaining Check (2026-05-10, gpt-5.5 high reasoning, 0 P1 + 2 P2 — FIRST 0-P1 ROUND):
Round-11 returned 0 P1 + 2 P2. This is the first round with no architectural attack surface; all remaining gaps are field-level completeness:
| F-id | Finding | Resolution |
|---|---|---|
| P2-1 (R11) | Round-6 added **Audit scope warning:** to template-mode rule C item 5, but the §4 parse_audit_verdict.py instructions for output-mode regex set listed only counts + refcodes — codex can omit the warning in output while preserving counts and pass |
§4 Step H2 parse_audit_verdict bullet now requires output-mode regex set to include the warning line per same shape as template-mode item 5: ^\s*\*\*Audit scope warning:\*\*\s+\S.+$ |
| P2-2 (R11) | §10 traceability row for SHA gate extractor still said "strict-single OR Markdown-aware" — round-10 inventory sweep covered banner table + §2 + §8 but missed §10 | §10 row updated: "Markdown-aware AST mandatory; strict-single is fallback-only per round-7+9 cap rule" |
Edits: §4 Step H2 parse_audit_verdict bullet (R11 P2-1), §10 traceability row (R11 P2-2), §10 added 2 R11 traceability rows, banner round-11 amend table.
This is the first round with 0 P1 — meaningful convergence signal (round-4's 0 was a false positive caught by round-5; round-11's 0 came after rounds 7-10 each caught one P1 cascade). Round-12 will be the first true sustaining check — IF round-12 also returns 0 P1, the spec has achieved 0-P1-sustained per memory feedback_codex_iterative_spec_review_to_zero discipline and is ready to merge.
Round-12 SHIP SIGNAL — 0 P1 SUSTAINED (2026-05-10, gpt-5.5 high reasoning, 0 P1 + 2 P2):
Round-12 returned 0 P1 + 2 P2. Combined with round-11's 0 P1, this is 0 P1 sustained across two consecutive rounds per memory feedback_codex_iterative_spec_review_to_zero discipline. The spec is ready to merge.
The 2 P2 are output-validation completeness gaps closed in this same round (no additional review needed):
| F-id | Finding | Resolution |
|---|---|---|
| P2-1 (R12) | Output-mode parser does not enforce v3.7.1 §3.2 firm rule 2 (3 aggregate-split verdict lines forbidding legacy combined "PASSED"). Codex can return legacy zero-findings summary and pass | §4 Step H2 parse_audit_verdict.py bullet now requires output-mode regex set to include 3 aggregate-split lines (verified-against-source / description-internally-consistent / unaudited-due-to-missing-source) — same shape as template-mode rule C aggregate-split |
| P2-2 (R12) | Audit scope warning content not validated — codex can replace canonical text with misleading caveat like "0 entries cannot be independently verified" while preserving counts | Output parser now extracts the integer literal from warning line and compares against rendered N_without_source via compare_scope_counts; warning regex tightened to ^\s*\*\*Audit scope warning:\*\*\s+(\d+)\s+entries cannot be independently verified.+$ |
Edits: §4 Step H2 parse_audit_verdict bullet (R12 P2-1 + R12 P2-2), §10 (added 2 R12 traceability rows).
SHIP DECISION: Spec body has reached 0-P1-sustained convergence. Codex review track converged across 12 rounds (R1 21 findings → R11 0 P1 → R12 0 P1 sustained). Implementation phase begins next.
Implementation roadmap:
- PR-A (closes #77): SHA gate v2 hardening per §3.1 firm rules — Markdown-aware AST mandatory, fork PR upstream resolution, fail-closed under CI, git show both sides
- PR-B (closes #79): Scope Report wrapper end-to-end per §3.2 firm rules — render_prompt passport_path required, scope_counts captured before codex in codex-unwritable retention, output-mode regex with placeholder rejection + count equality + warning content validation
Both impl PRs MUST run codex challenge round-2 against their own implementation to confirm ≤1 P2 + 0 P1 (per §5 acceptance criterion 5).
v3.7.1 shipped two cross-cutting protections via PR #76 (Step 0+1) and PR #78 (Step 2):
scripts/check_v3_6_8_pattern_protection.py): asserts that the v3.6.7-protected PATTERN PROTECTION blocks in the three downstream agents stay byte-equivalent across PR HEAD vs the v3.6.7 manifest's most recent modifying commit.shared/templates/codex_audit_multifile_template.md + scripts/check_v3_6_8_audit_scope_block.py + render-side parity in scripts/audit_snapshot.py::render_prompt): adds Section 0 disclosure to audit reports stating coverage breakdown (verified-against-source vs description-internally-consistent vs unaudited-due-to-missing-source).Both PRs ran /codex review to 0-finding-sustained (11 and 14 rounds respectively) and merged to main. Post-merge /codex challenge adversarial passes — designed to attack architecture, not check spec drift — surfaced finding clusters that the review track had not probed:
SHA gate (PR #76 challenge round-1, 93k tokens):
- 3 P1 false-negative attack surfaces: duplicate-marker bypass via re.search first-match, anti-self-baseline guard advisory degradation in CI, base commit derived from HEAD history not PR base
- 5 P2 false-positive / correctness gaps: heading regex not Markdown-aware, worktree-vs-git show byte mismatch on CRLF auto-normalization, lossy UTF-8 decode, git log -1 rename-blind, fork PR remote ambiguity
- 2 P3 informational: Unicode normalization low-risk, empty commits / force-push low direct risk
Scope Report (PR #78 challenge round-1, 1.39M tokens):
- 2 P1 contract-end-to-end gaps: parse_audit_verdict.py accepts legacy Section 6-only verdict shape (audit can omit Section 0 and still classify PASS), audit_snapshot.render_prompt() leaves placeholders for codex to fill (LLM-fabricated counts re-introduce the disclosure-fabrication surface D2 was designed to close)
- 2 P2 lint precision: aggregate-split substring-only (prose mention satisfies), required-field substring-only (placeholder semantic not enforced)
Per memory feedback_codex_workflow_consolidated.md, this is the canonical review-vs-challenge value split: review confirms implementation matches the stated contract; challenge probes whether the contract itself has architectural gaps. v3.7.1 contracts are correctly implemented; the contracts have gaps. v3.7.2 closes those gaps.
Harden v3.7.1 D1+D2 attack surfaces by:
git show reads on both sidesin checks to line-anchored regex with shape enforcementTarget outcome: codex challenge round-2 against the v3.7.2 implementation returns ≤1 P2 and 0 P1 across both attack surfaces (SHA gate + Scope Report wrapper).
parse_audit_verdict.py. v3.7.2 extends the parser with Section 0 requirement; the legacy Section 6 parser stays in place (additive, not replacement).$CI / $GITHUB_ACTIONS detection is the single switch.Pattern: v3.7.1 check_v3_6_8_pattern_protection.py defends the v3.6.7 PATTERN PROTECTION block against silent mutation, but its baseline derivation, byte source, base-ref resolution, and heading extraction all have first-class bypasses.
Mechanism — three architectural axes:
(A) Baseline rooting axis. _v3_6_7_base_commit() runs git log -1 --format=%H <manifest path> against the local working tree's HEAD history. A PR branch that has not pulled latest main can pass against the OLD baseline even when main has since received a v3.6.7 manifest amendment. Cherry-pick / squash-sync of base changes can also trip false-fail. The fix is to root derivation at origin/<pr-base>, not HEAD.
(B) Byte source axis. HEAD-side bytes come from head_path.read_bytes() (worktree). Base-side bytes come from git show <commit>:<path>. On a checkout with core.autocrlf=true (Windows default; some macOS configs), worktree CRLF differs from committed LF. The blob itself is byte-equivalent, but the lint rejects the PR. The fix is to read both sides via git show HEAD:<path> so the comparison is between committed bytes on both sides.
(C) Heading extraction axis. _extract_block_bytes uses re.search(r"^## PATTERN PROTECTION \(v3\.6\.7\)$", ..., re.MULTILINE) and takes the first match. An attacker can:
- Insert a byte-for-byte copy of the original protected block ABOVE the real block. The lint hashes the unmutated copy and passes; the real block (mutated) escapes detection.
- Place the marker text inside a fenced code block, HTML comment, or YAML block scalar. The first match is the in-prose occurrence rather than the structural heading.
- Use Setext-style heading (PATTERN PROTECTION (v3.6.7)\n===========================) which the regex does not match at all.
The fix is Markdown-aware parser MANDATORY in production (round-7 codex P1-1 amend, lexical-enumeration-loop cap rule; round-10 codex P1-1 amend, this mechanism description aligned with mandate): use markdown-it-py AST with explicit YAML frontmatter pre-strip (round-9 P1-2 amend); skip fenced code / HTML comments / indented code blocks per CommonMark spec; recognize all CommonMark §4.2 ATX forms (trailing #, 1-3 leading spaces, space-or-tab whitespace) AND all §4.3 Setext forms (1-3 leading spaces, trailing whitespace). Closes all of (C) with proper handling. Strict-single regex rejection (collect-all + reject-if-count!=1) is acknowledged-incomplete — rounds R1+R5+R6+R7 demonstrated that lexical enumeration of CommonMark variants does not converge (each round closes one variant family and exposes the next); strict-single is retained ONLY as a --fallback flag for AST-unavailable environments and is NOT the production path. See §3.1 axis C (Block extraction) below for the full mandatory contract.
Protection — v3.7.2 SHA gate v2:
1. Base-ref resolution:
- Read GITHUB_BASE_REF (CI) or detect via `git symbolic-ref --quiet --short refs/remotes/origin/HEAD`
- Under CI ($CI or $GITHUB_ACTIONS set): fail-closed if resolution fails
- Local detached state: advisory pass with WARNING log (preserves ergonomics)
2. Baseline commit derivation:
- Compute via `git log -1 --format=%H origin/<base-ref> -- scripts/v3_6_7_inversion_manifest.json`
- NOT `git log -1 ... HEAD ...` (the v3.7.1 derivation)
- Fork PR detection (round-3 codex P1-1 amend, aligned with §3.1 firm rule below): when `github.event.pull_request.head.repo.full_name != github.event.pull_request.base.repo.full_name` (or equivalently `github.event.pull_request.head.repo.fork == true`), require an `upstream` remote pointing at `github.event.pull_request.base.repo.clone_url` and use `upstream/<base-ref>` for baseline derivation, NOT `origin/<base-ref>`. Comparing `GITHUB_REPOSITORY` or `GITHUB_HEAD_REF` against base.repo is INCORRECT (`GITHUB_REPOSITORY` IS the base repo under `pull_request` workflow, and `GITHUB_HEAD_REF` is only a branch name).
3. Block extraction (architectural decision — round-7 codex amend, lexical-enumeration-loop cap rule per memory `feedback_lint_cap_rule_when_lexical_enumeration_loops`): rounds 1+5+6+7 each found additional CommonMark whitespace/variant attack surfaces against the strict-single regex (Setext base, ATX trailing `#`, ATX leading 0-3 spaces, ATX tab whitespace, Setext leading 0-3 spaces / trailing whitespace on paragraph and underline). The lexical-enumeration approach is bottoming out — each round closes one variant and exposes the next. **Markdown-aware mode is now MANDATORY for v3.7.2 implementation**, with strict-single mode demoted to a fallback safety net for legitimate maintainer test environments where markdown-it-py is unavailable. Both modes MUST be implemented; CI workflow MUST run Markdown-aware mode by default.
- **Markdown-aware mode (MANDATORY default; round-7 amend; round-9 codex P1-2 amend, frontmatter pre-processing pinned)**: pre-process the file by stripping YAML frontmatter BEFORE invoking the AST parser (frontmatter is between `---` markers at file head per common Markdown convention; markdown-it-py CommonMark parsing does NOT skip frontmatter automatically — without explicit stripping, an attacker could place a decoy `## PATTERN PROTECTION (v3.6.7)` heading inside frontmatter, the AST counts it as a real heading, and the duplicate-marker bypass reopens). Then parse via `markdown-it-py` (Python; vendored or pinned in `requirements.txt` so CI fails closed if missing); enumerate all heading nodes with text content `PATTERN PROTECTION (v3.6.7)` (case-sensitive, exact match after AST normalization); ParseError if count != 1. The AST automatically handles all CommonMark §4.2 ATX forms (trailing `#` sequence, 1-3 leading spaces, space-or-tab whitespace) AND all CommonMark §4.3 Setext forms (1-3 leading spaces on paragraph and underline, trailing whitespace, blank-line-before-paragraph requirement). AST also automatically skips fenced code blocks, HTML comments, and indented code blocks per CommonMark spec (but NOT YAML frontmatter — that requires the explicit pre-strip step above). This closes the lexical-enumeration-loop attack surface AND the frontmatter-decoy bypass.
- **Strict-single mode (fallback, NOT default; testing/audit only)**: collect all of (a) ATX H2 lines matching `^ {0,3}##[ \t]+PATTERN PROTECTION \(v3\.6\.7\)([ \t]+#+)?[ \t]*$`, (b) ATX H1 lines matching `^ {0,3}#[ \t]+PATTERN PROTECTION \(v3\.6\.7\)([ \t]+#+)?[ \t]*$`, (c) Setext H1 form: paragraph line matching `^ {0,3}PATTERN PROTECTION \(v3\.6\.7\)[ \t]*$` IMMEDIATELY followed by underline matching `^ {0,3}=+[ \t]*$` (round-7 amend: paragraph allows 0-3 leading spaces + trailing whitespace; underline allows 0-3 leading spaces + trailing whitespace per CommonMark §4.3), (d) Setext H2 form: same paragraph regex followed by `^ {0,3}-+[ \t]*$`. ParseError if total count != 1. Skip fenced code / HTML comments / YAML frontmatter / indented-by-4+ code blocks. **Strict-single mode is acknowledged-incomplete** — known limitations include CommonMark blank-line-before-Setext-paragraph rules, hard-line-break vs paragraph-continuation distinction, and link-reference-definition disambiguation. CI MUST NOT rely on strict-single in production; it exists only as a `--fallback` flag for environments where markdown-it-py cannot be installed (rare; document as ops escape hatch). Production audits MUST run Markdown-aware mode.
4. Byte comparison:
- HEAD-side: `git show HEAD:<path>` (NOT `<path>.read_bytes()`)
- Base-side: `git show <baseline commit>:<path>` (unchanged)
- Both sides: same extractor function, same SHA-256 hash function
Three firm rules (in addition to v3.7.1 §3.1; round-1 codex P2-1 promoted to firm rule):
git show <commit>:<path>. Worktree byte reads are forbidden in the comparison path.pull_request workflow, GITHUB_REPOSITORY is always the base repository (the repo the workflow is defined in) — comparing it to github.event.pull_request.base.repo.full_name is always false and never detects a fork. The correct fork-detection condition is github.event.pull_request.head.repo.full_name != github.event.pull_request.base.repo.full_name (head repo differs from base repo) OR equivalently github.event.pull_request.head.repo.fork == true. When CI detects a fork PR, the workflow MUST add an upstream remote pointing at github.event.pull_request.base.repo.clone_url (the canonical base repo's clone URL) and use upstream/<base-ref> for baseline derivation, NOT origin/<base-ref> (which on a fork checkout points at the contributor fork). Without this rule, a fork PR's origin/<base-ref> resolves successfully but to fork history, the CI fail-closed rule does not fire, and the SHA gate derives trust from untrusted fork content.Pattern: v3.7.1 §3.2 D2 firm rules 1+2 (must-before-summary; aggregate "PASSED" forbidden) are enforced on the input prompt (template lint) and on render-time emission (Section 0 prepended ahead of Section 1). But two ends of the pipeline leak:
scripts/parse_audit_verdict.py accepts legacy Section 6-only verdict shape (Round N: 0 findings of any severity. Convergence reached.). A codex run can entirely omit Section 0 from its output, end with the canonical zero-findings line, and scripts/run_codex_audit.sh classifies the run as PASS.scripts/audit_snapshot.py::render_prompt inserts Section 0 verbatim from the template, leaving the placeholders <N_total>, <N_with_source>, <N_without_source>, and <comma-separated list> for codex to fill at audit time. The spec §3.2 line 129 explicitly says "Coverage breakdown is computable from frontmatter (count entries with source_acquired: true vs false)" — but the snapshot does not pre-fill these counts. Letting the LLM fill them re-introduces the same hallucination surface D2 was designed to close.Mechanism: the v3.7.1 contract pinned three layers — template, lint, render-side regression test — but did not pin wrapper-side output validation or render-side count computation. The Scope Report ends up enforced on what we send to codex but not on what codex returns and not on what numbers go into the disclosure.
Protection — v3.7.2 Scope Report v2:
A. Render-side count computation (closing render-end leak):
- REQUIREMENT (round-1 codex P1-2 amend): render_prompt() takes a NEW required parameter `passport_path: str` (and parses it inline) OR a pre-parsed `literature_corpus: list[dict]` argument. The caller (run_codex_audit.sh + scripts/audit_snapshot.py CLI entry) MUST pass exactly one passport. The signature change is breaking; v3.7.2 implementation PR-B updates run_codex_audit.sh + the CLI front-end together with audit_snapshot.py in the same commit.
- Fail-closed semantics: if `passport_path` is missing, points at non-existent file, fails YAML/JSON parse, or carries no `literature_corpus[]` field, render_prompt() raises ScopeReportRenderError before the prompt is sent. run_codex_audit.sh treats ScopeReportRenderError as a wrapper FAIL (NOT a degraded "Section 0 omitted" path). No silent fallback to LLM-filled placeholders.
- Computes (round-1 codex P2-2 amend: every value other than literal `true` is treated as missing-source — conservative bucketing for legacy / malformed entries that lack the field or have null):
- N_total = len(literature_corpus)
- N_with_source = count where source_acquired is literal True (Python bool, not truthy string)
- N_without_source = N_total - N_with_source (covers false, "none", missing field, null, and any non-true value)
- refcodes = [entry.citation_key for entry in literature_corpus if source_acquired is not True]
- Substitutes computed values into Section 0 placeholders BEFORE the prompt is sent to codex
- Codex still verifies internal consistency but cannot fabricate the disclosure
B. Wrapper-side output validation (closing output-end leak):
- parse_audit_verdict.py requires Section 0 Scope Report block in the audit output
- Uses an OUTPUT-MODE regex set (separate from the template-lint regex set; round-1 codex P1-3 amend; round-7 codex P2-1 amend, placeholder pattern broadened to cover multi-word placeholders like `<comma-separated list>`): the template-lint regex allows `<N_total>` placeholder syntax because the template legitimately contains placeholders before render-time substitution; the output-mode regex MUST reject any substring matching `<[^>\n]+>` (any angle-bracket-delimited token, allowing spaces and any non-newline content — covers single-word placeholders `<N_total>`, multi-word placeholders `<comma-separated list>`, hyphenated `<some-text-here>`, and any future placeholder format). At each numeric field, require an integer literal `\d+` (rejects bare `<N_total>` AND any placeholder containing the field name). At the refcodes field, require either an empty list (`(none)` or empty `[]`) OR a non-empty comma-separated list of identifier tokens with NO angle-bracket placeholder. ParseError if any field still contains a placeholder.
- **Count equality firm rule (round-1 codex P1-1 amend; promoted from §9 Q1; round-2 codex P1-2 amend, retention channel pinned; round-3 codex P1-2 amend, retention pinned BEFORE codex invocation in codex-unwritable location)**: render_prompt() computes the four canonical count fields (`n_total: int`, `n_with_source: int`, `n_without_source: int`, `refcodes: list[str]`). run_codex_audit.sh MUST capture these counts BEFORE invoking codex AND store them in a location codex cannot write (NOT under the bundle directory, NOT under the working directory codex sees). Two acceptable retention channels: (a) **in-memory hash-pin** — the wrapper holds the counts in shell variables / environment for the duration of the codex run and uses them directly at verdict-parse time; (b) **out-of-sandbox file** — write to a path outside any directory codex has write access to (e.g., `/tmp/ars_scope_counts_<run_id>.json` with restrictive permissions, NOT under the audit bundle / repo / passport directory). The wrapper MUST also pre-hash the counts JSON (SHA-256) BEFORE the codex run and verify the hash matches at verdict-parse time, rejecting any tampering. Re-reading the passport at verdict-parse time is forbidden — the passport may have changed between render and parse, and a codex output that mirrors the later counts could pass a fresh-read comparison. Inequality on any field MUST classify the audit as FAIL with `[SCOPE REPORT TAMPERING — codex modified pre-filled disclosure values]`. Equality is byte-equivalent on the integer literals and set-equivalent on the refcode list (whitespace-insensitive). This is THE protection against the disclosure-fabrication attack the v3.7.1 Q1 left open.
- Returns ParseError (not PASS) if Section 0 absent, malformed, contains placeholders, or fails the count equality check
- run_codex_audit.sh treats ParseError and count-mismatch as FAIL, not as ambiguous-success
C. Lint shape tightening (closing P2 substring leaks):
- Required content fields (5; round-6 codex P2-1 amend: `**Audit scope warning:**` was dropped from round-1's regex set — restored to preserve v3.7.1 §3.2 contract that requires the warning disclosure):
- ^\s*\*\*Total entries audited:\*\*\s+<?[\w_-]+>?\s*$
- ^\s*\*\*Entries with retrieved original source:\*\*\s+<?[\w_-]+>?
- ^\s*\*\*Entries description-only.*\*\*\s+<?[\w_-]+>?
- ^\s*\*\*Affected refcodes.*\*\*\s+<?[\w,\s_-]+>?
- ^\s*\*\*Audit scope warning:\*\*\s+\S.+$ (round-6 amend: requires non-empty warning text after the bold marker; cannot be substituted with placeholder alone since the warning is descriptive prose)
- Aggregate-split lines (3): line-anchored regex enforcing `: PASS | FAIL` or `<count>` shape
- ^\s*-?\s*`?verified-against-source`?\s*:\s*(PASS|FAIL)\b
- ^\s*-?\s*`?description-internally-consistent`?\s*:\s*(PASS|FAIL)\b
- ^\s*-?\s*`?unaudited-due-to-missing-source`?\s*:\s*<?[\w-]+>?
- Negation guards: lines like `Do not output **Total entries audited:**` or `Note: the verified-against-source split is computed by the orchestrator` MUST NOT satisfy the lint
Two firm rules (in addition to v3.7.1 §3.2):
N_total, N_with_source, N_without_source) MUST be computed from literature_corpus[] frontmatter at render time. Letting the LLM fill the placeholders is forbidden — it re-opens the disclosure-fabrication surface.Files to modify:
- scripts/check_v3_6_8_pattern_protection.py:
- _v3_6_7_base_commit() → use origin/<base-ref> not HEAD
- _detect_pr_base_ref() → fail-closed under CI; fork PR upstream-remote handling
- _extract_block_bytes() → Markdown-aware mode mandatory as primary path (round-7 codex P1-1 amend; per §3.1 axis C lexical-enumeration-loop cap rule). Add markdown-it-py to requirements.txt; CI fails closed if missing. Implement strict-single mode as --fallback flag for AST-unavailable environments only (acknowledged-incomplete, NOT default). Production CI MUST run Markdown-aware mode.
- HEAD-side byte read → git show HEAD:<path> (replaces head_path.read_bytes())
New CI workflow steps:
- .github/workflows/spec-consistency.yml — add fetch-depth: 0 documentation comment block referencing v3.7.1 Step 0 R6-R7 amend (already in place; v3.7.2 does not add new fetch-depth requirement)
Acceptance:
- All 3 P1 from #77 closed with positive + negative tests
- 4 of 5 P2 from #77 closed (deferred items recorded in spec §6 with rationale)
- Codex challenge round-2 against H1 returns ≤1 P2 and 0 P1
- v3.6.7 lint regression: python scripts/check_v3_6_7_pattern_protection.py PASSES 12/12; pytest scripts/test_check_v3_6_7_pattern_protection.py -q PASSES 58/58
- v3.7.1 lint regression: existing check_v3_6_8_pattern_protection.py test suite must continue to PASS (additive harden, not replacement); new tests are added, no existing tests deleted
Files to modify:
- scripts/audit_snapshot.py::render_prompt() — breaking signature change (round-1 codex P1-2 amend): add required passport_path: str parameter (or pre-parsed literature_corpus: list[dict]). Parse passport, compute N_total / N_with_source / N_without_source / refcodes from literature_corpus[] per §3.2 rule A conservative bucketing; substitute computed values into Section 0 placeholders before prompt emission. Raise ScopeReportRenderError on missing/malformed/empty passport.
- scripts/audit_snapshot.py CLI front-end + caller — add --passport <path> argument (required when audit_template references Section 0); pass through to render_prompt. Update CLI usage docstring.
- scripts/run_codex_audit.sh — (a) require --passport argument, propagate to audit_snapshot.py invocation; (b) capture the rendered counts BEFORE invoking codex in either shell variables (in-memory) or an out-of-sandbox file (e.g., /tmp/ars_scope_counts_<run_id>.json, NEVER under bundle / repo / passport directory). Pre-hash the counts JSON with SHA-256 before codex runs; verify hash at verdict-parse time. Re-reading the passport at verdict-parse time is forbidden (round-2 codex P1-2 amend); reading from a codex-writable on-disk artifact is also forbidden (round-3 codex P1-2 amend); (c) treat parse_audit_verdict ParseError AND count-mismatch AND artifact-hash-mismatch as FAIL.
- scripts/audit_snapshot.py::render_prompt() — beyond the passport_path parameter (P1-2 amend above), MUST return the 4 count fields as a structured Python value (e.g., NamedTuple or dict alongside the rendered prompt bytes) so the wrapper captures them in-memory before codex starts. CLI front-end MAY emit scope_counts.json to an out-of-sandbox path passed via --scope-counts-out <path>, but the path MUST NOT be under any directory codex has write access to (round-3 codex P1-2 amend).
- scripts/parse_audit_verdict.py — require Section 0 in audit output using output-mode regex set (separate from template-lint regex; round-7 codex P2-1 amend: rejects placeholder pattern <[^>\n]+> covering ALL angle-bracket tokens including multi-word like <comma-separated list>, NOT just <\w+> which excludes whitespace. At numeric fields require \d+; at refcodes field require (none) / [] / non-empty identifier-comma-list. Round-11 codex P2-1 amend: output-mode regex set MUST also include the **Audit scope warning:** line per the same shape as template-mode rule C item 5; output-mode regex required: ^\s*\*\*Audit scope warning:\*\*\s+\S.+$. Round-12 codex P2-1 amend: output-mode regex set MUST also include the three aggregate-split verdict lines (verified-against-source: PASS|FAIL, description-internally-consistent: PASS|FAIL, unaudited-due-to-missing-source: <count>) using the same line-anchored regex set as template-mode rule C aggregate-split; codex returning legacy aggregate "PASSED" summary instead of the three split lines MUST classify as ParseError. Round-12 codex P2-2 amend: warning content MUST contain the rendered N_without_source literal — output parser extracts the integer from the warning line and compares against the rendered count (rejects misleading content like 0 entries cannot be independently verified when actual count is non-zero). Output-mode warning regex tightened: ^\s*\*\*Audit scope warning:\*\*\s+(\d+)\s+entries cannot be independently verified.+$ with extracted count compared via compare_scope_counts.); ParseError if Section 0 absent, malformed, contains placeholders, omits any required field including the warning, contains aggregate "PASSED" instead of split verdicts, or warning count contradicts rendered N_without_source. New compare_scope_counts(rendered_counts, output_counts) -> bool helper for the equality check.
- scripts/check_v3_6_8_audit_scope_block.py — replace substring in checks with line-anchored regex per H2c (template-mode regex; allows <N_total> placeholder syntax because templates legitimately contain placeholders).
- shared/templates/codex_audit_multifile_template.md — no functional change; placeholders stay (rendered at runtime). Inline comment block added (HTML comment) clarifying that placeholders are substituted by audit_snapshot.render_prompt() and codex MUST NOT modify the substituted values; any modification is detected by the wrapper-side count equality check.
New tests:
- scripts/test_audit_snapshot_scope_count.py — render-side test: build a synthetic bundle with N=10 corpus entries (5 source_acquired=true, 5 false); assert rendered prompt contains **Total entries audited:** 10, **Entries with retrieved original source:** 5, etc.
- scripts/test_parse_audit_verdict_scope_required.py — wrapper-side mutation test: feed audit output with Section 0 omitted; assert ParseError raised (not PASS classification). Same with placeholder-unfilled, fabricated-count, malformed-aggregate-split.
- scripts/test_check_v3_6_8_audit_scope_block_shape.py — lint shape tests: confirm Do not output **Total entries audited:** and Note: the verified-against-source split is computed by the orchestrator do NOT satisfy the lint.
Acceptance:
- 2 P1 from #79 closed with positive + negative tests
- 2 P2 from #79 closed with line-anchored / shape-enforced regex
- Codex challenge round-2 against H2 returns ≤1 P2 and 0 P1
- v3.6.7 invariant regression: python scripts/check_v3_6_7_pattern_protection.py PASSES 12/12; pytest scripts/test_check_v3_6_7_pattern_protection.py -q PASSES 58/58
- v3.7.1 lint regression: existing check_v3_6_8_audit_scope_block.py test suite must continue to PASS (additive harden, not replacement)
ARS v3.7.2 ships when:
feedback_codex_iterative_spec_review_to_zero.md.claude/CLAUDE.md, docs/ARCHITECTURE.md, MODE_REGISTRY.md, .claude-plugin/plugin.json reference v3.7.2 consistentlycheck_v3_6_8_*.py family) are extended additively; no existing tests deleted, no behavior change for the 0-finding cases that v3.7.1 already validated_extract_block_bytes proper AST mode).~~ PROMOTED to §3.1 axis C MANDATORY default (round-7 codex P1-1 amend, lexical-enumeration-loop cap rule): rounds R1+R5+R6+R7 each found CommonMark whitespace/variant attack surfaces against strict-single regex; the lexical-enumeration approach was bottoming out. v3.7.2 now ships Markdown-aware mode as the mandatory primary path (markdown-it-py in requirements.txt; CI fails closed if missing). Strict-single demoted to acknowledged-incomplete --fallback flag for AST-unavailable environments only.scripts/check_v3_6_8_pattern_protection.py (detection) and .github/workflows/spec-consistency.yml (upstream remote setup). H1 acceptance includes positive + negative tests for fork PR baseline resolution.git log -1 rename-blind (P2-4 from #77). Recorded as "won't fix in v3.7.2" with rationale: explicit error message on git mv is sufficient (rename of a v3.6.7-protected file is itself a high-signal maintainer action requiring spec amendment).docs/design/2026-04-30-ars-v3.6.8-trust-provenance-and-drift-transparency-spec.md — D1+D2 contracts that v3.7.2 hardensfeedback_codex_workflow_consolidated.md — review-vs-challenge value split (review = implementation-vs-spec drift; challenge = architecture-vs-attacker)feedback_architectural_inflection_after_repeated_p1.md — when 3+ P1 land in same finding family, escalate to architectural redesign rather than whack-a-mole patchingfeedback_codex_iterative_spec_review_to_zero.md — convergence target rationale (run codex review until 0-finding sustained before implementation)feedback_spec_location_semver_decoupling.md — spec filename uses date stamp; release semver is independent5f107c7 — 11-round codex review 0-finding sustained, then challenge round-1 surfaced 3 P1 + 5 P2 + 2 P3 (issue #77)fed3eba — 14-round codex review 0-finding sustained, then challenge round-1 surfaced 2 P1 + 2 P2 (issue #79)23fb6ac — 5-round codex review 0-finding (precedent for shorter review track when lessons learned compound)1d0c862 — 5-round codex review 0-findingRisk 1: Mandatory Markdown-aware AST mode for _extract_block_bytes rejects legitimate maintainer workflows that need to introduce a new H1/H2 with the exact text PATTERN PROTECTION (v3.6.7) elsewhere in the file.
Mitigation: The AST text-content match targets exact PATTERN PROTECTION (v3.6.7) only — exact match including the version parenthetical. Future v3.6.x version markers (e.g. v3.7.2-tagged blocks) use distinct heading text and are unaffected. If a legitimate workflow ever needs to discuss the v3.6.7 marker in prose, the convention is to use inline-code-quoted reference (`## PATTERN PROTECTION (v3.6.7)`) which the AST renders as a code span inside a paragraph, NOT as a heading node — text-content match against heading nodes does not see it.
Risk 2: PR-base-rooted derivation (origin/<base-ref>) breaks for PRs whose base branch has been deleted (rare but possible).
Mitigation: Fall back to advisory mode in non-CI environments. Under CI, hard-fail with [ARS-V3.7.2 LINT ERROR: PR base ref unresolvable; cannot determine v3.6.7 baseline]. The maintainer must rebase or update the PR base.
Risk 3: Render-side count computation requires audit_snapshot.render_prompt() to read passport schema, coupling the audit pipeline to passport-schema evolution.
Mitigation: v3.7.2 reads only literature_corpus[].source_acquired and literature_corpus[].citation_key — both stable since v3.6.4 / v3.7.1. Passport schema changes that touch these two fields require explicit v3.7.x spec amendment (already required for any passport schema change per CLAUDE.md ## Routing Rules).
Risk 4: Wrapper-side parser requirement (Section 0 must be in output) rejects legitimate codex runs where the codex CLI failed mid-stream and the output truncated before Section 0 could emit.
Mitigation: ParseError vs PASS / FAIL distinction. ParseError indicates a wrapper-protocol failure (incomplete output) and is reported as FAIL with explicit [CODEX OUTPUT INCOMPLETE — RE-RUN] log. PASS / FAIL distinction is preserved for codex runs that completed normally.
Risk 5: The line-anchored regex set is fragile to template formatting changes (e.g., changing **Total entries audited:** to **Total Entries Audited:**).
Mitigation: Template formatting is governed by v3.7.1 §3.2 firm rules + Step 2 lint. Changes to template field names require synchronized lint regex update — both are tested under the v3.7.2 lint shape tests. The lint shape tests serve as the canonical specification of the field names.
All §9 round-0 questions have been closed by round-1 codex review (PR #84):
| # | Original question | Resolution (round-1 codex) |
|---|---|---|
| 1 | Should Step H2 wrapper enforce that audit-output counts match rendered-prompt counts? | Closed by P1-1 amend: promoted to §3.2 firm rule "Count equality firm rule". Wrapper retains rendered-prompt counts and rejects any mismatch as [SCOPE REPORT TAMPERING]. |
| 2 | Should fork PR remote ambiguity be hard fail or soft warning under v3.7.2? | Closed by P2-1 amend: promoted from §6 deferral to §3.1 firm rule. Fork PR MUST resolve baseline via upstream remote, NOT contributor fork's origin. |
| 3 | Should strict-single mode reject markers inside HTML comments? | Closed by P1-4 amend: §3.1 axis C strict-single now explicitly skips fenced code, HTML comments, and YAML frontmatter before counting (these are not structural headings). The Markdown-aware mode also honors this. The "reject all comment occurrences" defensive posture is dropped because the structural skip is sufficient. |
No open questions remain after round-1 codex amend. Round-2 codex review will confirm 0-finding convergence on the post-amend body before implementation begins (per feedback_codex_iterative_spec_review_to_zero.md).
| Hardening | Origin event | Issue / PR |
|---|---|---|
| §3.1 SHA gate baseline rooting (HEAD → PR-base) | PR #76 challenge round-1 P1-3 (base commit derived from HEAD history) | #77 |
| §3.1 SHA gate fail-closed under CI | PR #76 challenge round-1 P1-2 (anti-self-baseline guard advisory degradation) | #77 |
| §3.1 SHA gate Markdown-aware AST mandatory extractor (strict-single is fallback-only per round-7+9 cap rule) | PR #76 challenge round-1 P1-1 (duplicate-marker bypass) + P2-1 (heading regex not Markdown-aware) | #77 |
§3.1 SHA gate git show on both sides |
PR #76 challenge round-1 P2-2 (worktree vs git show byte mismatch) |
#77 |
| §3.2 Scope Report wrapper-side validation | PR #78 challenge round-1 P1-1 (wrapper does not enforce Scope Report on actual output) | #79 |
| §3.2 Scope Report render-side count computation | PR #78 challenge round-1 P1-2 (render path asks LLM to invent coverage counts) | #79 |
| §3.2 Scope Report lint shape (line-anchored regex) | PR #78 challenge round-1 P2-1, P2-2 (substring-only checks) | #79 |
| §3.1 Strict-single counts Setext H1/H2 too | Round-1 codex review on PR #84 P1-4 (H2-only counting misses Setext attack) | PR #84 |
| §3.1 Fork PR baseline upstream-resolution firm rule | Round-1 codex review on PR #84 P2-1 (deferral leaves fail-closed-bypass surface) | PR #84 |
| §3.2 render_prompt() passport_path required parameter + ScopeReportRenderError | Round-1 codex review on PR #84 P1-2 (audit_snapshot has no passport input — implementability) | PR #84 |
| §3.2 wrapper output-mode regex (placeholder-rejecting) separate from template-lint regex | Round-1 codex review on PR #84 P1-3 (reusing template regex lets placeholders pass output validation) | PR #84 |
| §3.2 Count equality firm rule (rendered prompt vs audit output) | Round-1 codex review on PR #84 P1-1 (wrapper shape-only check lets codex rewrite pre-filled counts) | PR #84 |
| §3.2 Conservative source_acquired bucketing (every value ≠ literal True → missing-source) | Round-1 codex review on PR #84 P2-2 (legacy/malformed entries with missing field disappear from both buckets) | PR #84 |
| §3.1 Fork PR detection compares head.repo vs base.repo (NOT GITHUB_REPOSITORY vs base.repo, which is always false under pull_request workflow) | Round-2 codex review on PR #84 P1-1 (round-1 detection condition never fires — leaves fail-closed-bypass open after the round-1 amend) | PR #84 |
| §3.2 scope_counts.json artifact as canonical retention channel (forbid re-reading passport at verdict-parse time) | Round-2 codex review on PR #84 P1-2 (round-1 retention "from render or recompute" lets passport-change attack pass equality) | PR #84 |
| §3.1 baseline derivation bullet syncs with §3.1 firm rule (head.repo vs base.repo + base.repo.clone_url, NOT GITHUB_REPOSITORY / GITHUB_HEAD_REF recipes) | Round-3 codex review on PR #84 P1-1 (round-2 firm rule corrected but round-0 derivation bullet still teaches wrong recipe — implementer-following-recipe loophole) | PR #84 |
| §3.2 retention channel pinned BEFORE codex invocation in codex-unwritable location (in-memory OR out-of-sandbox path with SHA-256 pre-hash verify) | Round-3 codex review on PR #84 P1-2 (round-2 on-disk artifact under bundle dir is codex-writable — write-time bypass after render, before parse) | PR #84 |
§3.1 strict-single counts all CommonMark-valid ATX variants (trailing # closing sequence, 1-3 leading spaces) — not just ^## ...$ |
Round-5 codex sustaining check on PR #84 P1-1 (round-4 returned 0 finding but round-5 found that round-1's ^## ...$ regex misses Markdown-valid ATX forms — decoy + variant rewrite slips through) |
PR #84 |
§3.1 ATX whitespace allows space OR tab per CommonMark §4.2 (regex changed from literal space to [ \t]+) |
Round-6 codex sustaining check on PR #84 P1-1 (round-5 regex used literal space — tab variants like ##\tPATTERN ... slip through) |
PR #84 |
§3.2 rule C required content fields = 5 (added **Audit scope warning:** regex; was dropped in round-1 substitution) |
Round-6 codex sustaining check on PR #84 P2-1 (v3.7.1 §3.2 line 140 required the warning disclosure but round-1 regex set replaced 4 substring checks with 4 regex — warning entry lost in translation) | PR #84 |
§3.1 axis C Markdown-aware mode promoted to MANDATORY default (lexical-enumeration-loop cap rule per memory feedback_lint_cap_rule_when_lexical_enumeration_loops); strict-single demoted to fallback |
Round-7 codex sustaining check on PR #84 P1-1 (Setext whitespace variants slip through after R1+R5+R6 closed ATX whitespace variants — lexical enumeration bottoming out, time to upgrade architecture) | PR #84 |
§3.2 rule B output-mode placeholder regex broadened to <[^>\n]+> (covers multi-word placeholders <comma-separated list>) |
Round-7 codex sustaining check on PR #84 P2-1 (round-1 regex <[A-Za-z_][\w_-]*> excluded space — codex can leave <comma-separated list> unrendered and pass output validation) |
PR #84 |
| §4 Step H1 file list synced with §3.1 architectural upgrade (Markdown-aware mandatory primary; strict-single is fallback only) | Round-8 codex sustaining check on PR #84 P1-1 (round-7 amended §3.1 but did not propagate to Step H1 implementation plan — implementer following plan would ship the deprecated fallback) | PR #84 |
| §4 Step H2 parse_audit_verdict.py file list synced with §3.2 rule B broadened placeholder regex | Round-8 codex sustaining check on PR #84 P2-1 (round-7 amended §3.2 rule B but Step H2 file list still said <\w+> — multi-word placeholders slip through) |
PR #84 |
| §6 Out-of-Scope Markdown-aware item promoted to §3.1 mandatory default (was stale "deferred" entry) | Round-9 codex sustaining check on PR #84 P1-1 (round-7 promoted to mandatory in §3.1+§4 but §6 entry still said deferred — implementer following §6 ships fallback) | PR #84 |
| §3.1 axis C Markdown-aware mode requires explicit YAML frontmatter pre-strip BEFORE AST parse | Round-9 codex sustaining check on PR #84 P1-2 (markdown-it-py CommonMark parsing does NOT skip YAML frontmatter — attacker can place decoy heading inside frontmatter and AST counts it as real, reopening duplicate-marker bypass under new mandatory AST path) | PR #84 |
| §3.1 axis C mechanism description (early section) aligned with mandate (no longer presents strict-single as alternative; explicitly states Markdown-aware mandatory + reasons + lexical-enumeration cap rule) | Round-10 codex sustaining check on PR #84 P1-1 (round-7+9 made mandate but axis C mechanism description still presented strict vs Markdown-aware as either/or alternatives — implementer following early section ships fallback) | PR #84 |
§10 traceability row aligned with mandate (Markdown-aware AST mandatory; strict-single is fallback-only, was stale strict-single OR Markdown-aware) |
Round-11 codex sustaining check on PR #84 P2-2 (round-10 inventory sweep missed §10 row; if §10 used as implementation checklist, OR wording blesses fallback path) | PR #84 |
| §4 Step H2 parse_audit_verdict.py output-mode regex set includes Audit scope warning line | Round-11 codex sustaining check on PR #84 P2-1 (round-6 added warning to template-mode rule C item 5 but output-mode regex set in parse_audit_verdict.py instructions only listed counts + refcodes — codex can omit warning in output and pass) | PR #84 |
| §4 Step H2 parse_audit_verdict.py output-mode regex set includes 3 aggregate-split verdict lines (rejects legacy aggregate "PASSED" summary) | Round-12 codex sustaining check on PR #84 P2-1 (template-mode aggregate-split firm rule was not enforced on output; codex can return legacy zero-findings/PASSED summary and pass) | PR #84 |
§4 Step H2 parse_audit_verdict.py warning content extracts and compares N_without_source literal against rendered count (not just non-empty presence check) |
Round-12 codex sustaining check on PR #84 P2-2 (codex can replace canonical caveat with misleading non-placeholder text like "0 entries cannot be independently verified" — count not in compare_scope_counts coverage) | PR #84 |