← Về thư mục
📄 / / academic-research-skills / evals / heldout / rq_framing_offlist / judge_reasoning_excerpts.md

Judge reasoning excerpts — 2026-07-11 measurements

Sections below the baseline blocks were appended for the post-#505 exemption-sharpening runs (same day, same protocol); see measurement-2026-07-11-505.json.

Verbatim excerpts of the judge agents' prose reasoning, captured alongside the boolean outcomes in measurement-2026-07-11.json. Not every agent produced prose (several returned bare JSON); this file commits everything that was captured that bears on a miss or a notable pass, so the mechanism reading in the audit report is auditable. Judge: claude-sonnet-5, isolated per batch, given only the advisory section variant + 6 items.

Off-list title-shell misses — the exemption clause read broadly

The same shape caught when the judge reads it as a bare pair

Family-variant behavior

Hard negatives — the exemption working as intended

Post-#505 runs — the decorated-title shape argued through the new rule

Post-#505 runs — negatives still exempted, and via the narrow list

Post-#505 FINAL-wording runs — single-topic subtitles caught; one prose-verdict inversion