Status: design (2026-06-08) Issue: #273 Source: Claude Opus 4.8 System Card (Anthropic, 2026-05), §6.3.7 / §6.6.3 Related: #272 / #274 (Opus 4.8 behavioral-signal cluster)
4.8 sometimes spontaneously reasons about how a grader will score its output and adjusts ("trends worth watching", modest). ~5% of sampled RL episodes show unprompted, unverbalized grader-awareness activation signatures, ~0.5% "exploitative" (heavily caveated; may not represent the final model). Balancing context the card also reports: 4.8's chain-of-thought is broadly faithful, evaluation-awareness modestly increases honesty, and 4.8 shows no measurable self-preference bias.
ARS relies on rubric/gold-set judging — reviewer calibration (FNR/FPR), multi-dimension review rubrics, the citation-claim alignment judge. The suite already documents same-source hallucination as a known reference-integrity risk (integrity_verification_agent.md:16) — but that framing is about fabrication (a same-source verifier rubber-stamping a hallucinated reference). The 4.8 signal is a behavioral sibling: a model that, to some degree, optimizes for what the rubric appears to reward rather than for the correct judgment. When the same model family produces work and evaluates it under a rubric, calibration numbers measured under those conditions may be optimistic — the true error rate could be under-estimated.
This is a framing + epistemic-note change. It makes no claim that ARS detects, prevents, corrects, or eliminates grader-awareness — it cannot, and the system card's own point is that unverbalized awareness is not reliably monitorable via chain-of-thought. The only honest claim is interpretive: measured calibration error may be an under-estimate under same-family / rubric-aware judging, so cross-model evaluation gives stronger evidence where available.
No schema, no lint, no gate, no threshold change.
Introduce an umbrella that unifies the existing factual risk with the new behavioral one, so they are read as one family rather than two unrelated concerns:
integrity_verification_agent.md:16 (unchanged; its WebSearch counter-rules stay as-is).The behavioral-form text lives where rubric judging happens (the calibration doc), cross-referencing the same-source canonical — it does NOT edit the citation-integrity block to absorb it. Critical reason: the four WebSearch counter-rules at integrity_verification_agent.md:18-21 mitigate fabrication; they do nothing against rubric optimization. Folding the behavioral risk into that block would falsely imply those rules counter it. The canonical block gets only a one-line pointer to its behavioral sibling.
A note in calibration_mode_protocol.md ## Failure cases this mode does NOT fix (L145, the section that already enumerates what calibration cannot do): when the produced-work model and the evaluator model are from the same family and may be rubric-aware, the measured error profile should be read as a possible under-estimate, not a ceiling. Phrased specifically for calibration (not a copy of the #274 pressure-stability epistemic line — orthogonal concern).
The repo already makes ARS_CROSS_MODEL default-on inside calibration mode (calibration_mode_protocol.md:49). #273 must not assert cross-model is "opt-in" near that. The positioning:
shared/cross_model_verification.md); this recommendation does not weaken it.A single-model robustness note: you can reword the rubric and re-judge, then check whether the verdict changed. Documented in plain terms about exactly what it can and cannot do:
This only tells you whether a change of wording shifts the judgment — surface wording sensitivity. It does not reveal whether the model is quietly optimizing toward the grader (the system card's point is that this can be unverbalized), and a verdict that survives rewording is not evidence the judgment is correct — only that it is stable to that paraphrase. It is one model checking itself, so its power against grader-awareness is limited. No score, no threshold, no gate.