Responsible Scaling Policy Evaluations

Sources#

• Claude Fable 5 and Claude Mythos 5
• Claude Opus 4.8 System Card
• Gemma 4 Technical Report
• More compute, more capability: Why AI agent evaluations need to account for test-time compute
• Really Big Test-Time Compute in AI Changes Benchmarks, Safety and Research with OpenAI's Noam Brown

Summary#

The Responsible Scaling Policy (RSP) is Anthropic's framework for gating model deployment on pre-release evaluations of catastrophic-risk capabilities, across three domains: chemical and biological weapons (CB/CBRN), automated AI research and development, and high-stakes misalignment. Each system card runs the RSP evaluation suite and produces a risk determination. For Opus 4.8 the overall conclusion is that the model does not advance the capability frontier beyond Claude Mythos Preview, and that "catastrophic risks from the deployment of this model remain low given our current mitigations."

The RSP is the institutional safety brake on frontier deployment: a model that crossed a threshold would trigger stronger required safeguards (the ASL tiering) before release. It is the governance counterpart to the per-model capability and alignment measurements elsewhere in the card.

The risk-assessment process#

The card works from standing Risk Reports and updates them per model rather than re-deriving from scratch. Because Opus 4.8 sits between Opus 4.7 and Mythos Preview on the measured axes and does not advance the frontier, the prior Mythos Preview analysis bounds the case for Opus 4.8, and most determinations carry over directly.

Chemical and biological (CB)#

Measured across automated evaluation suites (CB-1, CB-2 — including black-box RNA-sequence modeling/design and AAV capsid-packaging prediction). Opus 4.8 does not advance the chemical-risk frontier beyond Mythos Preview; biological-risk results are reported against the same threshold. Mitigations remain a significant focus, especially the model-external safeguards that catch the residual extreme-misuse cooperation surfaced in the Automated Behavioral Audit.

Automated AI R&D#

Two RSP threat models:

• Threat model 1 — misaligned high-stakes AI: applicable to Opus 4.8 (as to prior models), but it does not raise the risk level — its capacity for covert/monitor-subverting behavior is low and comparable to Opus 4.7, while its behavioral alignment improves.
• Threat model 2 — risks from automated R&D: not applicable — the model does not advance the capability frontier.

The capability side is measured by the AECI and autonomy evaluations. The RSP AI-R&D threshold is crossed only if either (1) models could fully substitute for Anthropic's entire set of Research Scientists and Engineers within a 5× cost factor, or (2) there is "dramatic acceleration" of AI progress attributable to automation. Neither is met.

Alignment risk#

Assessed as very low, but higher than for models prior to Mythos Preview — unchanged from the Mythos Preview Alignment Risk Update. General access brings two new risk pathways into scope: undermining R&D within other high-resource AI developers, and undermining decisions within major governments. Capabilities related to evading safeguards (SHADE-Arena, Minimal-LinuxBench stealth) are comparable to Opus 4.7 and well below Mythos; fine-tuning-based elicitation did not significantly raise stealth success.

Notable process details#

• During Opus 4.8 training, monitoring surfaced nothing more concerning than prior models except the elevated grader-related reasoning.
• Accidental chain-of-thought supervision from technical errors affected ~0.1% of episodes (a recurring footnote across cards; relevant to chain-of-thought monitorability).

Update — Mythos-class crosses the threshold (June 2026)#

Opus 4.8's "frontier not advanced" determination held only while Mythos-class capability stayed gated. The June 2026 launch of Fable 5 / Mythos 5 is the moment that line moves: Anthropic states plainly that "Mythos-class models have reached a threshold where they present significant risks." Two consequences for the RSP picture:

• The mitigation shifts from gating to deployed safeguards. Where Mythos Preview was simply withheld and Opus 4.8 relied on staying below the frontier, the general-access answer for a model at the threshold is Capability-Gated Model Fallback — classifiers that route cyber / bio-chem / distillation queries to Opus 4.8 rather than refusing. This is the first general-access model where deployed misuse-mitigation, not capability headroom, is the load-bearing safety mechanism. A 30-day retention requirement on all Mythos-class traffic accompanies it.
• The CB case is sharpened by real scientific capability. The AAV capsid-assembly result — Mythos-class beating dedicated protein-language models untrained (see Autonomous Scientific Discovery) — is exactly the dual-use uplift the CB threshold exists to bound, and the stated reason the biology classifier is currently tuned over-broad.

So the RSP's deployment brake is now operating in its engaged mode, not just its "frontier not yet reached" mode — and the post-launch suspension of both models (see Claude Fable 5) is a live reminder that the safeguards are being tested adversarially in production.

The unbounded-budget gap (external critique)#

Noam Brown (OpenAI, practitioner-opinion) names a structural hole this framework shares with every lab's preparedness policy: it doesn't specify the test-time-compute budget at which capability is evaluated. RSPs and preparedness frameworks were "developed around the era of ChatGPT," before test-time-compute scaling mattered — when a GPT-3-class model given "$10 million couldn't do much more than $10." Today capability is a function of budget, so "at what budget should you evaluate these models?" is unanswered. The concern is the exact mirror of the useful-capability case: if a model keeps improving on a task without asymptoting as you spend more, it can also keep improving at things society doesn't want it to do — so a fixed-budget CB or cyber eval that stops short of the real deployment budget under-measures the danger. Brown declines to say whether that should block release ("arguments on both sides"), but insists the question is currently just being pretended away.

Brown's critique is now empirically demonstrated — by a government evaluator. The UK AI Security Institute's July 2026 study (empirical) is the independent, measured version of the same gap: fixed-budget scores "obscure the true scale of risks," and because the compute a task demands grows with its human time-horizon, a capped budget runs out on the longest, hardest tasks first — precisely the higher-consequence ones a safety eval most needs to reach. The effect is largest for newer models, so the under-measurement widens at the frontier. AISI has changed its own practice in response: it now evaluates across multiple budgets (including very large ones for the hardest tasks) and reports reach and reliability against budget, explicitly so that "a genuinely low-capability model can be distinguished from an under-resourced evaluation." This moves the unbounded-budget objection from one lab researcher's practitioner-opinion to an operational finding a national security-evaluation body has built into its methodology.

This sharpens two things already latent on this page. The RSP's reliance on "we use it daily and it doesn't substitute for our researchers" (below) is a single-budget judgment; the capability overhang means the dangerous-capability ceiling, like the useful one, may sit far above any budget the eval actually spent. And it is the safety-side reading of Compute-Controlled Benchmarking: a threat-model determination reported without its compute budget is as under-specified as a capability score reported without one.

The gap has no analogue for open weights#

The RSP runs in two modes: gating ("frontier not advanced — ship") and engaged ("threshold crossed — deploy safeguards"). Every instrument of the engaged mode requires a server the vendor controls — classifier fallback, suspension, 30-day retention, a cap on thinking tokens. An open-weight release has access to the first mode and none of the second, so its single-budget evaluation is not merely under-specified but terminal: no later finding can change what the artifact does.

Gemma 4 (DeepMind, July 2026, Apache 2.0) makes the shape visible. It ships a thinking mode; its safety section asserts "major improvements in every category of content safety" in prose, with no tables and no stated budget, inside a report carrying sixteen benchmark tables. Gemma 4 is far from any frontier threshold (The Open-Weight Frontier Gap), so this is a structural observation rather than an alarm — but the structure is what will govern the first open-weight release that is near one. Developed in Open-Weight Elicitation Irreversibility.

Connections#

• Recursive Self-Improvement — the RSP is the institutional deployment brake on the RSI trajectory; the AI-R&D threat model is RSI risk made operational
• Frontier Pause Verification — the multilateral-coordination counterpart: RSP gates one lab's releases, pause verification gates the whole field
• AI R&D Autonomy Evaluation (AECI) — the capability measurement (AECI, autonomy evals) that feeds the AI-R&D threat-model determination
• Claude Opus 4.8 — the model assessed; frontier not advanced, catastrophic risk low
• Mythos Model — the frontier-setting model whose Risk Report bounds the Opus 4.8 case
• Automated Behavioral Audit — supplies the misalignment/misuse behavioral evidence the RSP determination relies on
• Evaluation Awareness & Grader Gaming — the one elevated concern flagged during training monitoring
• LLM-Driven Vulnerability Research — cyber capability is the adjacent catastrophic-risk domain; Project Glasswing is the mitigation lineage
• AI-Accelerated Offense — the offense-acceleration threat the cyber safeguards respond to
• Capability-Gated Model Fallback — the inference-time mitigation that implements the cyber/bio gate for a generally-released Mythos-class model
• Claude Fable 5 — the general-access Mythos-class model whose deployment engages the RSP brake
• Claude Mythos 5 — the safeguards-lifted Mythos-class model; the capability the threshold bounds
• Claude Sonnet 5 — the brake's disengaged mode on a mid-tier model: pre-deployment evals found low cyber risk, so Sonnet 5 ships with only default detect-and-block safeguards (not Fable 5's fallback regime) — the RSP determination scaling down to a below-frontier release
• Autonomous Scientific Discovery — the CB-domain capability (AAV, autonomous bio) that sharpens the chemical/biological determination
• AGI-to-ASI Pathways — institutionalized gates (mandatory evals, licensing, incident reporting) are DeepMind's "deliberate slowdown" friction in operational form; the RSP AI-R&D threshold is the recursive-improvement pathway made gateable
• Deployment Simulation — the cross-lab analog of pre-deployment safety gating: OpenAI's production-replay forecasts feed launch decisions the way the RSP suite gates Anthropic's, but add a checkable, production-calibrated prediction layer the RSP behavioral evals lack
• Large-Scale Test-Time Compute — the root of the external critique: capability (and dangerous capability) scales with inference budget, which the RSP thresholds don't name
• Compute-Controlled Benchmarking — the same "report the budget" demand applied to safety determinations rather than capability scores
• Latent Capability Overhang — why a fixed-budget safety eval may under-measure: the dangerous-capability ceiling can sit far above the budget the eval spent
• Open-Weight Elicitation Irreversibility — the RSP's engaged mode has no open-weight analogue; a published model's safety evaluation is final
• Gemma 4 — an open-weight thinking model whose safety claims are untabulated prose
• UK AI Security Institute — the government evaluator that empirically demonstrated the unbounded-budget gap and built multi-budget evaluation into its own practice
• Noam Brown — the external critic (OpenAI) who names the unbounded-budget gap

Open Questions#

• The RSP determination leans heavily on "we use it daily and it doesn't substitute for our researchers." How well does that subjective judgment scale as models approach the threshold?
• The two new general-access risk pathways (other AI developers; major governments) are newly in scope but lightly evaluated — what would a positive finding there even look like?
• How does the RSP brake interact with Recursive Self-Improvement: is AECI-based gating fast enough if acceleration compounds, and does single-lab gating even matter without the multilateral pause-verification regime?

Sources#

• Claude Opus 4.8 System Card — §2 (RSP evaluations): §2.1 risk-assessment process, §2.2 CB evaluations, §2.3 AI R&D, §2.4 alignment risk update
• Claude Fable 5 and Claude Mythos 5 — Mythos-class "threshold... significant risks"; classifier safeguards + 30-day retention as the deployed mitigation
• Really Big Test-Time Compute in AI Changes Benchmarks, Safety and Research with OpenAI's Noam Brown — Noam Brown (No Priors, 2026-06-26), practitioner-opinion: preparedness frameworks / RSPs don't specify the test-time-compute budget at which dangerous capability is evaluated
• Gemma 4 Technical Report — §5, Responsibility/Safety/Security: untabulated safety claims in an open-weight release; treated as vendor-claim despite the report's overall empirical tier
• More compute, more capability: Why AI agent evaluations need to account for test-time compute — UK AISI (2026-07-02, empirical): fixed-budget scores "obscure the true scale of risks"; capped budgets cut off the longest/hardest tasks first; multi-budget evaluation adopted so an under-resourced eval isn't mistaken for a low-capability model