Frontier Pause Verification

Sources#

• A Framework for Frontier AI and the Dawning of a New Age
• How to pace the US frontier
• Silent Revision: Measuring Undisclosed Change in the Safety Frameworks of Frontier AI Developers
• The Singularity Has a Distribution: Exact Timing Laws and Detection Limits for a Stochastic Intelligence Explosion
• What if automating AI R&D triggers an intelligence explosion?
• What This OpenAI Insider Saw That Made Him Quit | The Ezra Klein Show
• When AI builds itself

Summary#

The governance response in When AI builds itself: if the RSI trajectory holds, the world should at least have the option to slow or temporarily pause frontier AI development so that societal structures and alignment research can keep up. But a pause is only useful if it is credible — multilateral and verifiable — because a unilateral pause merely changes who leads. The Anthropic Institute's stated agenda is to build the systems a credible slowdown would require. This is the policy bookend to the RSP's internal deployment brake: RSP gates one lab's releases; pause verification is the between-labs, between-nations coordination problem.

Why a unilateral pause isn't enough#

Anthropic's position: "if a slowdown simply lets the least cautious actors catch up technologically, it could leave everyone less safe." A unilateral pause by one lab "is achievable immediately, but accomplishes much less: it would change who the front-runner is, but it would not create the wider deliberative process that is currently missing." Anthropic says it would slow or temporarily pause if other frontier-or-near-frontier developers did so in a verifiable manner — making verification the linchpin.

Why verification is unusually hard for AI#

A credible pause needs multiple well-resourced labs, in multiple countries, agreeing to stop under the same conditions, each able to verify the others actually stopped. AI makes even detectability (a lower bar than full verifiability) harder than for other technologies:

• Training runs are easier to conceal than missile silos. No large physical signature to observe.
• Inputs are general-purpose. Compute, data, and talent aren't weapons-specific, so you can't gate the precursors the way you can with, say, fissile material.
• The incentive to defect quietly is enormous — "whoever continues while others pause could inherit the lead."
• A credible pause must also specify what triggers it, what lifts it, and who adjudicates — undefined today.

The precedent and the time problem#

It is "not necessarily impossible in principle" — the world built verification regimes for complex technologies, e.g. the Intermediate-Range Nuclear Forces (INF) Treaty. But those regimes "took decades to build both the infrastructure and the trust," and on the RSI timeline "we don't have that long." Hence the Institute's bet: start building the detectability/verification infrastructure now, ahead of any agreement, so the option exists when it's needed. In the coming months Anthropic plans to convene policymakers, researchers, civil society, and other AI companies, and to publish the output — explicitly inviting non-AI-company voices into the deliberation.

A weaker mechanism that needs no verification regime#

Musk's July 2026 proposal (Cross-Lab Pre-Release Review) is worth setting beside this page because it targets the same coordination gap and sidesteps the hard part. It asks for 1–2 weeks of competitor API access before a frontier release, with government alerted only if a lab refuses to act on a flagged danger. Verification is not needed because the developer volunteers the access — there is nothing to detect.

What that buys and what it doesn't: it produces a recommendation to delay one release, not a verified stop, and it says nothing about training runs, which is where this page's detectability problem actually lives. It is a partial answer to the third open question below — competitors adjudicate, on the Motion Picture Association model — and a poor one, since the proposal has no criteria, no secretariat, and no step between "a rival says delay" and "the government is alerted." Musk also arrives at it from the opposite premise: he signed the 2023 pause letter and now says that "even if there was a stop button we probably shouldn't press it," so his mechanism is designed to be compatible with acceleration rather than to enable a stop.

A slowdown proposed as a setting on a national body#

Hassabis's July 2026 Standards Body proposal (practitioner-opinion) reaches this page's subject from an unexpected direction. Its escalation ceiling is that the review regime "could be ratcheted up if the seriousness of the situation demands, including coordinating a slowdown in development among the Frontier Labs if deemed necessary" — a coordinated cross-lab slowdown, offered as a subordinate clause and a dial setting on a US standards body rather than as a treaty problem.

It is the only place in the corpus where a single institution is imagined to coordinate a slowdown, and it supplies none of what this page says such a thing requires: no detectability infrastructure, no verification of who actually stopped, no named trigger, no lift condition, no enforcement mechanism, and no multilateral scope beyond the hope that a US body becomes "a strong starting point for creating shared international standards." Read against this page, it is the coordination problem restated as an administrative power — which is informative about how the problem looks from inside a lab, and is not a solution to it.

The domestic answer: an itemized verification menu, and a challenge to the premise#

The AI Futures Project's August 2026 proposals (Lifland, Halstead, Dean, Larsen, Kodama, 2026-08-05, practitioner-opinion) are the first source in the corpus to answer this page's first question with a list rather than a gesture. Auditor access is laid out as five ascending rungs — public information only; the ability to have high-level questions answered or benchmarks run (METR's Frontier Risk Report as the exemplar); employee-level access to company systems (METR's access at Anthropic); being embedded in the company; and direct audit of compute allocation, for example via network taps, whose user-privacy cost the authors flag themselves. Alongside it: whistleblower interviews and protections as a cheaper substitute if penalties are harsh enough, and TEE attestations and zero-knowledge proofs as software-only options that "might be even more robust."

The structural claim underneath the menu is a downgrade of the difficulty, and it splits the problem in two. Embedded auditors plus government enforcement are argued to be sufficient for the domestic case; hardware-based verification such as network taps is argued to be necessary only internationally, where the counterparty is a nation-state with physical access and supply-chain reach. On that reading this page's detectability problem is a treaty problem, not a regulation problem — and a jurisdiction can pace its own labs with auditors carrying company laptops.

And it disagrees with this page's founding argument. Anthropic's case is that a unilateral pause "would change who the front-runner is, but it would not create the wider deliberative process that is currently missing." These authors make the objection concrete at the jurisdiction level rather than the lab level and answer it with numbers: the raw US capability lead is "about 4-8 months" on Epoch's ECI country view, but because much of China's progress "comes from distilling the American frontier and using American-discovered algorithms," they "estimate that if the US halted, China would take approximately a year to catch up" — enough breathing room to be worth using, with the option to speed up again "if Chinese dominance becomes imminent." They add the inversion: US labs are "currently far from having security that is robust to nation state actors," so pacing capability while sprinting on security "could actually result in a larger lead over China at superhuman capability levels." This is the corpus's first quantified answer to the front-runner objection. The numbers are the proposers' own estimates, and both sources are unmeasured proposals, so the disagreement is logged rather than adjudicated.

What the proposal does not supply is what this page's third question asks. Escalation between its own four options is a recommendation to a reader, not a power assigned to anyone, and authority is assumed away outright: "In this post, we assumed that the US government was enforcing the pacing."

A pause that is true and narrowly scoped (October 2026)#

This page asks whether a pause can be verified. An insider account adds a prior question: what the pause covers. David Robinson, OpenAI's system-card lead until October 2026 (What This OpenAI Insider Saw That Made Him Quit | The Ezra Klein Show, practitioner-opinion), says that when he left, OpenAI had "publicly said reinforcement learning training … was paused. They didn't pause pre-training," and that the company's descriptions of what it paused are "true and they're very carefully scoped." So a pause can be accurately disclosed, and checkable in principle, while the largest training process keeps running. Verification confirms the claim. It says nothing about whether the claim covers the hazard.

He also proposes a lift condition this page's third question lacks a form for. The gate is a criterion, not a duration: "if we stop for two months and try to meet them and at the end of that period we still have not met the criteria then as far as I'm concerned we're still blocked." That ties the lift to evidence rather than to the calendar. He does not say who adjudicates whether the criteria are met, which leaves the who where Frontier AI Standards Body left it.

Visibility first: a 22-author agenda with lab insiders on it (September 2026)#

Every proposal above comes from one lab, one founder or one forecasting group. What if automating AI R&D triggers an intelligence explosion? (Chan, Mindermann and 20 co-authors, arXiv 2609.36054, 2026-09-28, practitioner-opinion) is the first in the corpus signed across institutions. The signatories include OpenAI's chief scientist Jakub Pachocki, Anthropic co-founder Jack Clark, Anthropic-affiliated economist Anton Korinek, Geoffrey Hinton, Yoshua Bengio and Microsoft's Eric Horvitz. Its growth argument is on Software-Driven Intelligence Explosion. Its policy section ranks visibility first, ahead of pacing, and that ordering is its contribution here.

1. Obtain visibility. "Current mandatory reporting frameworks either do not adequately cover internal AI R&D use cases or do not specify indicators to be reported". The paper names California's SB 53, the EU general-purpose AI code of practice and New York's RAISE Act. Its proposal is standardized reporting of AI-R&D indicators to governments and third-party auditors, covering three things: the likelihood of an explosion (bottlenecks and the spending split), its onset (extent of automation, pace of algorithmic progress), and oversight (internal-deployment decisions, incidents). Beyond reporting, it floats independent evaluation of systems before internal deployment, and auditors or supervisors embedded in companies, modelled on Nuclear Regulatory Commission resident inspectors and the Office of the Comptroller of the Currency. That is rung 4 of the AI Futures Project's ladder, with regulatory precedents attached. It scopes the heavier requirements to companies at the AI-R&D frontier or above a capability threshold, and it leaves the threshold open.

2. Steer and constrain. The list matches this page's subject:

• Requirements for continued development or deployment, including "limits on the extent to which capabilities can increase within a given time period". That is a rate cap, the family of AI Futures' 1%-per-month option.
• Tools to verify compliance with future pacing agreements.
• Data-center oversight, with incident-response procedures that include "options to pause specific AI R&D workloads". That makes a pause per workload rather than per company.
• Air-gapped environments for certain evaluations and deployments of automated AI-R&D systems.

On the international side it lists confidence-building measures, verification research, clarity on whether and how states would deter a rival's scale-up, and war games. Its verification and deterrence citations include the same group's AI 2040: Plan A (not in raw/).

What it adds to this page's questions. It names the cost the other proposals leave implicit: "poorly crafted mechanisms could allow a government to slow R&D at all but a favored company". A pause power is also a tool of selective industrial policy. On the first open question it moves the verification target from training runs to internal AI-R&D deployment and workloads. A software-driven explosion would run on compute already installed, so the hazardous activity may never look like a new training run. On adjudication it is silent, like the others. Weight: an agenda, not a design. Nothing is piloted, the thresholds are deferred, and senior staff of two of the labs to be regulated co-signed it. That co-signature is a signal about the labs' insiders, not evidence that any mechanism works.

What onset detection can deliver, in theory (October 2026). Li et al. (arXiv 2610.18445, practitioner-opinion, proofs and no data) take the onset indicator literally. Give an ideal monitor every improvement and its exact arrival time, plus the baseline rate and the size of the acceleration. Telling a self-accelerating loop from steady exponential growth at 5% error still takes 130 improvements if each is 1%, and 9 if each doubles capability. On Chan et al.'s calibration, the second case decides at the tenfold-pace point with about 1.7 months left. Real indicators are coarser: manuscripts, experiments and releases. By data processing they can only do worse. And if the acceleration size is unknown, no fixed-sample test has a uniform error bound. Visibility therefore buys warning time only when progress is fine-grained. The same paper indexes any response plan to the current doubling time rather than a calendar date. With doubling-sized steps, a twelve-month response needed with 95% probability must start while the expected doubling time is still at least 5.59 months.

Connections#

• Domestic Frontier Pacing — the corpus's first itemized verification menu (a five-rung auditor-access ladder ending in network taps) and its first quantified challenge to the unilateral-pause objection: ~4-8 months raw US lead, ~1 year including the distillation and algorithm-theft channels. Its structural claim splits this page's problem — embedded auditors suffice domestically, hardware verification is for treaties — and it still names nobody to adjudicate escalation

• Frontier AI Standards Body — a coordinated cross-lab slowdown proposed as an escalation setting on a national standards body, with none of the verification, trigger, or enforcement machinery this page argues a credible slowdown needs; also the first proposal to name an institution that would hold the adjudication mandate below, without saying who inside it decides

• Cross-Lab Pre-Release Review — the weaker, volunteer-access alternative that needs no verification infrastructure, and correspondingly cannot deliver a stop

• Stochastic Explosion Timing and Detection Limits: the statistical floor under the onset indicators. Even an ideal monitor needs a cube-root number of improvements to tell acceleration from steady growth, and for lumpy progress the decision comes near the end

• Software-Driven Intelligence Explosion: the growth case the visibility-first agenda is written for. It also explains why the hazardous activity may be internal AI-R&D workloads on installed compute rather than a detectable new training run

• Recursive Self-Improvement — the trajectory that makes a pause option worth building; this is its governance response

• Responsible Scaling Policy Evaluations — the single-lab deployment brake; pause verification is the multilateral counterpart

• AI Accelerating AI Development — the compounding-acceleration evidence that makes "we don't have decades" the operative constraint

• Agentic Misalignment (AM) — losing control is the downside a credible pause is meant to hedge against

• AGI-to-ASI Pathways — DeepMind's "deliberate slowdown" friction (friction #6) is this same coordination problem, and its "military–economic adaptationism / anarchy as architect" analysis is the structural reason verifiable multilateral coordination is so hard

• Open-Weight Elicitation Irreversibility — the blind spot in the verification frame: pausing observable training runs does nothing about unbounded inference on weights already published

• Balance-of-Power Superintelligence — the opposite governance pole: Zuckerberg's case that distribution to individuals, not coordinated slowdown machinery, is what makes superintelligence safe

• Government Checkpoint Sharing — the mechanism explicitly designed against this one: oversight with zero release latency, achieved by transferring capability to the government instead of giving it a stop

• Safety Commitments That Cannot Bind the Actor Who States Them — the conditional's structural consequence: a pause commitment gated on a regime that does not yet exist imposes no present cost, and the front-runner premise carrying that conditional is itself disputed in the corpus

• Silent Revision Rate — a measured, prior instance of the same self-administration problem: before any pause commitment is even tested, a developer's own account of how its existing safety framework changed is silent on two-thirds of material changes (12 developers) and silences weakenings roughly twice as often as strengthenings — the same asymmetry this page's conditional-pause posture would need external verification to catch

• Frontier-Lab Safety Culture — an insider's report that a publicly announced pause was true and narrowly scoped (RL training paused, pre-training not), and a criterion-gated lift rule ("still blocked" until the safety criteria are met, however long it takes) in place of a time-based one

Open Questions#

• What does an AI-training "verification regime" concretely consist of — compute-accounting, datacenter inspection, hardware attestation, on-chip telemetry? The essay names the problem, not the mechanism. Partially answered (2026-08-12): Domestic Frontier Pacing supplies the corpus's first itemized menu — a five-rung auditor-access ladder (public info → benchmark-level questions → employee-level system access → embedded in the company → direct compute-allocation audit via network taps), plus whistleblower protections, TEE attestations and zero-knowledge proofs as alternates, and a claim about which rung suffices where (embedded auditors domestically, hardware verification internationally). Partial on two counts: it is practitioner-opinion with nothing piloted, and it answers the compute-allocation verification question rather than the training-run detectability question this page opens with — an auditor inside the company does not solve detecting a training run you were never told about. Reframed (2026-10-15): Chan et al. (practitioner-opinion) point verification at internal AI-R&D deployment: pre-internal-deployment evaluation, embedded auditors on the NRC resident-inspector model, and data-center procedures able to pause specific AI-R&D workloads. On their software-only account the hazard runs on installed compute, so training-run detectability may be the wrong target. Nothing is specified beyond the list. Settler (2026-10-21): a pilot or published specification of an audit (rung, attestation protocol, miss rate) from a lab, regulator or standards body.
• Detectability < verifiability: can detection even be made reliable when training runs leave no physical signature and inputs are dual-use? Partially answered (2026-10-21): How to pace the US frontier sidesteps training-run detection by making compute-allocation shares and capability measures the audited quantities, with auditors inside the company, and says sandbagging on AI-R&D evals "seems likely to be detectable" by fine-tuning plus automated auditors. That is a design assertion with no test; whether dual-use inputs and signature-free runs can be detected at all stays open. Settler: a red-team of an allocation audit against a lab hiding a training run.
• Who adjudicates triggers and lifts? No institution currently holds that mandate, and standing one up is itself a decade-scale task. Partially answered (2026-08-12) — a candidate institution, and the same gap one level down: Hassabis's Standards Body is the first proposal to name a body that would hold the mandate, with a stated escalation power to coordinate a cross-lab slowdown. It then reproduces the question inside itself: the essay never says who declares the assessment protocol "effective and robust," who deems a slowdown necessary, or what enforces one. So the answer moved from "no institution holds it" to "one has been proposed to hold it, and the proposal does not say who inside it decides" — which is progress on the where and none on the who. practitioner-opinion, unimplemented. A fourth proposal three weeks later (Domestic Frontier Pacing) leaves the who exactly where it was: it assumes the US government enforces, assigns escalation between its four options to nobody, and specifies dispute machinery only for an auditor's redaction decisions.

Sources#

• When AI builds itself — §"What should we do?" (verifiable multilateral pause; detectability vs verifiability; INF Treaty precedent; Anthropic Institute convenings)
• A Framework for Frontier AI and the Dawning of a New Age — Demis Hassabis, 2026-07-14 (practitioner-opinion): the escalation clause ("ratcheted up… including coordinating a slowdown in development among the Frontier Labs if deemed necessary") and the international-template framing. Full treatment at Frontier AI Standards Body
• How to pace the US frontier — AI Futures Project (Lifland, Halstead, Dean, Larsen, Kodama), 2026-08-05 (practitioner-opinion): §"Auditor access" (the five-rung ladder), §"How to prepare to pace the frontier" (TEE attestations, zero-knowledge proofs, network taps, dark compute), and §"But wouldn't domestic pacing let China win?" (the 4-8 month raw lead and the ~1-year catch-up estimate). Full treatment at Domestic Frontier Pacing
• Silent Revision: Measuring Undisclosed Change in the Safety Frameworks of Frontier AI Developers — Louis Yiven Zhu (Oxford), arXiv 2609.08789, 2026-09-08 (empirical): cited here only for the self-administration parallel (a developer's account of its own framework's revision, unverified); full treatment at Silent Revision Rate
• What This OpenAI Insider Saw That Made Him Quit | The Ezra Klein Show — Ezra Klein with David Robinson (OpenAI system-card lead until his October 2026 resignation), The Ezra Klein Show, 2026-10-07 (practitioner-opinion, YouTube auto-captions with ingester-assigned speaker labels). COI: Robinson holds OpenAI equity; the host's employer is suing OpenAI. Cited for the RL-paused/pre-training-not account and the criterion-gated lift rule. The pause announcement itself is not in this corpus. Full treatment on Frontier-Lab Safety Culture
• What if automating AI R&D triggers an intelligence explosion?: Chan, Winter, Barto, Pachocki, Hinton, Horvitz, Bengio, Song, Clark, Greaves, Korinek, Hammond, Graepel, Bariach, Torr, McIlraith, Clune, Manning, Sastry, Davidson, Eth & Mindermann, arXiv 2609.36054, 2026-09-28 (practitioner-opinion). §"Policy implications" and its three subsections, plus Figure 1 (a schematic of the three priorities, with no data). COI: senior OpenAI and Anthropic staff among the authors, proposing regulation of their own employers
• The Singularity Has a Distribution: Exact Timing Laws and Detection Limits for a Stochastic Intelligence Explosion: Li, Gao, Yang & Sha, arXiv 2610.18445, 2026-10-13 (practitioner-opinion, theory with proofs, no data). Cited for Theorem 5.1, Remarks 4.3 and 5.4 and Table 3, against the onset-detection indicator. Full treatment on Stochastic Explosion Timing and Detection Limits