Anthropic Institute

Sources#

• When AI builds itself

Summary#

The Anthropic Institute is Anthropic's research and policy arm focused on the societal and governance implications of frontier AI. It published When AI builds itself (June 2026) — this wiki's primary source on Recursive Self-Improvement — and has a stated agenda to build, in collaboration with others, the systems that a credible AI slowdown or pause would require (Frontier Pause Verification).

What it does#

• Public-facing trajectory analysis. When AI builds itself combines public benchmarks (Task Time-Horizon Scaling) with previously-unreported internal Anthropic data (AI Accelerating AI Development) to argue AI is already accelerating AI development and to lay out three futures for RSI.
• Coordination infrastructure. It plans to "conduct research — in collaboration with many others — and take actions to help build the systems that a credible slowdown or pause would require": verification that other developers have actually stopped, and that a bad actor cannot exploit a coordinated slowdown to jump ahead in secret (Frontier Pause Verification).
• Convening. In the months after the essay, the Institute plans to organize conversations among policymakers, researchers, civil society, and other AI companies, and to publish the results — explicitly inviting voices outside AI companies into the deliberation.

People#

• Marina Favaro and Jack Clark co-authored When AI builds itself (editorial support from Santi Ruiz; visuals by Shan Carter, Romello Goodman, Nikki Makagiansar from data by Brian Calvert and Jun Shern Chan).

Connections#

• Anthropic — parent organization

• Recursive Self-Improvement — the subject of the Institute's flagship essay

• Frontier Pause Verification — the Institute's concrete governance agenda

• AI Accelerating AI Development — the internal evidence base the essay draws on

• Responsible Scaling Policy Evaluations — the Institute's external-coordination work complements Anthropic's internal RSP brake

• Safety Commitments That Cannot Bind the Actor Who States Them — why the option-to-pause framing costs nothing today: the commitment is conditioned on a verification regime the Institute is itself still building, while the part of Anthropic that can bind now (the RSP) is self-administered

Open Questions#

• How does the Institute's policy posture (favoring an option to pause) interact with Anthropic's commercial incentive to ship frontier models? The essay acknowledges the competitive/geopolitical pressure but doesn't resolve it. Partially answered (2026-08-19): Safety Commitments That Cannot Bind the Actor Who States Them settles the shape of the interaction without settling motive. The pause commitment is conditioned on a verifiable multilateral regime that does not exist and that the Institute is itself still building, so it imposes no present cost; the part of Anthropic that can bind today is the RSP, which is self-administered, bound once at a moment of its own choosing (Mythos Preview withheld until Fable 5's safeguards existed), bent toward shipping at its two closest calls (the Opus 5 CB-2 determination, the dropped rule-out suite), and has published a forecast of crossing CB-2 before its own recommended security bar exists. The load-bearing front-runner premise is itself disputed in the corpus by Domestic Frontier Pacing's ~1-year catch-up estimate. Still open, and unanswerable from this corpus: whether the conditional shape is chosen because the argument is right or because it is convenient — every observation above is Anthropic assessing Anthropic.
• What concrete verification mechanisms will the Institute prototype, and on what timeline relative to the RSI trend it warns about?

Sources#

• When AI builds itself — Anthropic Institute, When AI builds itself (Marina Favaro & Jack Clark, June 2026)