Emmanuel Ameisen

Summary#

Entity. Researcher on Anthropic's interpretability team, which sits inside its alignment organization. Before Anthropic he worked in applied machine learning. Across the vault's raws he appears as:

• Lead author of the circuit-tracing methods paper (Ameisen et al. 2025, attribution graphs). Anthropic's later work cites its ERA/TWERA coactivation metrics as the effectiveness proxies that Fisher effectiveness replaced (Characterizing interference weights in a tiny language model).
• Co-author of the global-workspace paper. He did the mechanistic localization with Adam Pearce and gave regular feedback on the project (Verbalizable Representations Form a Global Workspace in Language Models, author contributions). See The Global Workspace in Language Models (J-space).
• Co-author of Mechanisms of introspective awareness (Macar, …, Ameisen, Lindsey, arXiv 2603.21396, 2026), which Introspective Coupling: Self-Explanation Training Tracks Behavioral Change Despite Fixed Supervision cites. He is also on the author lists of Marks et al.'s Auditing language models for hidden objectives (2025), the reward-model-appeaser organism on Internal Signatures of Misalignment, and of the 2026 natural-language-autoencoder paper (both reference lists are in Introspective Coupling: Self-Explanation Training Tracks Behavioral Change Despite Fixed Supervision and The Model Organism Lottery: Model Organism Interpretability Strongly Depends on Training Methodology).

The public account (Lenny's Podcast, October 2026)#

A 72-minute general-audience interview (Inside the black box of AI: How AI models actually think | Emmanuel Ameisen (Anthropic), practitioner-opinion, first-party). It is the vault's only source in which a member of the team explains the programme in its own words rather than in a paper. The positions he takes:

• "To make them is not to understand them." Models are grown, not written. Training fixes the architecture, which works like a scaffold, and the formulas inside are learned. So the work "feels so much closer to biology than to engineering": you get a system and are asked why it does something, without readable source code.
• Next-token prediction is deceptively hard at the top end. Text has easy, medium and hard words to predict. Once a model gets the easy ones (syntax) and the medium ones (facts), what is left is the last paragraph of a PhD thesis. He dates LMs beating humans at next-word prediction to "around 2019". The capability argument built on this is on Jagged Intelligence (Ghosts, Not Animals).
• Shared concepts and planning. He gives the two-hop Dallas → Texas → Austin example. The same "eye" direction fires on a photo, on text in any language and on an SVG drawing. He describes poetry planning (picking the rhyme word "rabbit" before writing the line) as settled. The vault records that the J-lens version of the poetry result failed to replicate (The Global Workspace in Language Models (J-space)).
• Steering as validation. Golden Gate Claude was a deliberate experiment. Clamping the decoded feature on everywhere and seeing the model behave accordingly "is also one way that you convince yourself that you found the right thing."
• What interpretability is for in alignment (characterize propensities; catch what is never said), plus the answer-thrashing anecdote. See Internal Signatures of Misalignment.
• Anthropomorphism is the useful frame. This runs against Andrej Karpathy's "treat them as tools" on Jagged Intelligence (Ghosts, Not Animals).
• A plausibility check separate from the computation. A model flags "4 + 5 = 23" as wrong without doing the sum. See Automatic vs. Flexible Cognition in LLMs.
• Consciousness: "very hard to say one way or another." Interpretability is the less biased route because it does not depend on what the model says. See Access-Consciousness Indicators in AI.
• Convergent internals. Models trained "years apart on … different data, by different people" end up with surprisingly similar internal structure. His explanation is that some problems have only one efficient solution. This is an unmeasured assertion in the talk.
• Optimism with a push-pull. Better models are easier to study, because their world model is clearer, but they are also better at evading basic security measures. He puts the field at "hundreds, not thousands" of people and says it is open to self-study.

Attribution note: the transcript is auto-captioned and its turns are unlabelled. Several framings are the host's, not Ameisen's: "it's like we've created this new life", "we're basically next token predictors ourselves", "slow takeoff", "J space", and the claim that Anthropic is criticized unfairly. The model the host calls "Jev" and the site given as "interpedia.org" are likely ASR errors and are not compiled.

Connections#

• The Global Workspace in Language Models (J-space) — co-author (mechanistic localization); in public he restates the planning result without its replication status
• Internal Signatures of Misalignment — his first-hand account of the answer-thrashing investigation and of the two jobs interpretability does in alignment
• Jagged Intelligence (Ghosts, Not Animals) — he gives the counter-position to "ghosts, treat as tools", and a smooth-loss / per-task-threshold account of capability jumps
• Interference Weights — his circuit-tracing metrics are the baseline that work replaced
• Jack Lindsey — co-author on the workspace and introspection papers
• Anthropic — interpretability team, inside alignment

Sources#

• Inside the black box of AI: How AI models actually think | Emmanuel Ameisen (Anthropic) — Lenny's Podcast, 2026-10-18, auto-captions with unlabelled turns (practitioner-opinion)
• Verbalizable Representations Form a Global Workspace in Language Models — author list; Author Contributions (mechanistic localization; regular feedback)
• Characterizing interference weights in a tiny language model — cites "ameisen2025circuit" (circuit-tracing methods; ERA/TWERA)
• Introspective Coupling: Self-Explanation Training Tracks Behavioral Change Despite Fixed Supervision — reference list (Mechanisms of introspective awareness, 2026; the NLA paper)
• The Model Organism Lottery: Model Organism Interpretability Strongly Depends on Training Methodology — reference list (Marks et al. 2025, Auditing language models for hidden objectives)