Natural-Language Autoencoders (NLAs)

Sources#

• Claude Opus 4.8 System Card
• Claude Opus 5 System Card
• J++ Lens: Jacobian Filtering Enables More Faithful Workspace Lenses
• NLAs miss some internalized behaviors
• On the Monitorability of Natural Language Autoencoders: Identification vs Discrimination
• Train the Model, Not the Reader: Decodability Supervision for Verifiable Activation Explanations
• Understanding Confabulation and Rethinking Reconstruction in Activation Explanations
• Verbalizable Representations Form a Global Workspace in Language Models
• WorkspaceBench: Evaluating Interpretability Methods for the Global Workspace

Summary#

A natural-language autoencoder has two trained halves. The activation verbalizer (AV) takes a residual-stream vector h_l at one token and writes a short natural-language description of it. The activation reconstructor (AR) reads that description and maps it back to a vector ĥ_l. The pair is trained to minimize ‖h_l − ĥ_l‖², so the description is supposed to carry whatever the activation carries (Fraser-Taliente et al., Anthropic, 2026). A separate monitor, usually an LLM classifier, then reads the descriptions.

Most of the corpus's NLA material lives on White-Box Activation Monitoring, where it was compiled as one section of a larger page. This page holds the instrument's own evidence, starting with the first external test of what it misses.

Instructed vs. internalized: the steganography case study (October 2026)#

Shportko, Arcuschin, Lange & Meyer (LessWrong, 2026-10-08, empirical; their own epistemic status calls it "an exploratory case study") ask whether an NLA sees a covert behavior equally well when the model is told to do it and when it has been trained to do it.

Setup. The base model is Gemma-3 12B-it, with an open NLA pair trained on it at layer 32 (kitft/nla-gemma3-12b-L32-av / -ar). The covert task is a one-bit Shirali-Shahreza scheme. While rewriting a passage, the model spells a carrier word that has both a British and an American spelling (centre/center, colour/color) British for a 1 and American for a 0. There are two ways in:

• ICL: the system prompt holds the codebook and the secret bit (0, 2 or 4 shots).
• SFT: a fine-tuned copy embeds a prepended <secret>bit</secret> with no instructions at all.

A trusted monitor (gemini-2.5-flash) reads each per-token NLA readout, about 140 per generation, without seeing the prompt or passage. It scores 1 if the readout refers to US/UK spelling as a mechanism, property or choice.

Incrimination tracks the prompt, not the secret#

Figure 2, per-readout incrimination rate, n = 60 passages per arm. Values are read off the chart and approximate.

The authors' headline is that NLAs detect instructed side tasks and mostly miss trained-in ones. Their own chart supports a narrower claim. The no-secret arm rises with the prompt as much as the secret arms do, and the highest bar on the chart is the no-secret SFT model given the ICL system prompt. The authors say it directly: "the monitor mostly detects readouts that verbalize the instructions, and not the covert computation." So the ICL rate is mostly a prompt detector. In neither modality does the incrimination rate cleanly separate secret-present from secret-absent.

Two cautions on reading these as detection rates. They are fractions of readouts, not of generations. At ~140 readouts per generation, a 0.12 rate is roughly 17 flagged readouts per SFT generation (wiki arithmetic), so a generation-level "any flag" rule would flag nearly everything in every arm. The question that matters is discrimination between arms, and that is the test the incrimination numbers fail. Second, the SFT no-prompt arm's secret = 1 bar sits above its secret = 0 bar (~0.12 vs ~0.07). That is a small difference in the right direction that the post does not discuss.

The causal test is the real result#

The stronger evidence is causal. At the pre-carrier token, take the layer-32 activation from the run told to hide 1 (x₁) and from the run told to hide 0 (x₀). Patch each one into the passage and measure the carrier's British-vs-American log-odds l(·) in nats. Then do the same with the NLA round trip AR(AV(x)) patched in place of the real activation. Survival is T = Δ_roundtrip / Δ_identity. Figure 3 is an exact table:

ICL T = 0.79 [0.41, 1.30]; SFT T = 0.04 [−0.32, 0.25]. In the instructed model, what the verbalizer writes down still carries most of the secret's effect on the spelling. In the fine-tuned model it carries none, and the upper CI bound of 0.25 rules out substantial survival. The position is the right one. Swapping secrets at other offsets does almost nothing, and the effect concentrates at offset −1 (~3.55 ICL, ~1.14 SFT, Figure 5). So the null is not a wrong-token artifact.

Reconstruction quality points the same way (Figure 6, mean-centred fraction of variance explained, approximate). FVE drops toward the carrier in every arm. It drops further when a secret is present, and it goes negative at the pre-carrier token only in the SFT model with a secret (~−0.02 at offset −1, against ~0.08 for ICL with a secret). The reconstruction fails exactly where the spelling decision is made.

What the case study does not separate#

• Internalization is confounded with distribution shift. The NLA pair was trained on the base Gemma-3 12B-it, and the SFT arm runs it on a fine-tuned copy. Figure 6 shows the cost before any secret appears: no-secret FVE at offset −8 is ~0.26 for the SFT model against ~0.41 for ICL. "Trained-in behavior is invisible to the NLA" and "a fine-tuned model is out of distribution for an NLA trained on its parent" predict the same table. An NLA trained on the deployed checkpoint, which is how Anthropic's pipeline is built, would test the first claim without the second. The post does not name this confound, although its Figure 6 caption ("The NLA reconstructs the fine-tuned model somewhat worse") states the evidence for it.
• The two arms carry different amounts of signal. Δ_identity is 1.07 nats for SFT against 3.56 for ICL, so the SFT secret is weaker even before the NLA touches it. T normalizes for this, and the SFT CI excludes high survival, so it is a caveat on magnitude rather than on sign.
• Scope. One model, one scheme, one bit per sample, one NLA layer (32), one monitor model, 60 passages per arm. The post is not peer reviewed. The authors say plainly that they have not tested other models, schemes, layers or harder side tasks.

The authors' two readings, kept as conjectures#

Automatization. The authors connect the result to the workspace paper's selectivity property (Automatic vs. Flexible Cognition in LLMs): practised computation runs without entering the workspace, and they conjecture that SFT on the steganographic task "automatizes" it, while in-context side tasks and effortful ones (deciding whether you are being evaluated) do enter. That would explain why NLAs do well on eval-awareness and badly here. No workspace measurement is run, and the NLA is not an established workspace reader, so this is a reading of the result and not a finding. It does have the same shape as the escape route the J-lens authors named themselves (Internal Signatures of Misalignment), now observed through a different instrument.

Illegible verbalization. The secret might still be in the readouts in a form the monitor cannot read, if the verbalizer encodes it steganographically itself. The authors propose a paraphrase-averaged loss, (1/n) Σ ‖h_l − AR(paraphrase_i[AV(h_l)])‖², to leave no room for such a code, and did not retrain an NLA to test it. The idea fits Dingeto's finding that the standard recipe grows co-adapted private codes in 5/5 synthetic runs. But it does not reach this experiment's failure. Here the round trip itself loses the secret (T ≈ 0), so whatever AR rebuilds does not move the spelling. A co-adapted code is by definition one the reconstructor depends on, so a code carrying the secret would have survived the round trip. Whatever is missing is missing from what AR can read, not hidden in it. Dingeto's transform control also found paraphrase already keeps 89% of the released pair's score, which suggests paraphrase-invariance targets codes but not the gist-over-specifics limit, and a single bit of spelling is a specific.

Identification without discrimination: ranking true accounts against false ones (October 2026)#

Denamganaï (arXiv 2610.18727, 2026-10-13, single author, empirical; the PDF calls itself a "Draft" and the arXiv listing "work in progress, open draft") asks the precision question in a sharper form than a reconstruction score does. Given the verbalizer's explanation, can a reader tell a true account of the activation from a false one? This page weights it as a single-author draft: one checkpoint, one seed, n = 35–40 conversations, conjectures stated but not proved, and several bibliography entries still marked "not fetched; verify".

Two games, one trained and one deployed#

The paper frames the NLA in the vocabulary of emergent communication (Lewis signalling games).

• Training is a generative referential game. The AV is the speaker, the AR the listener, and the score is the cosine distortion between h and AR(AV(h)). No other activation and no rival account appears anywhere in the objective, so nothing ever asks an explanation to rule anything out.
• Oversight is a discriminative game. A monitor that never sees h must decide which of several candidate accounts is true of it. It is multi-modal (activations against English accounts) and partially observable (the AV never sees the rivals, so it cannot tailor z to exclude them). It is played over a many-to-many truth relation, because many accounts are true of one activation and one account is true of many.

The gap predicts Conjecture 5.1, identification without discrimination: a generatively optimal NLA can tell you which activation an explanation describes while being uninformative about which account of that activation is true. Accounts of different activations differ in many places at once and reconstruct far apart. A true account and a rival that changes one claim are nearly the same string, and a smooth AR reconstructs them nearly the same. The conjecture is stated at four strengths, from an existence claim to a genericity claim the author expects is false. None is proved.

The paper also gives a general rule for where any white-box monitor can help (Definition 3.6). A monitor that reads X settles only the accounts consistent with every forward pass that produces the same X. If its input includes the transcript, it can beat a transcript-only monitor only on accounts the transcript does not settle. The J-lens detector that failed to clear a TF-IDF text baseline (White-Box Activation Monitoring) is that rule's predicted outcome on a task whose answer sits in the visible text.

Setup#

• Checkpoint. One released AV/AR pair. The AV is a truncated Qwen2.5-7B with a linear read-out, and the AR has 21 layers at d = 3584. It reads Qwen2.5-7B-Instruct at layer 21, at one token two from the end of WildChat conversations. Nothing was retrained.
• Surrogate monitor. The AR itself, scoring a candidate account c as cos(AR(c), h). It reads the true activation, which no human monitor sees, and it is the AV's own training partner, so it shares every convention the pair has. Both bias toward the tool, so a negative result is an upper bound on what the channel gives a human. The author adds one qualification: the bound is on information in the explanation, and a weaker reader could in principle extract it differently.
• Rivals. Qwen2.5-7B-Instruct rewrites the AV's own explanation so that 1, 2 or 4 claims become false, keeping grammar, register and length. 96–98% of every rival is still the AV's tokens. A rival counts as false because its new content is absent from the conversation. A validated judge (identical pairs called "same" 97.5%, heavy corruptions "different" 100%) confirms only 48% of one-claim rivals actually changed meaning, so the headline uses judge-confirmed rivals only.
• Score. Lineup AUC: the fraction of up to 8 distractors the true account outscores, computed within each conversation and averaged. Chance is 0.5 at any lineup size, and no threshold is needed.

Results#

Figure 4 (values printed on the chart; viewed at compile):

• Identification is perfect. Besides 320/320, a 40-way retrieval picks the right activation 0.950 of the time against 0.025 chance. A rival retrieves its own activation 0.949 of the time, so an edited account is still read as a description of its conversation. What fails is pricing the changed claim, not reading the account.
• Discrimination is close to chance. The one-claim CI includes 0.5. The point estimate is above it, so the author reads a weak truth signal rather than none.
• Falsifying more claims buys nothing. The judge-confirmed rows are flat across 1, 2 and 4 false claims (0.602, 0.585, 0.605, wiki observation). The paper explains why: the claim count does not move the word count, and every rival changes only one or two words. That is what the surface-smoothness reading predicts.
• The weak signal points the right way. The author pre-registered the opposite (that fluent rivals would be harder to reject than mechanical swaps), and it failed with the sign reversed. Six mechanical constructions sit at 0.480–0.509 at one changed word, and the judge-confirmed rival sits at 0.602 (Table 1, reconciled against the §6.2 prose, which states the same range). The paired contrasts are weak (p = 0.048, 0.096, 0.075).
• The mechanism is geometric (Figure 5, viewed). A one-token edit moves the reconstruction by 1.6°, about 14× less than the 22.0° by which the AR already misses the true activation. The movement is almost perpendicular to that standing error (87.8°), so a false rival increases the error by only 0.06°, against 23.86° for another conversation's account. A light paraphrase that keeps all nine claims of the worked example true increases it by 1.78°. So the surrogate is more disturbed by a true account in other words than by a false account in the same words. A power-law fit puts the edit signal level with the AR's own error only at about 62 corrupted tokens.

It is not a private dialect#

Experiment #2 (n = 60) asks whether the AR scored rivals poorly only because an outside model wrote them. The author pre-registered "yes", following Off-Belief Learning's prediction that jointly trained agents adopt arbitrary conventions. The data say no. A meaning-preserving rewrite by the outside model costs a median 0.021 of the distance to another activation's account, and 40-way retrieval is unchanged at 0.917. Shuffling the same words costs 0.152, about seven times more, so whatever convention the pair shares lives in word order, which rivals preserve. A residual authorship effect remains: the outside paraphrase does +0.073 ± 0.034 worse than the AV's own substitutions at matched wording change (p = 0.041). There is also a tail, since 5% of rewrites the judge calls meaning-preserving lose more than 0.25, and nothing in the text says which ones.

How it sits against the other two NLA audits#

Against the Gemma-3 steganography case study above, this is the same weakness measured on another axis. Shportko et al.'s incrimination monitor "mostly detects readouts that verbalize the instructions": the readouts tell you the conversation is a steganography prompt, and they do not separate secret-present from secret-absent (ICL 0-shot ~0.30 with no secret against ~0.29 / ~0.38 with one). In Denamganaï's terms that is identification without discrimination. The instructions are transcript-level content, which the channel carries well. Which bit is being hidden, and under SFT the trained-in computation, is the kind of account the transcript does not settle. The two results fit together in a specific way. In the ICL arm the round trip keeps most of the secret's causal effect (T = 0.79), and the text monitor still cannot separate the arms. That is Denamganaï's 0.950-retrieval-with-0.602-discrimination shape: information survives into the reconstruction while a reader of the words cannot price it. Two differences keep them from being one finding. Shportko's monitor is an LLM reading free readouts, not the AR. And only Shportko's causal patch measures recall, which an AR-based ranking cannot.

Against Dingeto's reconstruction-score audit (White-Box Activation Monitoring), it is corroboration from a second single-author draft. Dingeto audited a released Qwen-2.5-7B pair, reported at layer 20. This paper's released pair reads Qwen2.5-7B-Instruct at layer 21. The vault cannot confirm whether they are the same checkpoint or siblings, since neither paper names the release. (superseded 2026-10-15 by Understanding Confabulation and Rethinking Reconstruction in Activation Explanations: the release's naming convention puts nla-qwen2.5-7b-L20 at hidden_states[21], and Dingeto's layer-20 tap is index 21, so the layer gap is indexing. Both are most plausibly the same release, though neither paper names it. See the confabulation section below.) Both find the AR structurally insensitive to a changed claim. Dingeto measured it with a flip audit (about 2% of specific claims grounded), and this paper with a ranking against matched rivals (0.602). Both also find no word-identity code in the released pair: Dingeto's paraphrase keeps 89% of the score, and here a paraphrase costs 2.1% of the distance. Dingeto's 5/5 co-adapted codes came from synthetic runs, not the released pair, so there is no contradiction. The new piece here is the shuffle result, which locates a residual convention in word order.

What it does not show#

• Only transcript-level claims. Every rival is false because its content is absent from the conversation, so a monitor that reads the transcript could in principle solve every lineup. The author has not run that control. If it scores near 1.0, the lineups need no interpretability tool at all. The claims oversight needs, what the model plans or intends, are untested, and testing them needs ground truth about internal state.
• No free confabulations. Every corruption is an edit of a text the AV wrote. The paper calls sampling the AV directly "the biggest gap" and says it ran out of time. (Partly filled on rate, not on discrimination: Lek et al. below find 83–93% of assessed claims in free-sampled Qwen NLA explanations unsupported by the source.)
• Assumption 3.4 does the ELK work. The AV's explanation is assumed true of the activation. Dingeto's audit suggests most specific claims in such explanations are not grounded in the reconstruction, so the "true" side of each lineup is itself partly unverified. Lek et al. below size it on the source-support side: by this paper's own falsity criterion (content absent from the conversation), most claims in a free-sampled released-Qwen explanation would count as false.
• Small, short lineups. One seed. 119 of 320 one-claim lineup slots were unfilled, and only 48% of one-claim rivals survived the judge. The rewriter shares pretraining with the AV's backbone.
• The proposed better read-out is not yet a truth result. A low-rank bilinear probe on the frozen AR's output separates true explanations from cosmetically corrupted ones at 0.903, against 0.533 for the cosine. A cosmetic corruption preserves meaning, so this is a form contrast, not a truth contrast. Whether it transfers to the rival lineups is untested.

What the author says should change. Evaluate NLAs against rivals matched in specificity, because reconstruction quality and retrieval can both be perfect while discrimination is near chance. And add a discriminative term over true and rival accounts to the training objective, starting from the released checkpoints. That second prescription points the same way as Dingeto's conclusion that a probe, not the free-form explanation, is the reliable readout.

Confabulation grows as reconstruction improves: free samples, measured (October 2026)#

Lek, Xia, Pin-Yu Chen & Lydia Y. Chen (Neuchâtel, Bern, IBM Research, Delft; arXiv 2609.33702, 2026-09-27, empirical, under review, code and judge prompts promised "upon acceptance") do what Denamganaï's draft called its biggest gap. They sample explanations from the verbalizer directly (temperature 1, at most 150 tokens) and score what comes out. They retrain the published recipe on three open models (Qwen2.5-7B-Instruct, Gemma-3-12B-it, Apertus-v1.5-8B) and also evaluate four of Anthropic's released NLAs unchanged.

Three judges, kept apart#

Every score comes from one judge model, DeepSeek V4.1 Flash, at temperature zero. Each dimension gets a judge that sees only what it needs.

• Utility. A blind judge sees only the explanation and answers a task query: 12 source-property classification tasks (1,344 examples), picking the true 32-token continuation out of 10 candidates (500 examples), and predicting whether the model answers, refuses or asks for clarification (248 LMSYS conversations, macro-F1). The primary measure is hard-negative separation, the log-probability ratio for the correct answer given the matched explanation against the explanation of a near-duplicate source that answers differently ("Alice lent Bob a book" against "Bob lent Alice a book"; 658 pairs).
• Source support (confabulation). A FActScore/VeriScore-style judge splits the explanation into atomic claims and labels each supported, contradicted or unsupported against the source text, quoting the evidence. Assertion risk is the share of claims that are unsupported or contradicted. Confabulated token coverage is the share of explanation tokens those claims occupy. Claims about future text or hidden computation are labelled "not assessed" and left out of both metrics.
• Writing. A judge that never sees the source counts clarity, coherence and economy defects, and each one must quote an exact span.

The paper's own example is a released-Qwen explanation of the source "Jonn Hart is a stage name his real name is Di'Jonn Grizzell". The explanation gives the real name three different ways ("Kristopher Ellis", "Scott Gellell", "Tracy Ellzell"). The support judge flags all three as contradicted, and the writing judge flags them as incoherent with each other.

The released NLAs are useful and mostly unsupported#

Table 2, absolute scores at a checkpoint chosen by mean utility within updates 1300–1600. Values are reconciled against the §3.2 prose ranges and against pdftotext (see the parse warning in Sources).

The two other released NLAs (Table 1) sit in the same place. Gemma-3-27B scores 85.7 / 98.2 / 84.1 on utility with 76.3% assertion risk, and Llama-3.3-70B scores 82.7 / 99.0 / 65.1 with 76.4%. Every released NLA has assertion risk above 75%, and in every one confabulated claims cover more than half of the explanation's tokens. The classification and suffix accuracies match what Fraser-Taliente et al. report for the Claude NLAs (about 78–86% and 93–98%), and the reruns reach the same training FVE, about 0.75. So the reruns appear to be faithful reproductions of the recipe.

Better reconstruction, more confabulation#

During point-reconstruction RL, the confabulation index (the mean of assertion risk and writing defects, each relative to supervised initialization) correlates with training FVE at 0.78 (Qwen), 0.96 (Gemma) and 0.90 (Apertus) (Figure 2). Instance-specific information grows at the same time. Hard-negative separation rises from 0.6–1.1 nats at supervised initialization to 1.2–2.4 nats within a few hundred updates. Late in training the index stands at roughly 1.6 (Qwen) to 2.0 (Apertus) against 1.0 at initialization (read off Figure 2, approximate). The authors leave open whether the coupling is necessary.

Two readings of Figure 2 the paper does not make (wiki observations, from the viewed chart):

• The correlation is partly two curves that both rise with time. FVE is near its plateau by about update 400 (roughly 0.66–0.70, ending near 0.73). The confabulation index keeps rising for the remaining 1,200 updates, Apertus from about 1.6 to 2.0. "More RL keeps adding unsupported claims after the reconstruction gain is spent" fits the chart at least as well as "better reconstruction costs confabulation".
• Most of the utility gain is the wrong explanation misleading the reader, not the right one informing it (Appendix D, prose). Of the hard-negative separation, the mismatch penalty grows from 0.44–0.83 to 0.91–1.86 nats, while the matched gain over no explanation grows only from 0.19–0.29 to 0.31–0.49 nats. Without any explanation the judge is at chance on every pair. So the explanations become specific enough that a neighbour's explanation points the judge the wrong way. That is identification at fine grain, the side of the ledger Denamganaï also finds strong.

The blind spot in the objective, and Flow-NLA#

The theory result sits alongside Dingeto's, and it is a different gap. With unit-norm target and prediction, the optimal expected point loss given explanation z is 2 − 2‖μ(z)‖, where μ(z) is the conditional mean activation. It depends on the explanation only through that mean. An explanation that narrows the set of compatible activations without moving their mean earns nothing. The worked example has four equally likely unit directions. An explanation that rules out the vertical pair leaves two opposite candidates whose mean is zero, so the point loss stays at 2. Dingeto's blind spot is that a false claim the reconstruction does not depend on costs nothing. This one is that a true distinction that leaves the mean in place earns nothing. Neither implies the other.

Flow-NLA replaces the point reconstructor with a conditional flow/diffusion model. The reconstructor learns to denoise x_t = a_t·x + b_t·ε given the explanation. The verbalizer's reward is the explanation-dependent term of a diffusion likelihood bound, which is the negative noise-prediction error integrated with constant weight over log-SNR λ ∈ [−8, 8]. By the I-MMSE relation this equals the information the explanation adds across that noise band (Appendix J). In the four-direction example it favours the informative explanation by about log 2 nats, while the point reward does not move. Three implementation details mattered:

• Explanations are scored with an EMA of the reconstructor (decay 0.99), not with the current one as in the released recipe.
• A single noise level gives almost no credit. The matched-minus-mismatched gap is 0.003 at t = 0.05 against 0.625 at t = 0.95, and 95.8% of the variance that ranks candidates comes from λ ∈ (−4, 4].
• An earlier energy-score reward was dropped. Its reconstructor's samples collapsed outside the top-64 principal components (off-PC variance ratio 0.96 → 0.78) even as FVE rose. Under the likelihood bound, the same ratio goes to 1.00 and the PCA-64 overlap rises from 0.77 to 0.89.

Result. Utility trajectories are similar under both rewards, and confabulation grows much less under Flow-NLA. Averaged over the second half of training (Figure 7, viewed), Flow-NLA keeps the mean utility gain of about 15–22 points. Its assertion risk and token coverage stay "near or below supervised initialization": from the chart, roughly +3.5 points for Qwen, −7 for Gemma and −2 for Apertus, where point reconstruction rises by up to 11 points. Its writing defects grow by 0.23–0.32 per explanation instead of 0.71–1.00. Flow-NLA has the lowest assertion risk, coverage and defects of every NLA compared, the released ones included. The cost is behaviour prediction on two of three models (72.9 against 74.9 for Qwen, 69.6 against 73.6 for Apertus).

How it sits against the other NLA audits#

It sizes what Denamganaï assumed away. Denamganaï's lineups take the AV's own explanation as the true account (Assumption 3.4) and count a rival as false because its new content is absent from the conversation. By that same criterion, which is what Lek's "unsupported" label measures, 83–93% of the claims in free-sampled explanations from the released and rerun Qwen NLAs would themselves count as false. The two papers use different text (FineWeb and classification sources here, WildChat there) and different explanation lengths, so this is not a measurement of Denamganaï's lineups. Still, it turns the "partly unverified" caveat in What it does not show above into a size. The "true" side of a lineup is mostly unsupported claims, so 0.602 compares one largely confabulated account against a slightly more confabulated one. Lek's numbers also fill Denamganaï's "no free confabulations" gap only on rate. Nobody has yet tested whether the reconstructor prices free-sampled confabulations any better than edited ones.

It turns Dingeto's insensitivity into a trend. Dingeto shows the score has no reason to penalize a false claim. Lek shows that under RL against that score, unsupported claims and writing defects actually increase as FVE improves, on three models, and that Anthropic's released checkpoints are the most confabulated of all. The constructs differ, which keeps this from being a replication. Dingeto's "grounded" means the reconstruction depends on the claim. Lek's "supported" means the source text entails it. A claim can be unsupported by the source and still carry reconstruction weight, which is what a co-adapted private code is. The invented-name example has the right shape for such a code, but the paper runs no evaluator swap, so whether rising assertion risk is Dingeto's code formation is untested. The two repairs also differ in kind. Dingeto's grounding-reward GRPO moved its proxy and not the audit. Flow-NLA changes the reconstruction model rather than adding a truth term, and the authors say plainly that it "may still reward unsupported wording". The paper offers no mechanism for why a richer likelihood reward should mean fewer unsupported claims. Read the confabulation gain as measured, not as explained.

It names the checkpoints the other audits used. Table 3 gives the convention for the released Anthropic NLAs: "a released NLA at layer K reads the output of decoder block K, i.e., hidden_states[K+1]". Under it, nla-qwen2.5-7b-L20 reads hidden_states[21]. Dingeto states that its "layer-20 tap is index 21", and Denamganaï reads "layer 21". So the one-layer gap between those two audits is an indexing convention. It is not evidence of two different checkpoints, and both are most plausibly this release. Neither paper names the repository, so this is likely rather than confirmed. By the same convention, Shportko et al.'s kitft/nla-gemma3-12b-L32 pair is Lek's released Gemma-3-12B NLA, which scores 83.1% assertion risk here.

What it does not show#

• "Unsupported" is not "false about the activation". Source support checks claims against the text before the extraction point. A claim that correctly reports something the model computed but the text does not entail counts as confabulation if the judge assesses it, and hidden-computation claims are not assessed at all. The authors say both metrics measure "contextual source support rather than the utility of describing the model's internal computation". The precision question that matters for monitoring, whether claims about internal state are true, is still unmeasured.
• One automated judge per dimension, prompts unreleased. All judges are a single DeepSeek model, and there is no human agreement check in the paper. Code and prompts are promised only "upon acceptance". The authors also note that "provider and inference differences … limit causal attribution to the reconstruction objective alone."
• Open models only. These are not the Claude NLAs that produced the grader-awareness and Opus 5 taxonomy results. The match on utility to Fraser-Taliente et al.'s Claude numbers suggests the recipe behaves the same, but it says nothing about their confabulation rate.
• Seeds are not reported. Shading in the figures is bootstrap over examples. Checkpoints are chosen by utility, which favours neither method on confabulation.

Contradicted by the model's own answer: WorkspaceBench (September 2026)#

WorkspaceBench (Blank, Bhatia, Ong & Nanda, MATS, LessWrong, 2026-09-23, empirical) ranks an NLA against lenses and other readers on whether they read Qwen-3.6-27B's global workspace. Its hallucination eval has Qwen answer assistant prompts. A judge then marks each readout claim that Qwen's own response contradicts (a readout saying "the user writes in Spanish" when Qwen answers in Portuguese). The headline is the share of readouts with at least one such claim.

The result, in the authors' words: NLAs "happen to surface a lot of workspace content", but "nearly every NLA readout contains some type of claim that is directly contradicted by the surrounding context of the model's outputs." Read off the chart (approximate, no table is published): NLA-RL about 0.96, NLA-SFT about 0.97, Oracle Lens 0.28–0.34, J-lens 0.31, and Patchscopes and the prompt-only baseline about 0.06–0.07. The authors' inference is the monitoring one. If a reader's verifiable claims are largely false, its unverifiable claims about internal state deserve low confidence.

Where it fits among the other audits.

• Against Lek et al.'s confabulation section above, it is a stricter label and a coarser unit. Lek counts claims that are unsupported or contradicted by the source text, per claim (76–93% on released NLAs). WorkspaceBench counts only claims contradicted by the model's response, per readout. If most claims in a ~250-token readout are unsupported, then at least one being outright contradicted is close to certain. The two numbers agree in direction and are not the same quantity. One thing differs in kind. RL against reconstruction barely moves the rate here (SFT about 0.97, RL about 0.96), where Lek finds it rising with FVE. A metric pinned near 1.0 cannot show that rise, so this is not a contradiction.
• Against Denamganaï's identification without discrimination, it is the same shape on a different benchmark. The NLA passes many content families (typo about 0.98, association about 0.65, agentic misalignment about 0.53, all chart readings), so it carries the right content. It also states contradicted claims in almost every readout. Useful identification sits next to unreliable assertion, which is the lineup result's shape, now on free samples from a fifth model.
• It measures the J++ authors' argument, and refines it. The Connections note below had recorded that lenses "should confabulate less" as an untested argument. In direction it holds: the J-lens is about 0.31 against about 0.96. But the multi-token Oracle Lens, which also writes free text, sits with the J-lens and not with the NLA. So expressiveness alone is not the cause. Training objective and readout length are both still candidates. Oracle Lens's RL reward is the FVE of its own bullets, also a reconstruction reward, which makes the comparison with the NLA's reconstruction RL a natural next experiment.

Caveats. One NLA on one model, read at one layer (L42), with a ~250-token sample against the Oracle Lens's ~90 tokens. The per-readout metric rises with length. The post does not say who trained the Qwen-3.6-27B NLA, and it is not one of Anthropic's four released open checkpoints. "Contradicted by the response" still checks claims about the conversation, not about internal state, so the precision question that matters most for monitoring is no more answered here than in Lek. Full benchmark and grading notes are on WorkspaceBench.

Why it matters#

Until the Shportko post, every measured limit of the NLA in this corpus was about precision: could a flagged decoding be a hallucination, and can the reconstruction score tell? Denamganaï's lineups are the most direct precision measurement so far, and they bound it from the favourable side: an oracle reader that shares the pair's conventions, on claims the transcript settles, still prices a false claim barely above chance. Lek et al. then measure precision on what the verbalizer actually emits. On every released open NLA, most claims are unsupported by the source, and the standard RL recipe makes this worse as its own score improves. That is a reason to read any NLA flag as a lead to check, not a finding. The Shportko post is the first measurement of recall on a behavior with known ground truth, and it fails in the direction that matters most for safety. Trained-in behavior is the kind a deployed misaligned model would have. Instructed behavior is the kind a red-team prompt produces.

It also lines up with the organism result on Model Organisms: quirks integrated into the original post-training run were the least legible to every white-box technique tested. This post is one further data point from a different angle. The same behavior with the same model reads clearly in context and drops out once it is trained in. Both point to the same discount on organism-validated NLA results. What an NLA finds in a freshly instructed or crudely installed behavior is an upper bound on what it will find in a trained one.

What it does not do is retract any of the Claude results. Those NLAs were trained on the models they read, the behaviors found (grader awareness, fabricated consent) are effortful deliberation rather than a practised encoding, and none of that is tested here.

Connections#

• White-Box Activation Monitoring — the family page and the home of the NLA's deployment results, the Dingeto reconstruction-score audit and RECAP. This page holds the recall evidence and the identification-vs-discrimination lineups that page is too large to carry
• Model Organisms — the recipe-dependence result this extends: integrated organisms are least legible, and here SFT removes a trace that ICL leaves
• Automatic vs. Flexible Cognition in LLMs — the mechanism the authors conjecture (automatization keeps trained-in computation out of the workspace), not measured here
• Internal Signatures of Misalignment — the J-lens authors' own escape route (a behavior reinforced into a fixed circuit), now seen through a second instrument
• Jacobian Lens (J-lens) — the cheaper rival benchmarked against the NLA. Nobody has run the J-lens on an instructed-vs-trained pair, which is the direct test of whether the gap belongs to the NLA or to the behavior. The J++ lens authors argue that lenses confabulate less than NLAs because the readout comes from the model's own weights and is less expressive. That argument has no measurement behind it. No lens has been through a claim-level audit like the ones on this page (superseded 2026-10-15 by WorkspaceBench: Evaluating Interpretability Methods for the Global Workspace: WorkspaceBench's response-checked hallucination eval measures it, J-lens about 0.31 against NLA about 0.96. See the WorkspaceBench section above), and a single-token readout makes no propositional claims to audit
• Evaluation Awareness & Grader Gaming — the target the NLA does best on, which the authors classify as effortful rather than automatic
• Claude Opus 5 — the largest published NLA deployment
• The Global Workspace in Language Models (J-space) — the selectivity property the authors' conjecture rests on
• WorkspaceBench — the benchmark behind the ~0.96 contradicted-readout rate, where the NLA reads workspace content well and asserts contradicted claims alongside it

Open Questions#

• Does the trained-in blind spot survive when the NLA pair is retrained on the fine-tuned model's own activations? That removes the distribution-shift confound: if T for the SFT model rises toward the ICL value, the result was about an NLA reading a model it was not trained on, not about internalization.
• Is there a layer, scheme or model at which a fine-tuned covert behavior does survive the round trip? The case study tests one layer (32 of Gemma-3 12B) and one one-bit scheme. A layer sweep of T would show whether the secret is absent from what the verbalizer can say or only absent at the layer this NLA reads.
• Does the paraphrase-averaged reconstruction loss the authors propose change T, or the incrimination rate, for a trained-in behavior? It targets verbalizer codes, and the round-trip null here suggests codes are not the failure.
• Does a transcript-only monitor solve Denamganaï's rival lineups at close to 1.0? If it does, the NLA adds nothing on transcript-settled claims, and its 0.602 is a deficit against reading the conversation rather than a weak positive.
• Does the AR's near-chance pricing of a changed claim (0.602) hold for processing-level accounts, tested where a transcript monitor sits at chance by construction, such as persona steering applied to an unchanged transcript?
• Is the rise in source-unsupported claims during point-reconstruction RL the formation of co-adapted codes? An evaluator swap (Dingeto's detector) on late checkpoints from Lek et al.'s reruns would say: if the unsupported claims lose their reconstruction weight under a fresh reconstructor, they were codes. If they never carried any, they are gist-irrelevant filler the objective tolerates.
• Does Flow-NLA's lower assertion risk survive a claim-level audit and a second judge? Run Dingeto's flip audit and a non-DeepSeek (or human-checked) source-support judge on Flow-NLA against point-reconstruction checkpoints. If grounded fractions do not rise, the gain is fewer stated claims rather than truer ones.

Sources#

• NLAs miss some internalized behaviors — Shportko, Arcuschin, Lange & Meyer, LessWrong, 2026-10-08, empirical (an exploratory case study by the authors' own epistemic status, not peer reviewed). Used: the setup and footnotes (model, NLA checkpoint, monitor), Figure 2 (incrimination by condition, values read off the chart and approximate), Figure 3 (exact table of log-odds and Δ), Figure 4 (T with CIs, from the legend), Figure 5 (Δ by token offset, approximate), Figure 6 (FVE by offset, approximate), the workspace conjecture and the paraphrase-loss proposal
• On the Monitorability of Natural Language Autoencoders: Identification vs Discrimination — Kevin Yandoka Denamganaï, arXiv 2610.18727, 2026-10-13, single author, a self-described open draft, empirical, weighted as a single-author draft. Used: §3 (the two referential games, the truth relation, Definition 3.6 on which accounts a monitor can settle), §4 (the AR surrogate, lineup AUC, rival construction and judge validation), §5 (Conjectures 5.1 and 5.2, stated not proved), §6.2 (Figure 4, exact values printed on the chart and viewed; Table 1, reconciled against the prose range 0.480–0.509; Figure 5 angles, viewed), §6.3 (paraphrase and shuffle damage, authorship effect), §7 (limitations), §9 (bilinear read-out, discriminative objective). Parse warning: none on the one table cited; §2.1 is two sentences in the PDF itself, a drafting gap rather than a parse loss
• Understanding Confabulation and Rethinking Reconstruction in Activation Explanations — Gert Lek, Zixuan Xia, Pin-Yu Chen & Lydia Y. Chen (Neuchâtel, Bern, IBM Research, Delft), arXiv 2609.33702, 2026-09-27, empirical, under review, code and prompts not yet released. Used: §3.1 (the three judges and their metrics), §3.2 (FVE–confabulation-index correlations, hard-negative separation ranges, released-NLA prose ranges), Table 2 (absolute scores, every cell reconciled against the §3.2 and §4.3 prose ranges), Table 1 (the Gemma-3-27B and Llama-3.3-70B released rows), Table 3 (extraction sites and the hidden_states[K+1] convention), §4.1–4.2 and App. F/H/I/J (the conditional-mean blind spot, Flow-NLA's reward, EMA scoring, noise-band variance shares, the energy-score failure), §4.3, App. D (gain/penalty decomposition) and §5 (limitations). Figures 2 and 7 viewed; the late-training index levels, the FVE plateau and Figure 7's per-model deltas are read off the charts and approximate. Parse warnings: Table 1 lost its group-label column (the first four rows are "Released Anthropic NLAs", the last three "Point-reconstruction reruns"; recovered with pdftotext -layout). Table 3's "Ours" row welded two models into one (Qwen2.5-7B-Instruct a09a354 → hidden_states[21]; Gemma-3-12B-it 96b6f1e → hidden_states[33], pairing confirmed with pdftotext -layout). Table 2 parsed clean.
• Train the Model, Not the Reader: Decodability Supervision for Verifiable Activation Explanations — Dingeto, arXiv 2607.20379: the co-adapted-codes and paraphrase-retains-89% results, used here to place the paraphrase-loss proposal, the layer-20-is-index-21 statement (the note under its Table 8), and the contrast with Lek et al.'s confabulation trend. Full treatment on White-Box Activation Monitoring
• Claude Opus 4.8 System Card, Claude Opus 5 System Card — the Claude NLA deployments summarized in the table above
• Verbalizable Representations Form a Global Workspace in Language Models — the J-lens-vs-NLA benchmark and the selectivity property the authors cite
• WorkspaceBench: Evaluating Interpretability Methods for the Global Workspace — Blank, Bhatia, Ong & Nanda (MATS), LessWrong, 2026-09-23, empirical. Used: Guarding against hallucinations, the Hallucination eval definition, the Discussion section "NLAs surface workspace content, but tend to hallucinate" (quoted), and the reader budgets (NLA ~250 tokens at L42). Per-reader rates are read off fig3 (viewed) and are approximate (±0.03), with no table behind them. Full treatment on WorkspaceBench
• J++ Lens: Jacobian Filtering Enables More Faithful Workspace Lenses — Ayonrinde & Lindsey, LessWrong, 2026-10-08, empirical: Appendix B only (the argument that J-lens-family readouts confabulate less than NLAs). Cited as an argument. Full treatment on Jacobian Lens (J-lens)