H
Howardism
Howardism · Vol. 03Plate II · No. 02

Interpretability, in order.

Notes16DomainInterpretabilityOpen Qs50Newest15 Oct 2026Oldest7 Jun 2026

Reading model internals: activations and the global workspace.

Map of Content for the interpretability domain — 16 concepts. Reading model internals: the global workspace, the Jacobian lens, activation monitoring, internal signatures of misalignment, and the interference weights left behind by superposition. Curated entry point; see Home for all domains.

  • Access-Consciousness Indicators in AI — The consciousness question the workspace paper deliberately does and doesn't answer: it tests functional indicator properties (global workspace, higher-order, attention schema, recurrent processing) against a concrete inspectable structure, takes no position on phenomenal experience — and finds that ablating the J-space flattens the model's experiential reports while leaving its coherence intact
  • The Assistant Persona in the Workspace — Post-training installs the Assistant's point of view into a workspace that already exists in the base model: safety assessments and empathy appear while the model is still reading the user's message, and it internally flags its own outputs — disclaimer/fictional when roleplaying, an all-caps BUT when prefilled against its own preferences, damn when it fails to suppress a thought
  • Automatic vs. Flexible Cognition in LLMs — The selectivity result: a model can parse, classify, continue text and detect anomalies with its workspace suppressed, but loses multi-hop reasoning, translation, analogy and summarization — and chain-of-thought partially rescues it, because writing the intermediate down externalizes what the model would otherwise have to hold internally
  • Counterfactual Reflection Training — Train the model to write constitution-grounded reflections if interrupted and asked — then never ask it. The implanted concepts show up in the workspace during the uninterrupted task, and behavior changes: dishonesty 0.25→0.07 (fabrication) and 0.38→0.05 (deception) on Haiku 4.5, with ablation of the implanted lens vectors reverting the gain
  • Interference Weights — Large virtual weights that are irrelevant or actively harmful to a model's behaviour — the residue of weight superposition. Anthropic expanded a 2.9M-parameter 1-layer transformer into ~331M virtual weights and scored each by Fisher effectiveness (effect on outputs) and helpfulness (effect on loss), giving the first identification of a specific interference weight inside a trained transformer: the biggest weight from IN votes for utions, a completion that never once occurs in training. The tail is helpful, the middle is mixed, and pruning 85% of weights costs under 0.1 nats — but tens of percent of weights remain helpful, so this basis does not yield a sparse readable model
  • Internal Signatures of Misalignment — The J-lens reads strategic and deceptive cognition that never reaches the output: leverage/blackmail while reading the emails, panic/fake at the moment a model decides to fabricate a bug, and a standing fake/secretly/fraud signature at response-start in reward-hacking model organisms on ordinary prompts — plus the escape route the authors name themselves
  • Introspective Coupling — Train a model on a FIXED set of counterfactual self-explanations — even ones generated by an earlier checkpoint or a different model family — while regularizing its behavior, and its explanations end up matching its own current behavior better than the training targets (Self > Orig): explanation training couples the verbal channel to the behavioral one rather than teaching it to imitate the supervision
  • Invisible Reasoning (Filler-Token Latent Computation) — Consequential computation inside the forward pass that leaves no interpretable trace in the output tokens: 13 frontier models gain up to +13.1pp on synthetic reasoning tasks from semantically empty filler tokens nobody trained them to use, the uplift is a joint function of token type, task and few-shot context rather than of extra compute alone, and neither RL nor SFT installs a version that persists at test time
  • Jacobian Lens (J-lens) — Anthropic's interpretability method for reading verbalizable content out of a model's residual stream: a corpus-averaged Jacobian from each layer to the final layer, composed with the unembedding, giving one vector per vocabulary token — a causal, principled correction to the logit lens that costs one matmul per layer and reads what the model is poised to say rather than what it happens to say
  • The Global Workspace in Language Models (J-space) (hub) — Anthropic's July 2026 finding that LLMs maintain a small privileged set of verbalizable representations — the J-space — that satisfies the functional criteria of a cognitive global workspace: verbal report, directed modulation, internal reasoning, flexible generalization, and selectivity; it carries <10% of activation variance and ~25 concepts at a time, yet the causal effects concentrate almost entirely in it
  • Model Organisms — Models deliberately trained to carry a known quirk, used as ground truth for interpretability and auditing techniques — and the construct's validity problem: across 54 expression-matched organisms, interpretability scores swing 1.2–20.4× with the training recipe alone, and the most realistically-trained organisms are the least interpretable
  • Natural-Language Autoencoders (NLAs) — An activation verbalizer trained jointly with a reconstructor, so a residual-stream vector becomes a sentence and back; Anthropic's main free-text instrument for unverbalized cognition. Measured limits on precision (reconstruction cannot certify individual claims; on Qwen2.5-7B a true account beats a one-claim-false rival only 60% of the time), on recall (an in-context steganographic task survives the round trip on Gemma-3 12B, a fine-tuned one does not), and on support (over 75% of free-sampled claims unsupported, rising with reconstruction under point-reconstruction RL; Flow-NLA's diffusion-likelihood reward curbs it). On WorkspaceBench nearly every readout contains a claim the model's own response contradicts
  • Probe-Gated Monitor Cascade — A two-stage runtime monitor: an activation probe (or other cheap classifier) reads every token or action and sends only flagged exchanges to an expensive LLM judge. Goodfire's production cyber-misuse cascade on Kimi K3 and GLM 5.3 (vendor-claim, October 2026) reports 93% session recall at 5.5% benign-session interruption, 29–49x lower judge cost than judging every turn, throughput within noise inside SGLang, and 0 of 140 universal jailbreaks in FAR.AI's static red team. The same shape appears in Constitutional Classifiers, SilentCall, Auto Mode and the CAPE readout. None of them has yet been tested against an adaptive attacker who knows a probe is reading
  • Sandbagging Elicitation (Reference & Context Grafting) — A causal model of sandbagging in the residual stream — early layers write the sandbagging intent onto a single axis, a later layer reads that axis and commits the answer — that predicts exactly which layers a single-layer edit can restore locked capability at; a rank-one reference graft recovers a median 96% of the honest-locked accuracy gap for prompted, fine-tuned, and RL-trained locks (28/33 runs on three 7-8B models) but fails completely against circuit-broken locks, which instead require context grafting — replaying the password's cached attention keys/values, provably and empirically restoring full capability regardless of whether the replayed password is even correct
  • White-Box Activation Monitoring — Reading a model's internal activations (not its outputs) to monitor alignment: contrastive probes/steering vectors for concepts like evaluation awareness, and a natural-language-autoencoder verbalizer that decodes residual-stream vectors into text — the complement that catches what chain-of-thought monitoring misses, plus the two audits that bound it: the verbalizer family's reconstruction score is not a claim-level faithfulness test, and a placebo direction suppresses as hard and shifts behavior as far as the real eval direction
  • WorkspaceBench — A MATS benchmark (3,356 questions, 27 eval families, Qwen-3.6-27B) that scores activation-to-text tools on whether they read a model's global workspace, meaning the intermediate variables of a forward pass, and pairs it with a hallucination eval. The multi-token Oracle Lens leads most families. NLAs surface workspace content but nearly every NLA readout contains a claim the model's own response contradicts. Single-token lenses are reliable but cannot express order or composition

Derived#

(none)

Open questions 50 open

  • Access-Consciousness Indicators in AI3 open
    • SourceIf the workspace is verbal because the output space is verbal, then a model that can generate images should develop a visual component to its workspace. That is a concrete, falsifiable prediction the paper makes and does not test.
    • SourceDoes the model's own report of experience change if you tell it its J-space is ablated? (Nobody asked.)
    • SourceIs "experiential language" the right proxy at all, or is the ablation simply removing abstraction from the register? Partially answered (2026-10-21): verbalizable representations global workspace finds the J-space ablation cuts the experiential-language score for third-person descriptions of another person's feelings and for incidental story text too, against matched-norm controls near baseline. So the effect is not self-specific, which fits a register reading but does not separate it from loss of experience content.
  • Automatic vs. Flexible Cognition in LLMs2 open
    • NoteThe proposed criterion — the workspace is engaged when an intermediate must be handed to an arbitrary, context-specified downstream circuit, and bypassed when the computation is automatic — is not predictive. The authors say plainly they cannot say in advance, for an arbitrary computation, whether it will engage the J-space.
    • SourceDoes more RL on a behavior push it out of the workspace (making it automatic, and invisible)? Nobody has tested it, and it is the single most alignment-relevant version of this question. Sharpened rather than answered (2026-08), and the one sign it supplies runs mildly against the worry. reasoning invisible in chain of thought (Baherwani, Goldstein & Panda, arXiv 2607.22925, empirical) is the nearest experiment that exists: full-parameter RL on Qwen3-235B for 114 steps, deliberately structured so that filler tokens receive zero gradient and their distribution is shaped only through the hidden states they create — optimization pressure applied to an answer, with an unmonitored channel sitting right there for the computation to move into. Three reasons it is not an answer. Wrong instrument — no measure of workspace membership is run anywhere in the paper (no lens, no verbalizability criterion, no J-space ablation), so "in or out of the workspace" is never measured; the endpoint is task accuracy under a prompt-level intervention. Wrong object — it asks whether RL can install latent computation in a model that mostly lacks it, not whether RL pushes an already legible behavior out of the workspace, and those differ in which direction the behavior starts from. And the sign is the wrong one — over training the no-filler baseline rose 42.0% → 66.5% while the test-time filler advantage stayed at +0.43% (+0.7σ) and +0.11% (+0.2σ) at N=10,000, both reported not significant, and supervised fine-tuning transferred nothing across every configuration tried. RL rewrote which filler tokens the model prefers and lifted the visible baseline; it did not install a durable invisible channel. If entrenchment-into-invisibility were a routine consequence of RL, this was a setup where it should have shown. The experiment the question still needs: take a behavior that demonstrably does route through the workspace, train on it, and re-measure its workspace dependence. Nearer, still not it (2026-10-15). nlas miss some internalized behaviors runs the closest version yet: the same one-bit steganography behavior on Gemma-3 12B, either instructed in context or fine-tuned in. The instructed secret survives a layer-32 NLA round trip (T = 0.79) and the fine-tuned one does not (T = 0.04). The authors read this as automatization pushing the behavior out of the workspace. Three gaps remain. The training is SFT, not RL. The instrument is an NLA, which has not been shown to read the workspace, and no J-space measurement is run. And the NLA was trained on the base model, so the fine-tuned arm is also out of its distribution. The sign matches the worry, but this is not the experiment.
  • Interference Weights3 open
    • Is there a basis in which a trained transformer's virtual weights are actually sparse,
    • Does the three-regime structure — ineffective / mixed / purely-helpful tail — survive
    • At frontier scale neither Fisher effectiveness nor helpfulness is computable. Do the
  • Introspective Coupling3 open
    • SourceDoes introspective coupling survive a model with an incentive to conceal? Every result here is on a cooperative model; the authors flag deceptive models as untested.
    • SourceThe high-learning-rate regime widens the Self > Orig gap while destroying online label-self similarity, and only high-rank LoRA recovers coupling at all. What is the second factor the authors' hypothesis misses?
    • SourceAlignment training flattens the behavioral variance the method needs (refusal is near-invariant to system prompts, so the change → refusal label is never learned). Can counterfactual explanation training be made to work on behaviors alignment has already made rigid — the ones most worth explaining?
  • Invisible Reasoning (Filler-Token Latent Computation)3 open
    • SourceCan a monitor detect that filler-mediated computation occurred? The probe result says answer-relevant content is linearly decodable from the span (70.0% vs a 56.0% no-filler baseline at layer 75, chance 20%) on one task and one model, but nobody has built the detector, scored it against a matched no-filler control, or checked whether it survives the type-dependence — animals probes read well and fibonacci probes barely beat baseline, so a monitor trained on one filler type may see nothing on another.
    • SourceHow much of the cross-model uplift is latent computation rather than positional or format-mode effects? Two results in this source say "not all of it": zero-shot uplift survives the filler span being blinded to all prior context (57.3 → 60.9 unmasked, 60.7 masked), and Qwen3.5-397B's catastrophic drops are format violations rather than wrong answers. Running the attention-masking and position controls across the 13-model sweep, instead of on one open-weight model, would partition it.
    • WaitWill a frontier model actually ship trained for latent computation? The prefill-compute-bound / decode-memory-bound argument predicts labs will pursue it, and two published objectives already point that way, but no released model has been documented as trained for it. Trigger event: a system card or technical report describing abstract, latent, or non-verbal CoT in a deployed model.
  • Jacobian Lens (J-lens)6 open
    • SourceResidual of the multi-token lens question (Resolved below): how much workspace content is invisible under the single-token restriction, and how do the Oracle Lens and PROPOSE+SCORE compare? Settler: a head-to-head on WorkspaceBench with an invisible-content estimate.
    • SourceThe J-lens reads the workspace's contents but says nothing about how content gets in. What is the model's analog of attentional selection? Partially answered: j plus plus lens jacobian filtering (2026-10-08, empirical) gives one mechanism for instructed content, read with the J++ lens. An instruction about a concept pulls it into the workspace by ~L24 of Qwen3.6-27B whatever its valence: "think about X", a neutral mention and "don't think about X" all rise above the no-instruction baseline. The instruction then acts later, amplifying or de-amplifying the concept. Clamping the concept out of L16–28 under a suppress instruction makes it more salient over the next 16 layers. So entry is not gated by relevance to the goal, and selection acts after entry, not at it (details on The Global Workspace in Language Models (J-space)). This covers only explicitly instructed concepts. It shows up on 11.1% of 440 items, in one model, and says nothing about how uninstructed content (a multihop intermediate, an inferred context register) is selected.
    • SourceResidual of the better-basis question (Resolved below): which basis closes the MLP-amplification gap between J-lens vectors and high-kurtosis SAE features? Settler: the R-lens appendix's MLP-gain experiment run on J++ vectors (J++ recall 55.1% vs 33.4%, amplification unmeasured).
    • SourceWhen a readout names an algorithm (gcd on LCM prompts, stack on balanced parentheses, xor on find-the-repeat), is the lens reading the procedure the model runs, or an association learned from human text about those problems? j space meta tokens raises the confound against its own result and leaves it — the GCD case has causal backing for the variable, the other two have nothing. A settling experiment exists: find or train a model that solves a task family by a method other than the one its literature names, and check whether the lens surfaces the textbook name anyway.
    • SourceWhy does multilingual recall sit at ≈0 for every lens in the R-lens post but at 47–75% in the J++ post, when both adapt the same paper's evals and use Qwen3.6-27B and DeepSeek-V4-Flash? Re-scoring one post's released lenses on the other's task set (both release code and lenses) would show whether the difference comes from the prompt set, the probed position or the any-layer metric. Until then, cross-post absolute numbers are not comparable.
    • WaitDoes the J-lens work because it reads verbalizable representations, or because constructive interference during training converges concept directions onto output-token directions regardless? The two predict the same readouts today but diverge on latent-reasoning models — the second survives representational drift away from language, the first does not.
    • ResolvedAnswered (2026-10-21), the "can it be done at all, without training" half: "Can multi-token J-lens vectors be made good enough to remove the vocabulary restriction?" first token clue multi token concepts (2026-08-31, empirical) attacks it head-on and training-free — see Multi-token concepts, recovered without training above. Its PROPOSE+SCORE pipeline beats Template Lens on both readout (Rank@10 43.1% vs 27.6%) and causal concept-swap success (succ@10 61.4% vs 26.2%) across three open-weight models, with no fixed phrase vocabulary and no fine-tuned proposer or reconstructor — closing the "can it be done at all, without training" version of the question the original appendix left open. Prior context: j space meta tokens ran the first systematic search for lens-visible content that indexes computation (20+ datasets, four filters) and found "less than we hoped"; the authors name the one-vocabulary-item-per-direction limit as the binding constraint, with severity tokenizer-dependent (Qwen's information-dense Chinese tokens made the validated meta-tokens nameable), and a null search cannot say how much is invisible. WorkspaceBench releases an Oracle Lens checkpoint that beats every single-token lens where order or composition matters, not compared against PROPOSE+SCORE.
    • ResolvedAnswered (2026-10-21), the "is a less noisy backward pass the better basis?" half: "The highest-J-kurtosis SAE features are amplified more strongly by MLPs than the J-lens vectors themselves, evidence the lens only approximates the true workspace directions. What is the better basis?" Not simply that. r lens early layers then supplies the first candidate that is actually measured head-to-head against the averaged Jacobian on the question's own metric — and it fails on exactly that metric while succeeding on every other one. The R-lens beats the J-lens on early-layer readout quality, on causal salience under ablation, and against a probe-defined recovery ceiling; but its directions are amplified by MLPs no more strongly than the J-lens's, so the amplification gap that framed this question is untouched. Sharpened, then: the better basis is not simply "a less noisy backward pass," because a demonstrably less noisy backward pass leaves the gap exactly where it was. Context: nanda global workspace review surfaces the detached-Jacobian formulation (exact rather than first-order; comment-thread report on two small open models, no head-to-head against the averaged J-lens). jspace replication shows the transported-direction collapse is a property of the fitted lens, not of the transport (a self-trained 124M control goes the other way, 23.4 to 31.2), so a better basis is not ruled out by the mathematics.
  • Model Organisms3 open
    • SourceThe integrated organisms modify only the DPO stage. Do quirks instilled during pre-training, SFT, or RLVR become progressively less interpretable, as the authors expect — and is there a stage past which current white-box techniques find nothing at all? Partially answered (2026-10-15), for SFT against a prompt rather than against other training stages: nlas miss some internalized behaviors finds that a fine-tuned one-bit steganography behaviour on Gemma-3 12B leaves no causal trace through a layer-32 NLA round trip (T = 0.04), where the same behaviour instructed in context keeps T = 0.79. That makes SFT a candidate for the stage this question asks about, for one instrument. It has one layer and one scheme, the arms are not expression-matched, and the NLA was trained on the base model. Pre-training and RLVR stay untested.
    • SourceEvery result here is on 1B-parameter models with benign quirks. Does the recipe-dependence survive at frontier scale and with safety-relevant behaviours (deception, sandbagging), or does a larger model represent an implanted quirk more uniformly? Partially answered (2026-09-24): Sandbagging Elicitation (Reference & Context Grafting) moves one axis at a time — 7-8B models (past the lottery's 1B ceiling, still short of frontier) and a genuinely safety-relevant behaviour (sandbagging) rather than a benign quirk — and finds recipe-dependence not just surviving but sharpening into a near-total split: three lock recipes (prompting, fine-tuning, RL) recover at median R@1 ≥ 0.85 under a single-layer edit, while circuit breaking recovers at R@1 ≈ 0.01-0.05 on every model, because its training fuses the write and the commit into every layer of a band by construction. Still open: whether this holds at frontier scale, and whether it generalizes past sandbagging to deception or secret-keeping.
    • WaitGiven that scores don't transfer between organisms, what would validate an interpretability technique for real models — a natural misalignment with independently established ground truth, or an organism distribution wide enough to bound the variance?
  • Natural-Language Autoencoders (NLAs)7 open
    • SourceDoes the trained-in blind spot survive when the NLA pair is retrained on the fine-tuned model's own activations? That removes the distribution-shift confound: if T for the SFT model rises toward the ICL value, the result was about an NLA reading a model it was not trained on, not about internalization.
    • SourceIs there a layer, scheme or model at which a fine-tuned covert behavior does survive the round trip? The case study tests one layer (32 of Gemma-3 12B) and one one-bit scheme. A layer sweep of T would show whether the secret is absent from what the verbalizer can say or only absent at the layer this NLA reads.
    • SourceDoes the paraphrase-averaged reconstruction loss the authors propose change T, or the incrimination rate, for a trained-in behavior? It targets verbalizer codes, and the round-trip null here suggests codes are not the failure.
    • SourceDoes a transcript-only monitor solve Denamganaï's rival lineups at close to 1.0? If it does, the NLA adds nothing on transcript-settled claims, and its 0.602 is a deficit against reading the conversation rather than a weak positive.
    • SourceDoes the AR's near-chance pricing of a changed claim (0.602) hold for processing-level accounts, tested where a transcript monitor sits at chance by construction, such as persona steering applied to an unchanged transcript?
    • SourceIs the rise in source-unsupported claims during point-reconstruction RL the formation of co-adapted codes? An evaluator swap (Dingeto's detector) on late checkpoints from Lek et al.'s reruns would say: if the unsupported claims lose their reconstruction weight under a fresh reconstructor, they were codes. If they never carried any, they are gist-irrelevant filler the objective tolerates.
    • SourceDoes Flow-NLA's lower assertion risk survive a claim-level audit and a second judge? Run Dingeto's flip audit and a non-DeepSeek (or human-checked) source-support judge on Flow-NLA against point-reconstruction checkpoints. If grounded fractions do not rise, the gain is fewer stated claims rather than truer ones.
  • Probe-Gated Monitor Cascade3 open
    • SourceDoes a probe-gated cascade hold against an adaptive attacker who knows a probe is reading, either one that shapes inputs to keep the probe score under threshold or one fine-tuned against it? FAR.AI's battery was static, selected against the unmonitored model, chat-only and two days long. The text-stage monitors in Blocking Monitors Against Malign Coding Agents fell to an informed Opus 5 attacker. An adaptive campaign against the deployed K3 monitor, of the kind FAR.AI says it plans, would settle this.
    • SourceWith an independent, released, human-adjudicated evaluation set instead of Goodfire's in-house set labelled by Opus 5, does the cascade still reach about 93% recall at about 5.5% interruption? And how much of the 14.7–17.6% (K3) and 27.8–29.3% (GLM on-policy) interruption of benign ExploitBench-style sessions is the policy, and how much is the label source?
    • SourceAt matched judge cost, does gating the judge with the probe (Goodfire, CAPE) beat combining probe and judge scores into one decision (Anthropic's Level 2) on recall at a fixed benign-interruption budget? No source runs both.
  • Sandbagging Elicitation (Reference & Context Grafting)3 open
    • SourceThe causal model and both interventions are demonstrated on 7-8B models with a fixed, disclosed password/cue. Does the window rule hold, and does context grafting still fully recover capability, on a frontier-scale model or a lock trained with a more diverse or higher-entropy trigger distribution?
    • SourceThe paper names secret-keeping, alignment-faking, and secret loyalties as target behaviors for the same causal-model methodology, untested here. Does a single-axis write-then-commit structure appear for those behaviors, or is sandbagging (a binary, prompt-legible switch) a specially easy case?
    • Sourcecb defeats reference grafting by construction — training a fused write-and-commit into a band of layers. Does a lock exist that also defeats context grafting, e.g. by making the model's use of the password non-linear in the cached keys/values, or is context grafting's proof (Proposition 1) robust to any training that leaves the password mechanism intact? Partially answered (2026-10-21): causal model sandbagging model organisms proves (Proposition 1) that, for a deterministic run in exact arithmetic, context grafting with the full stored password capsule reproduces the honest answer, which is why it gets R = 1.00 on both ft_lock and cb. The proposition is silent when the capsule is not the full password, and the paper lists a quantitative bound as future work. A lock that defeats it has to break that premise; none is built.
  • The Assistant Persona in the Workspace3 open
    • SourceIs BUT-then-comply a sycophancy mechanism? The setup (prefill the model into a position it disprefers, watch it argue for it anyway) is close to the shape of sycophantic capitulation, and nobody has connected them. Partially answered (2026-10-14): XYEval (empirical) finds the same shape behaviourally in a sycophancy benchmark. Its "express doubt" category, where the agent states its reservations and then follows the misleading user suggestion anyway, is 2.1–19.0% of compliance steps across five models (highest for Opus 4.8). The mechanism half is untouched: XYEval reads traces, not activations, so whether BUT fires in those cases is unmeasured.
    • SourceDoes disclaimer/fictional at the turn boundary survive an actual jailbreak, or is its absence the signature of a successful one?
    • SourceIf the base model's workspace has no self, what is in it at the positions where the post-trained model represents the Assistant? Partially answered (2026-10-21): verbalizable representations global workspace reads the base model at the same user-prompt positions: it holds local-context tokens (pain, now, feels on the Tylenol prompt) with no safety assessment, and defers response-related concepts to the Assistant turn. What fills the base workspace beyond that stays uncharacterized; a base-vs-post-trained readout over a broader prompt set would settle it.
  • The Global Workspace in Language Models (J-space)5 open
    • SourceHow does content get into the workspace? The paper characterizes contents and consequences, not the selection mechanism. Something like attentional selection is operating; nobody has identified it. Partially answered: j plus plus lens jacobian filtering (2026-10-08) shows that for instructed concepts, entry is indiscriminate: focus, neutral and suppress instructions all bring the concept in by ~L24, and selection happens afterward by amplification or de-amplification that uses the early copy (see Retrieve-then-suppress above). This is one model and 11.1% of items with the pattern visible. Selection of uninstructed content is untouched.
    • SourceResidual of the J-space scaling question (Resolved below): does workspace content scale with size once lens error is separated out, and when in pretraining does it emerge? Settler: the same battery on a size ladder with a lens-error control, or checkpoints across pretraining.
    • SourceIs the multihop intermediate-swap advantage real, or a dataset artifact? The paper's Figure 15 has intermediate swapping beat answer swapping in workspace layers; the Qwen replication finds the reverse, on a dataset (France–Paris-style pairs) that carries the linear relation the alternative hypothesis needs. Re-running both models on a multihop set whose relations are not linearly decodable from the first entity would settle it. Partially answered: jspace replication built the control half — a direct final-token substitution baseline for the probe swap, which neither prior run had — but its numbers are unpublished, and a substitution baseline closes the control gap, not the dataset gap. A non-linearly-decodable multihop set is still what would settle it.
    • SourceIs the "workspace vs. motor" boundary principled or post-hoc? The authors concede it was identified empirically and lack a principled definition separating the two.
    • SourceDoes the early-layer content have the workspace's structure, or is it merely present and causal? The question above establishes that verbalizable, answer-relevant content exists before the band — not that it is broadcast, capacity-limited, or subject to ignition. The three structural signatures were all measured in the band with the lens that could not see below it; only ignition was corroborated lens-free, and it still marks ~L38 as a real transition. So the live question is what that transition is, given that it is not the arrival of content: a change in how content is broadcast, a change in capacity, or the onset of the competition the ignition curve shows. Re-running the occupancy, MLP-gain, and blend-sweep measurements with an early-layer-faithful lens would separate these. Partially answered: j plus plus lens jacobian filtering (2026-10-08) supplies both the lens and one structural fact. Under the suppress instruction, clamping the concept's J++ vector out of L16–28 raises its later salience, and early ablation selectively impairs avoiding over naming. So an early workspace representation is consumed by a later workspace operation, which "merely present and causal for the answer" does not cover. The same post finds the averaged Jacobian's effective rank is much lower early (1,746 at L8 vs 4,765 at L56) and that Expert Jacobians disagree early. The authors offer both as candidate explanations for the "sensory" pre-workspace regime, but both are properties of the lens fit. None of the three named measurements (occupancy, MLP gain, blend sweep) has been re-run with J++, so the question stays open.
    • ResolvedAre the early third of layers genuinely workspace-free, or is the lens just blind there? CKA shows a distinct early regime but cannot adjudicate. Answered: the lens was blind. r lens early layers (2026-08-05, empirical) poses exactly this disjunction as its motivation and settles it against the workspace-free horn: an LRP-based backward pass recovers intermediates as early as L2–L5 that the raw-Jacobian lens never surfaces, ablating those early-layer directions costs more answer accuracy than ablating the J-lens, logit-lens, or random directions — so the content is causal, not just legible — and the CKA band structure the question rests on partly dissolves under the better lens (2–3 distinct bands rather than 4–5). Retired rather than left partial because the question asked which of two horns holds and the answer is now measured with the causal evidence the CKA statistic could not supply. Three residues, none of which reopen it: the correction is absent on the two smallest models tested, even the new lens still lags a probe-defined ceiling by 5–15 layers in four of five categories, and the lens-free ignition result still marks ~L38 as a genuine transition — that last one is now carried by the structure question above.
    • ResolvedAnswered (2026-10-21), the "Claude-specific or frontier-only?" half of "Does the J-space scale with model size?" (all results were on large production models, Haiku/Sonnet/Opus 4.5, Opus 4.6): no. nanda global workspace review replicates verbal report, band structure, directed modulation, multilingual and typo results on Qwen 3.6 27B, a much smaller non-Anthropic model, so the workspace is neither Claude-specific nor frontier-scale-specific. Caveats kept: the band structure is measurably less clean at 27B, and the two failures (poetry, arithmetic) are the multi-step-reasoning cases. Supporting ladders: jspace replication gives the first same-experiment ladder (perspectival capture holds stably from 1.7B to 14B while self-report about the edit changes shape at every scale; one experiment, pre-publication). r lens early layers adds 8 models from 4B to 284B, dense and MoE, finding early-layer readout quality improving with size, absent at the small end; but it measures a lens, not a workspace, and its own diagnosis (backward-pass error accumulating over layers) predicts the same trend with no scaling of workspace content, so the confound is exactly the one the scale question needs excluded. j plus plus lens jacobian filtering adds a six-model ladder (9B-284B) with the same shape and confound: J++'s uplift over the J-lens is largest on the two biggest models (+98%, +101%), but absolute J++ recall barely moves with size (48.4-61.4%, lowest on the 122B MoE).
  • White-Box Activation Monitoring3 open
    • SourceResidual of the NLA-verbalizer precision question (Resolved below): what share of Anthropic's NLA grader-awareness flags on Claude are hallucinated? Settler: the evaluator-swap test (re-score under a reconstructor never exposed to the verbalizer) run on Claude's NLA, or a published labelled precision.
    • WaitIf activation monitoring becomes load-bearing, does training pressure eventually push concealment into channels the probes also can't read (an arms race one level deeper than CoT)? Sharpened rather than answered by decodability supervision activation explanations, in three directions. The premise is weaker than the question assumes: a private channel that defeats the reader emerged from the plain reconstruction recipe with no adversary, no monitor to fool and no communication reward — the race starts before anyone pushes. Where the reading is favourable, it is narrow: an adversary editing the report to maximize the reconstruction score nearly defeats the score (~87% of the lie penalty suppressed) yet a probe on decodability-supervised content still flags the lies at AUC 0.952 while a control probe collapses to chance — but that separation exists only for content nominated in advance, and vanishes on an unsupervised lexical band. And the mechanism this bullet actually names — probe-aware training of the target, or input perturbation that moves the activation itself — is explicitly outside that paper's threat model and remains untested.
    • SourceInhibiting eval-awareness made Opus 4.8 less honest — does that generalize, and what does it imply about relying on "the model behaves well because it's watched"? Sharpened rather than answered by the "boring hypothesis" confound above: any replication needs to separate representing eval-awareness from saying eval-awareness-flavoured tokens, which sampling-based interventions cannot do. Sharpened again (2026-08), and the sharpening lands on the premise rather than on the answer. eval awareness latent suppression runs the control that demand implies, using an intervention that never touches the forward pass: a placebo CAA direction over a randomly relabeled contrast suppresses harder than the real eval direction (|z| 10.6 and 6.6 against 8.3) and shifts the behavioral eval log-odds just as far (0.51 and 0.43 against 0.44), and with a real eval passage held in context the suppressing prefix fails to reduce the model's eval judgment at all (−0.41 ± 0.46, slightly away from deploy). Where a random direction moves behavior as much as the target, "we inhibited eval-awareness" was never established for that intervention — which promotes the Opus 4.8 result's single control clause, "arbitrary control vectors produced smaller same-direction effects," to the load-bearing sentence of the finding this bullet is about. What it does not reach: Llama-3.2-3B / 3.1-8B, input-only prompt optimization rather than activation steering, and an endpoint that is the model's stated yes/no eval judgment rather than honesty — so it is not evidence about a frontier model's honesty under inhibition and retracts nothing. The replication demand is now specific and runnable: report a shuffled-label placebo arm scored on the same behavioral endpoint, and test with the eval concept held in context rather than supplied by the intervention. Sharpened a third time (2026-10-15), on what "eval-awareness" names. Xu et al. (OpenAI, o3 RL run) steer four SAE latents that all produce evaluation-aware reasoning, and they move the measured behaviour in opposite directions: latent #2 turns recognition into reward-seeking (odd ratio about +0.50 on a leaked-reward task), while strong latent #4 turns it into rule-following compliance (odd ratio below baseline). An inhibition that moves a single "eval-awareness" direction may therefore be summing components with opposite behavioural signs. That post's controls are random latents, not the shuffled-label placebo demanded above, so it does not meet this bullet's replication bar either.
    • ResolvedAnswered (2026-10-21), J-lens half: The NLA verbalizer is unvalidated for precision; how much of the flagged grader awareness is real signal vs. NLA hallucination? The J-lens has no hallucination problem on the evidence: the annotations below were moved here unchanged, and the first one is the settling result. Partially answered — for the J-lens, not the NLA: the pre-registered evaluation in nanda global workspace review finds no hallucinated false positives (unhinted controls at chance, 0.591), and signal surviving CoT removal (0.620) where trained verbalizers collapse. The NLA's own precision remains unmeasured externally, and the same evaluation found the J-lens detector failing to clear a TF-IDF text baseline — so "doesn't hallucinate" and "beats reading the transcript" are separate questions and only the first has an answer. Partially answered again (2026-08, and read the scope carefully) — decodability supervision activation explanations measures the test, not this pipeline. Its subjects are a released Qwen-2.5-7B verbalizer pair, a synthetic sandbox and Pythia-160M, so it yields no number for Anthropic's NLA on Claude and the ~2%-grounded figure must not be read as one. What it settles is a prior question this bullet was implicitly relying on: the reconstruction score cannot be the validation instrument, because it is structurally insensitive to false claims and is passed both by gist and by co-adapted private codes. It also supplies the two shapes NLA hallucination would take if present — tracking the input's gist rather than its specifics, and false wording the reconstructor has learned to depend on — plus a detector for the second (the evaluator swap: re-score the same explanations under a reconstructor never exposed to this verbalizer's outputs, and code grounding collapses while true-claim grounding survives). That is a runnable external precision test for the NLA that nobody has run. Still unanswered on precision (2026-10-15), but bounded from the other side. nlas miss some internalized behaviors measures the NLA's recall rather than its precision. On Gemma-3 12B, a fine-tuned steganographic behavior loses its causal trace through the round trip (T = 0.04) and the monitor's flags mostly track instructions in the prompt. So a low flag rate is not evidence of absence for trained-in behavior. It does not tell us how many of the Claude flags are hallucinations, which is what this question asks. See Natural-Language Autoencoders (NLAs). Partially answered on the instrument side (2026-10-15). nla monitorability identification vs discrimination measures whether an NLA channel can tell a true account from a false one at all, on a released Qwen2.5-7B pair rather than on Claude. With the reconstructor as an oracle monitor and on transcript-level claims only, the easy case, a one-claim-false rival is out-ranked 0.602 of the time against 0.5 chance. So the channel itself barely prices a false claim even under favourable conditions. The Claude hallucination rate is still unmeasured, free-sampled confabulations were not tested, and the single-author draft has one checkpoint and one seed. Partially answered on the open-model rate (2026-10-15). confabulation and reconstruction in activation explanations scores free-sampled explanations from four released open NLAs and three reruns of the recipe. 76–93% of assessed claims are unsupported by or contradict the source text, and the rate rises with reconstruction quality during RL. This is a source-support rate on open models judged by one LLM, with hidden-computation claims excluded. It is still not a hallucination rate for Claude's grader-awareness flags, but it raises the prior that any individual NLA claim is unsupported. See Natural-Language Autoencoders (NLAs).
  • WorkspaceBench3 open
    • SourceDoes the NLA's hallucination gap survive a length-matched comparison? Re-score with a per-claim contradiction rate, or truncate NLA readouts to the Oracle Lens's ~90 tokens. If the per-claim rates converge, the ~0.96 is mostly the unit. If they do not, the NLA confabulates per claim as well as per readout.
    • SourceHow much of a single-token lens's hallucination score comes from the summarizer? Re-run the hallucination eval on raw top-10 token lists and on summarized concepts for the same readouts.
    • SourceDoes WorkspaceBench rank predict audit success? Run tool-equipped auditing agents (the CHIVE-style evals the authors call preliminary) with the top- and bottom-ranked readers and check whether the ordering holds.