Jacobian Lens (J-lens)

Sources#

• A Review of Anthropic's Global Workspace Paper
• jspace-replication: Independent Replication of Anthropic's Global-Workspace Paper on Small Open Models
• R-lens: Making J-lens More Faithful on Early Layers
• Towards surfacing model algorithms with meta-tokens in the J-Space
• Train the Model, Not the Reader: Decodability Supervision for Verifiable Activation Explanations
• Verbalizable Representations Form a Global Workspace in Language Models

Summary#

An interpretability technique from Anthropic's interpretability team (Wes Gurnee, Jack Lindsey et al., July 2026) that answers the question: which concepts is this activation disposed to make the model verbalize? For each layer $\ell$ it computes the average Jacobian of the final-layer residual stream with respect to the layer-$\ell$ residual stream, taken over the source position, all subsequent positions, and a corpus of ~1,000 pretraining-like prompts:

J_ℓ = E_{t, t' ≥ t, prompt} [ ∂h_final,t' / ∂h_ℓ,t ]

Composing $J_\ell$ with the unembedding $W_U$ and reading the top entries gives a ranked list of vocabulary tokens for any activation. The rows of $W_U J_\ell$ are the J-lens vectors — one direction in residual-stream space per vocabulary token. Their sparse span is the J-space.

The averaging step is the whole idea#

A Jacobian computed on a single prompt conflates two things: the model's general disposition to verbalize a concept, and the particular use that concept is being put to right now. Averaging across contexts isolates the former. This is what separates verbalizable representations — poised to be spoken about, should the occasion arise — from ones that merely happen to be verbalized in one context. Everything the workspace results rest on follows from this distinction.

Why it should work at all — the first-principles account#

The paper establishes the lens empirically; the strongest argument for why such a technique must exist comes from outside it, in Neel Nanda's commissioned review. Worth keeping because it predicts the method's failure modes as well as its successes.

He separates two things the paper runs together: J-lens, the technique (Jacobian, then final layer norm and unembedding), and J-space, the span of sparse combinations of $W_U J_\ell$ rows — a hoped-for approximation of the model's actual cognitive space, not identical to it. Every argument below is about why the approximation should be a good one, not about why it should be exact.

Why a working memory must exist. Problems with multiple serial computation steps require representing intermediate states somewhere, and the residual stream is the only channel between layers. Under the linear representation hypothesis, intermediates should be directions in that space. The worked example: on "Michael Jordan plays the sport of", the Jordan-token lookup necessarily retrieves all Michael Jordan knowledge before the model knows which fact will be needed downstream. Intermediate storage is forced by the architecture, not a design choice.

Why the directions should be consistent across contexts. A concept flexibly serves many downstream circuits; a single shared per-layer direction is the efficient "API" for that. The caveat is the interesting part — a narrowly used concept has no such pressure and may not have a shared direction, so the lens should be systematically better on general concepts than on specialized ones.

Why intermediate concepts should relate to output logits. A looked-up fact eventually gets uttered, so Jacobians from intermediate positions to future logits align with concept directions. This is also the cleanest account of why the J-lens beats the logit lens: an intermediate concept is unlikely to be the immediate next token but is plausible as a subsequent one — and the paper's own ablation corroborates, since single-token-restricted J-lens variants beat the logit lens only marginally. Nanda offers an alternative mechanism he does not rule out: constructive interference during training may naturally converge concept directions onto output-token directions, in which case the lens works for a reason unrelated to verbalizability.

Why Jacobians rather than a learned regression. Tuned-lens-style regression captures whatever downstream concepts are correlated with the current state. An infinitesimal Jacobian perturbation cannot engage the network's nonlinearities, so the readout is closer to "the model reports the contents of this activation without further processing" — the property that makes the lens a read rather than a prediction.

Why single tokens are the binding constraint. One-to-one concept↔token correspondence is a priori unlikely: digits are tokenized separately while "twenty-two" may be one direction. So the vocabulary restriction is not an implementation detail to be engineered away but a structural mismatch — which is what makes the multi-token extensions the important open thread.

What the approximation costs. Causal interventions should suffer more than observational reads: an ablation removes only the fraction of the concept that lies in the lens direction, and negative steering compensates by steering noise components. This predicts exactly the pattern the review's own meta-token experiments hit — steering worked, ablation was largely ineffective. Nanda's blunt expectation for practice: the J-lens will not "reliably flag everything important" and will produce many false positives. He considers that acceptable only because essentially no existing interpretability technique meets that bar either.

Versus SAEs. Comparable — useful in the same settings, similarly limited. The J-lens is cheaper, easier, and has fewer degrees of freedom but is restricted to single-vocabulary-token concepts; SAEs capture more important concepts and more noise. The substantive claim: the J-lens emphasizes verbalizable representations, and verbalizability likely correlates with importance better than sparsity does. It is also closer to causal than most concept-direction methods, which are purely correlational — "if the model thought about this a bit more it would be likelier to say this token."

Variable interpretability, not algorithm interpretability. The lens reads features, not circuits — with the meta-token results as a partial exception, where the readout appears to expose a subroutine rather than a stored variable. The follow-up (Towards surfacing model algorithms with meta-tokens in the J-Space, 2026-07-20) is the sharpest version of the exception, and it separates the two things that "expose" can mean. On LCM prompts the token gcd fires in the workspace layers; the concept is causal (a probe recovers the GCD's value, CV-Ridge R² 0.34 → 0.85 across layers 8–12, and steering a difference-of-means GCD vector changes the answer as if the GCD were the swapped value) while the gcd J-lens vector itself is not. So the lens is doing something it is not usually credited with — naming which algorithm a causal variable belongs to — without the readout direction carrying the causation. Their framing, borrowed from Engels et al.: variable interpretability surfaces the quantities a model computes with, algorithm interpretability surfaces the procedure, and a non-causal readout can serve the second while failing the first.

Relation to other lenses#

Method	Per-layer map	Objective	Weakness the J-lens fixes
Logit lens	$J_\ell = I$ (identity)	none — assumes coordinates are shared across layers	Degrades in early/middle layers; the J-lens is the principled correction. The two agree closely in the last few layers
Tuned lens	learned linear map	trained to match the output distribution — correlational	On prompts with unverbalized intermediates it "skips ahead" to the output instead of surfacing the intermediate
J-lens	corpus-averaged Jacobian	causal (first-order effect on outputs)	—

The paper is candid that the logit lens still captures much of the same workspace structure, just less reliably (especially in earlier layers). The tuned lens is judged less useful than either.

The margin over the logit lens now has an external measurement, and it is thin. On Qwen 3.6-27B (Towards surfacing model algorithms with meta-tokens in the J-Space, appendix), correlating each lens's ranking of candidate tokens against the model's verbalizable ranking (final-layer logprobs): both lenses sit inside or near the same 95% null band through ~L30, the J-lens leads through the middle layers (a local peak of ≈0.45 at L41 against the logit lens's ≈0.15–0.25), and the two converge to ≈0.9–1.0 by L60–63. The authors' own summary is that the J-lens correlates better in early-to-middle layers "though not significantly" — which corroborates rather than contradicts the paper's ablation that single-token-restricted J-lens variants beat the logit lens only marginally, and independently confirms the last-few-layers agreement the table claims.

The two modes: read and write#

Reading. Three forms: the full ranked readout (softmax(W_U · norm(J_ℓ h))); a per-token probe (inner product of $h$ against one chosen J-lens vector, for measuring whether a specific concept is present); and sparse decomposition by gradient pursuit, solving for the $k$ J-lens vectors that best reconstruct $h$ — this yields a discrete inventory of active concepts rather than a ranked list, and underlies the capacity/occupancy results.

Writing. Steering (h ← h + α·v_t), ablation (project out $v_t$, or zero the top-$k$ J-space directions), and — the workhorse of the paper — patching in lens coordinates: given a source token $s$ and target $t$, read the coordinates $c = V^\dagger h$ for $V = [v_s\ v_t]$ and write back $h + V(\sigma(c) - c)$ with $\sigma$ swapping the two entries. Everything orthogonal to $\mathrm{span}{v_s, v_t}$ is left untouched — which is what licenses the causal claims: swap spider for ant and nothing else changes.

Why it is practically attractive#

• Cheap. One precomputed matrix per layer, computed once per model; a single matmul per layer at inference. No auxiliary training, no dictionary to learn. Nanda's independent accounting: precomputation is ~10 prompts of 128 tokens = 10 × d_model backward passes, "doable on frontier models"; his own scaling run on Qwen3.5-397B-A17B at n=4 took ~1 hour on 8×H200. Coding agents implement the method fairly well from the paper, though the output needs sanity-checking against the paper's Appendix 10.2 evals.
• Directly human-readable. Output is a list of words, not a feature index needing a further interpretation step.
• Uniformly applicable. Works on activations, on weights (attention head Q/K/V/O matrices, transcoder encoder/decoder directions), and on other methods' feature directions (SAE decoders, steering vectors, probe weights).

This makes it cheap enough to run at scale to flag transcripts for review — the practical argument for J-lens-based alignment monitoring.

"Computed once per model" has an expiry date#

The precomputation above is a one-time cost for a fixed model. An external drift measurement (Train the Model, Not the Reader: Decodability Supervision for Verifiable Activation Explanations, 2026-07, empirical) prices what happens when the model keeps training: linear readers matched to a training target model decay to near-chance within 1,500 continued-training steps (3/3 seeds) and read nothing across seeds (6/6 pairs), while the information itself stays exactly recoverable — a linear map fitted on ~7,000 paired activations, no labels required, restores native reader performance in 9/9 cases, with rotation alone recovering about half and full recovery needing a non-orthogonal change of rank ~16–64. Duan (2026) reports the same staleness-and-repair concurrently on 2B- and 7B-parameter models.

Those measurements are on linear probes at ≤160M, not on a J-lens fit, so this is an inherited risk rather than a demonstrated one. What makes it worth carrying: the paper's own generalization is that staleness applies to any frozen reader, and a corpus-averaged Jacobian composed with the unembedding is exactly that — fit once against one checkpoint's activations, then read. A lens released alongside a model that is still in post-training should be assumed stale until realigned, and the realignment is cheap and label-free, which makes it a procedure rather than a refit. It is distinct from the collapse failure in the same paper, where freshly retrained probes also fail and no realignment exists. The obvious untried experiment is a page's worth of compute: refit a lens on a later checkpoint, then compare it against transporting the earlier fit through a fitted affine map. Note this cuts the opposite way from the shelf-life worry below — drift within a training run is repairable; drift away from language-aligned representations is not.

Limitations (the paper's own)#

• Single-token vocabulary. The lens names exactly those concepts that have a single-token name. "Prompt injection" arrives as separate prompt and injection tokens; diffusely-tokenized abstractions may not surface at all. Swap failures concentrate where the source concept's lens vector was weakly active to begin with — and one reason it might be weakly active is precisely this. Appendix extensions derive multi-token J-lens vectors, but the authors "suspect they can be improved."

Two additions from Towards surfacing model algorithms with meta-tokens in the J-Space that change how this limitation should be read. First, its severity is tokenizer-dependent, not a fixed property of the method: the meta-tokens found in Qwen 3.6-27B are overwhelmingly Chinese, on the authors' reading because Chinese tokens carry more information per character, so a one-token-per-direction lens reaches abstractions through Qwen's vocabulary that an English-heavy vocabulary would leave unnamed. The same lens on a different tokenizer sees a different slice of the same workspace. Second, they conclude from a failed search that this is now the binding constraint on the whole approach — separate sentences in the original, worth keeping separate: "We think the binding constraint is the J-lens itself: each direction reads out exactly one vocabulary item, so the discoverable meta-descriptions are bounded by the computations that happen to be lexicalized as a single token in Qwen's vocabulary," and, on the next step, "We are excited to see if multi-token J-Lens methods can find them."

• A bag of concepts, not a structure. A readout of spider, legs, eight says the concepts are present but not how they bind. Whatever relational grammar the model imposes on workspace contents, this readout cannot see.
• Inconsistent interpretability. At some (position, layer) slots the top tokens are simply not interpretable, and the authors do not know whether that is Jacobian-averaging noise, multi-token concepts, or content they are failing to recognize. For the early-layer share of this failure the answer is now none of the three: it is error accumulated in the backward pass, and it is fixable — see the R-lens section below.
• First-order only. The linearization is a first-order approximation of a nonlinear network; it is a fixed context-independent map, trading exactness on any one prompt for a stable readout.
• Interventions are weaker than reads. Ablating a J-lens direction removes only the component of the concept that lies in it, and negative steering compensates by steering noise; so a null causal result is much weaker evidence than a null observational one. External evidence bears this out: the meta-token experiments in A Review of Anthropic's Global Workspace Paper found steering effective and ablation largely ineffective on the same vectors, and single-position steering failed where multi-position steering worked. Towards surfacing model algorithms with meta-tokens in the J-Space puts numbers under the same pattern with matched random-vector controls — on context disambiguation, negative steering drops the target behaviour to ~0.29–0.68 across three prompt types while ablation at coefficient 1.0 stays at the unsteered baseline (~0.79–0.99) and random-vector steering stays at ~0.77–1.00 — and states the mechanism the prediction implies: the lens direction approximates the true one, so projecting it out leaves a residual that a negative steer can cancel and an ablation cannot. This is the prediction-then-confirmation pair worth noting, since the prediction was published first and the confirmation is by the same group. It also has a methodological consequence they act on: where a lens vector is not causal, they fall back to a difference-of-means probe vector for the intervention and keep the lens only as the readout.
• A shelf life tied to language-aligned representations. Nanda's forecast, and the sharpest risk to the method's future: latent-reasoning models would make workspace reads more important (more computation happening per forward pass, hidden from the CoT), but heavy latent-reasoning training could drift internal representations away from language-aligned directions and break the technique outright. Same trend, opposite implications; unresolved.

R-lens: the early-layer noise was the backward pass#

The same MATS group (R-lens: Making J-lens More Faithful on Early Layers, 2026-08-05, empirical) built a drop-in replacement that keeps the J-lens fit unchanged and swaps only the backward pass, and the result is the strongest external correction to the method so far: the early-layer readouts everyone had been treating as evidence about the model turn out to have been evidence about the estimator.

The diagnosis. The J-lens is fit by backpropagating from the final-layer residual stream down to the readout layer, so error accumulates in proportion to the number of layers traversed — worst exactly where the readouts are worst. That is a property of the tool, not of the residual stream, and it predicts the "inconsistent interpretability" limitation above rather than being predicted by it.

The fix. Borrowed from Relevance Patching, itself built on Layer-wise Relevance Propagation (LRP): replace the raw local gradient with a relevance coefficient, which in practice means installing stop gradients where ordinary gradients break relevance conservation. Three rules do the work:

• LN-rule — treat the normalization denominator $\sqrt{\epsilon + \mathrm{Var}[x]}$ as constant, making the norm linear and preventing relevance collapse. Applied to the residual-stream RMSNorms.
• Identity-rule — detach the nonlinear factor of GELU/SiLU so the activation's backward pass is a per-element linear map ($\mathrm{GELU}(x) \to x \cdot \mathrm{stop_grad}(\mathrm{GELU}(x)/x)$; for SiLU the gradient becomes plain $\sigma(z)$).
• Half-rule — split relevance evenly across a multiplicative gate's two branches instead of double-counting through the bilinear product.

Left alone: all linear layers (the LRP 0-rule and autograd's gradient coincide), attention, and q/k norms. Because the change is only stop gradients, the forward pass is bit-identical and the overhead is negligible — which is what makes this a replacement rather than a competing method. On MoE models the rules extend to all routed experts, the routing is frozen (softmax/sigmoid scoring, router logits, gate projection), and a shared expert's output is multiplied by a swept constant to amplify its importance. DeepSeek-V4-Flash additionally needs its manifold-constrained hyper-connection (mHC) residual-mixing coefficients frozen, since its inter-sublayer state is a $4 \times d$ matrix rather than a vector.

How much it buys, and where it doesn't. Mean per-layer pass@10 (does the intermediate appear in the lens's top 10 at the target position), five eval categories modeled on the paper's own Appendix A.6, 8 models. The gap is concentrated in the first half of layers and grows with model size:

Model	R-lens (1st half)	J-lens (1st half)	R-lens (all)	J-lens (all)
DeepSeek-V4-Flash (284B/13B-active MoE)	0.23	0.03	0.29	0.16
Gemma-3-27B	0.10	0.04	0.21	0.13
Qwen3.6-27B	0.12	0.09	0.16	0.14
Qwen3.5-9B	0.09	0.07	0.12	0.10
Qwen3.5-4B (smallest dense)	0.05	0.06	0.08	0.09
Qwen3.6-35B-A3B (smallest MoE)	0.09	0.13	0.18	0.20

An 8× first-half margin on the largest model and a reversal on the smallest two. The authors state the trend (advantage increases with model size, absent for the smallest dense and MoE models) without explaining it, but their own diagnosis supplies the obvious candidate: fewer layers means less accumulated error for LRP to recover, so the correction has less to correct. That reading is not in the post — but if it is right, the R-lens advantage is a measure of how badly the J-lens was broken, not of how much workspace content a model has, which matters for how the scaling result on The Global Workspace in Language Models (J-space) should be read.

Individual readouts, for calibration on what "better" means here. On the typo "aganst", R-lens puts the correct "against" at rank 1 by layer 4 and holds top-10 continuously through L29, while J-lens never enters the top 10 at that position at any layer (its best anywhere is rank 16). On "The capital of the country where sushi originated is", R-lens has "Japan" at rank 5 by layer 2; J-lens's first top-10 hit is layer 14 (rank 3). On "The athlete Michael Jordan plays the sport of", "basketball" appears at layer 4 versus layer 20 (prose only, no figure). And the qualitative difference is not subtle: J-lens's early layers are full of what the authors call trash tokens — 锁定, 尷, ＊＊＊＊＊＊＊＊, euw, tav, zinho — while R-lens's early readouts show structure even when uninformative, returning 颜色的 ("of color") after "color" and fifth after "fourth".

The ablation is what makes this more than a prettier readout. On 30 multihop questions, project each lens's intermediate-token direction out of the activations at the penultimate prompt position, sample 8 completions before and after, and score relative accuracy loss with a GPT-5.4-nano autorater. Ablating only the first half of layers, R-lens directions cost substantially more accuracy than J-lens, logit-lens, or random-control directions on nearly every model — so the early-layer content R-lens reads is causally load-bearing, not merely readable. Two exceptions, both MoE: on Qwen3.5-122B-A10B all four arms including the random control sit near zero with heavily overlapping error bars (nothing is distinguishable), and on Qwen3.6-35B-A3B the J-lens bar is the taller one. Ablating all layers erases both exceptions — R-lens ≥ J-lens everywhere. Parse note: neither ablation chart prints value labels, so only the ordering and the near-zero/overlapping-error-bar readings are cited here; the bar magnitudes are gridline estimates and are deliberately not quoted.

Against a probe-defined ceiling. Train a linear classifier to separate 10 concepts within a category and find the earliest layer it reaches 0.99–1.00 accuracy: a crude upper bound on how early any lens could reasonably recover the concept. The chart's printed deficits (layers later than the bound at which the lens first reaches top-10 recovery, with the share of examples that never recover at any layer) put R-lens closer in every category — musicians +15 (15% never) vs J-lens +27 (18%), cities +6 vs +17 (3%), authors +5 (25%) vs +17 (28%), athletes +11 (8%) vs +19 (6%) — and on landmarks R-lens's deficit is −11: it recovers the concept eleven layers before the probe bound, which the authors flag themselves as a reason not to take the bound too seriously. The logit lens is nowhere close, at +36 to +54 with 16–81% never recovering.

The one place R-lens is not better, and it is the diagnostic one. The appendix measures MLP gain — how strongly the next MLP block amplifies a direction — the same experiment whose J-lens-versus-SAE-feature gap motivated the "what is the better basis?" question below. On Qwen3.6-27B, J-lens and R-lens directions are amplified similarly across the whole depth, tracking each other closely and rising above a standard MLP neuron's amplification only in the later workspace layers. So the R-lens is a better readout without being a better basis on the metric that raised the question: whatever the true workspace directions are, LRP does not find them any more than the raw Jacobian does. Both lenses even sit below random amplification in the middle layers where the MLP-neuron baseline is above it.

CKA, and a number collision worth not tripping over. R-space CKA shows roughly 2–3 distinct bands against J-space CKA's 4–5 on Qwen-3.6-27B — under the better lens the early/late discontinuity partly dissolves, which is the direct evidence bearing on whether the early layers are a real regime boundary or a lens artifact (→ The Global Workspace in Language Models (J-space)). Do not conflate this with the review's Qwen replication finding, which used the same digits to say something different: there, J-space on Qwen showed the paper's 4–5 bands with 2–3 of them overlapping rather than crisply partitioned. Same numerals, different claim.

The transported directions collapse — but that is the fit, not the transport#

An independent audit (jspace-replication: Independent Replication of Anthropic's Global-Workspace Paper on Small Open Models, July 2026) measured effective dimensionality of raw versus J-transported directions across a model ladder. On off-the-shelf models the transported directions occupy a narrower cone than the raw ones — the readout basis is more collapsed than the space it reads from, which is the geometric form of the worry that the lens only approximates the true workspace directions.

One model breaks the pattern, and it is the load-bearing one: a self-trained 124M GPT-2 reproduction (standard nanoGPT recipe, trained for an unrelated project and published beforehand, so not a hand-picked point) whose transported directions come out more isotropic than its raw ones — effective dimensionality 23.4 → 31.2. The non-collapse is robust across fit scale (150 → 1000 fitting prompts; 31.2 → 31.1) and backend (MPS → CUDA), and it is the only lens in the ladder the auditors fit themselves — refittable from the public weights, with both fitted lenses released.

The inference: since the transport is the same operation in both cases, the collapse seen elsewhere is a property of the fitted lens on a given trained model, not a mathematical necessity of composing an averaged Jacobian with the unembedding. Whatever narrows the readout is contingent — on the model, the corpus, or the training run — which makes it a thing that could in principle be fixed rather than a floor on the method.

Caveat: pre-publication README figures, and a 124M model is far outside the regime the workspace claims are made in — the counterexample proves the collapse is not necessary, not that the fits on real models are wrong.

Where it sits among interpretability tools#

The J-lens occupies the cheap-and-grounded end of the readout spectrum. Linear probes are cheap but supervised and correlational (one concept per probe). Sparse dictionary learning is unsupervised but expensive, and each feature needs a further interpretation step. At the expressive end sit free-text verbalizers like the natural-language autoencoder (NLA) of White-Box Activation Monitoring, which can articulate multi-token concepts and relations the J-lens cannot — at higher cost and with confabulation risk. The authors are explicit that the J-lens is complementary, not competitive, with these methods; in the head-to-head automated-audit benchmark it matches the NLA on four of six cases and loses on the two that demand a higher level of abstraction than a single-token readout can carry.

Availability#

Open-source implementation at github.com/anthropics/jacobian-lens; interactive readouts on open-source models hosted on Neuronpedia — since July 2026 including J-lens suites for Qwen 3.6 27B (qwen3.6-27b/jlens), released alongside the external replication.

The R-lenses above ship with their matched J-lenses at huggingface.co/camilablank/workspace-lenses — the only released artifact that lets the two backward passes be compared on the same fit.

An alternative exact formulation is circulating: James Golden's detached Jacobian — gradient-detaching the nonlinear components makes inference an exact input-dependent linear transform, whose singular vectors decode input and output tokens and steer concepts effectively; demonstrated on Qwen 3 14B and Gemma 3 12B. Reported in a comment on the review rather than a paper, so treat as an unvetted lead — but it is a direct candidate answer to the "what is the better basis?" question below.

Connections#

• Evaluation Awareness & Grader Gaming — a cheap lens-derived eval-awareness score (AUC 0.853 vs NLA-flagged positions) that ranks eval suites above real deployment traffic
• Agentic Misalignment (AM) — the blackmail eval's deliberation, read before the first output token
• Agentic Prompt Injection — silently-recognized injections surface in the lens (fake, injection, poison) even when the model never mentions them
• Reward Hacking — the lens reads a standing deception signature in reward-hacking model organisms on ordinary prompts
• Model Organisms — the ground truth behind those case studies. The 54-organism benchmark tests the plain logit lens this method corrects, not the J-lens — but it finds logit-lens scores swinging with the organism's construction recipe and sitting at the noise floor at mid layers, so the J-lens's own organism results inherit the transfer caveat even though its margin over the logit lens is untested there
• The Assistant Persona in the Workspace — the base-model-vs-post-trained comparison the lens makes possible
• Access-Consciousness Indicators in AI — the instrument that lets consciousness-theory indicator properties be checked against a concrete structure
• The Global Workspace in Language Models (J-space) — what the lens found: the J-space and its workspace-like properties
• White-Box Activation Monitoring — the family this joins; the NLA verbalizer is its closest and most-benchmarked rival
• Interference Weights — the same programme run on weights instead of activations, and the sharpest available statement of the basis problem. The per-path decomposition there is the plain logit lens applied path-by-path; its conclusion is that a better basis, not a better metric, is what stands between the field and a readable global description — which is this page's open question arriving from the weight side
• Chain-of-Thought Monitorability — the lens reads the reasoning channel the chain of thought structurally cannot show
• Automatic vs. Flexible Cognition in LLMs — the selectivity result, established by J-lens ablation
• Internal Signatures of Misalignment — the alignment-auditing application
• Automated Behavioral Audit — an auditing agent equipped with a J-lens tool, benchmarked against NLA- and SAE-equipped auditors
• Counterfactual Reflection Training — the lens is what makes the training technique's mechanism observable
• Wes Gurnee, Jack Lindsey — the method's originators

Open Questions#

• Can multi-token J-lens vectors be made good enough to remove the vocabulary restriction — and how much of the "workspace" is currently invisible because of it? Partially answered: Towards surfacing model algorithms with meta-tokens in the J-Space moves the second half and leaves the first untouched. It ran the first systematic search for lens-visible content that indexes computation rather than variables — 20+ heterogeneous datasets at 500–1000 samples each, every vocabulary token's rank recorded at every (layer, position) cell, four filters (prevalence, position type, surface-predictability probe, LM-judge dissociation) — and found "less than we hoped": a handful of non-obvious survivors, five of which are reported with no rates and no causal test. The authors' own reading is that the one-vocabulary-item-per-direction limit is now the binding constraint on the method, and they add a genuinely new fact about the constraint's shape: its severity is tokenizer-dependent, since Qwen's information-dense Chinese tokens are what made the three validated meta-tokens nameable at all. What this is not is a measurement of how much is invisible. A null search cannot separate "the meta-concepts exist but are not lexicalized" from "the meta-concepts are rare," the authors say only that they "suspect" the former, and no multi-token lens was built or tested — so the question's first half is exactly where it was, and the second half now has a sharpened hypothesis rather than a number.
• The J-lens reads the workspace's contents but says nothing about how content gets in. What is the model's analog of attentional selection?
• The highest-J-kurtosis SAE features are amplified more strongly by MLPs than the J-lens vectors themselves — evidence the lens only approximates the true workspace directions. What is the better basis? Partially answered: A Review of Anthropic's Global Workspace Paper surfaces one concrete candidate — the detached-Jacobian formulation above, which is exact rather than first-order — but it is a comment-thread report on two small open models with no head-to-head against the averaged J-lens, so the question stands. jspace-replication: Independent Replication of Anthropic's Global-Workspace Paper on Small Open Models adds a geometric constraint on the answer: the transported-direction collapse is a property of the fitted lens, not of the transport (a self-trained 124M control goes the other way, 23.4 → 31.2), so a better basis is not ruled out by the mathematics — but no such basis is measured. R-lens: Making J-lens More Faithful on Early Layers then supplies the first candidate that is actually measured head-to-head against the averaged Jacobian on the question's own metric — and it fails on exactly that metric while succeeding on every other one. The R-lens beats the J-lens on early-layer readout quality, on causal salience under ablation, and against a probe-defined recovery ceiling; but its directions are amplified by MLPs no more strongly than the J-lens's, so the amplification gap that framed this question is untouched. Sharpened, then: the better basis is not simply "a less noisy backward pass," because a demonstrably less noisy backward pass leaves the gap exactly where it was.
• When a readout names an algorithm (gcd on LCM prompts, stack on balanced parentheses, xor on find-the-repeat), is the lens reading the procedure the model runs, or an association learned from human text about those problems? Towards surfacing model algorithms with meta-tokens in the J-Space raises the confound against its own result and leaves it — the GCD case has causal backing for the variable, the other two have nothing. A settling experiment exists: find or train a model that solves a task family by a method other than the one its literature names, and check whether the lens surfaces the textbook name anyway.
• Does the J-lens work because it reads verbalizable representations, or because constructive interference during training converges concept directions onto output-token directions regardless? The two predict the same readouts today but diverge on latent-reasoning models — the second survives representational drift away from language, the first does not.

Sources#

• Verbalizable Representations Form a Global Workspace in Language Models — Methods (the Jacobian lens, the J-space, comparison to related techniques, technical details); Related work (lens methods, linearization, comparison to other interpretability tools); Limitations

• A Review of Anthropic's Global Workspace Paper — "Why does J-Lens work? First-principles reasoning" (working memory, consistent directions, why tokens, why Jacobians rather than regression, approximation costs, vs SAEs); "Is J-Lens useful?" (SAE comparison, CoT complementarity, latent reasoning); comment section (Neuronpedia release; James Golden's detached-Jacobian formulation)

• Towards surfacing model algorithms with meta-tokens in the J-Space — Bhatia, Blank, Nanda, LessWrong, 2026-07-20 (MATS), empirical. Sections: Motivation (variable vs. algorithm interpretability, after Engels et al.); What are "meta-tokens"; Case studies → GCD meta-token (linear-probe decodability of a computed quantity; non-causal lens vector with a causal concept; swap methods) and Interpretative meta-tokens (the ablate-vs-steer gap and the residual-term account of it); Searching for more meta-tokens (the binding-constraint conclusion); Appendix (verbal-report correlation vs the logit lens on Qwen 3.6-27B; CKA). Chart-only figures are flagged where cited

• R-lens: Making J-lens More Faithful on Early Layers — Blank, Bhatia, Nanda, LessWrong, 2026-08-05 (MATS), empirical. Sections: Motivation (backward-pass error accumulation as the early-layer diagnosis); What is RelP and how do we apply it (LN-rule / identity-rule / half-rule); Methods (dense and MoE modification lists, mHC handling for DeepSeek-V4-Flash); Results → quantitative pass@10 across 8 models, qualitative first-appearance-layer examples, trash-token coherence grids, causal ablations, probe-defined upper bound, CKA; Appendix (MLP-gain amplification; per-model pass@10 breakdowns). Figure-only numbers: the aggregate pass@10 bars, the probe-bound deficit labels, and the Verona/Einstein chart titles carry printed values and are cited exactly; the two ablation-accuracy-delta charts print no labels, so only their orderings and near-zero verdicts are used here. The MLP-gain axis is log-scale with per-gridline ×10ⁿ labels and is cited qualitatively only. Verona example: prose says J-lens rank ">1000" while the chart title prints "J-lens min 242" — not a contradiction, a scope difference (see the note in wiki/sources.md)

• Train the Model, Not the Reader: Decodability Supervision for Verifiable Activation Explanations — Hiskias Dingeto (StackOne Technologies), arXiv 2607.20379, 2026-07-22, empirical, single author. Used here only for §4 + App. H (frozen-reader drift to near-chance within 1,500 continued-training steps, 3/3 seeds and 6/6 cross-seed pairs; label-free affine realignment restoring 9/9; the rank decomposition; the drift-versus-collapse separation) and its Related Work statement that staleness applies to any frozen reader. The paper never tests a J-lens — its readers are linear probes on an 8-layer sandbox transformer and on Pythia-160M — so the transfer to a precomputed Jacobian fit is an argument, not a measurement. Its main subject, an audit of reconstruction-scored natural-language activation explanations, is treated on White-Box Activation Monitoring; see that page's Sources entry for the full parse warning (Table 11 row-bleed, cited nowhere)

• jspace-replication: Independent Replication of Anthropic's Global-Workspace Paper on Small Open Models — tao-hpu, GitHub README + results/cone_*.json, fetched 2026-07-30; paper forthcoming on arXiv. Sections: The self-trained 124M control (raw vs J-transported effective dimensionality 23.4 → 31.2; robustness across fit scale 150 → 1000 prompts and MPS → CUDA backend; selftrained-124m-lens-v1 release); Setup (pinned upstream commit; ~100 fitting prompts sufficient per upstream)