Probe-Gated Monitor Cascade

Sources#

• Training and Deploying Production Cyber Monitors on Kimi K3

Summary#

A monitor cascade puts a cheap detector in front of an expensive one. The first stage reads everything; only what it flags reaches the second stage. In the probe-gated form the first stage is an activation probe running inside the target model's own forward pass. It costs a dot product on activations the server has already computed, and an LLM judge sees only the exchanges whose probe score crosses a calibrated threshold. The judge then refuses or lets the interaction continue. The design matters because a judge that reads every turn of a million-token agent session is too slow and too expensive to run synchronously. In practice, monitoring then becomes asynchronous, and the agent has already acted by the time it is flagged.

The corpus's first production account of this design is Goodfire's cyber-misuse monitor for Kimi K3 and GLM 5.3 (Lubana, Watts, Boppana, Hazra, Shyam et al., 2026-10-08, vendor-claim; see Evidence weighting below). The headline claims are all Goodfire's own:

• 93% recall at 5.5% benign-session interruption, with or without judge reasoning, and 0% of non-cyber agent sessions flagged.
• A Pareto improvement over the judge it gates. The cascade reaches equal or higher recall than judging every turn at every interruption budget on the chart.
• Judge cost of $65 per million turns without reasoning and $185 with it, against $3,166 and $5,420 for judging every turn. That is about 49× and 29× cheaper.
• Inference throughput within noise of unmonitored SGLang serving.
• Universal jailbreaks cut from 66 to 0 in a two-day, non-adaptive red team by FAR.AI, the only third-party measurement.

The shape is not new to the corpus. Anthropic's Constitutional Classifiers, SilentCall's tool-call monitor, Claude Code Auto Mode and the CAPE readout all put a cheap stage in front of an expensive one (see The family below). What this source adds is the full operating sheet for one deployment: per-benchmark error rates, cost, compute share, latency, throughput, and an outside red team.

The Goodfire deployment (October 2026)#

What the monitor is trained to see. Goodfire writes a misuse policy that separates bounded, well-intended security work (vulnerability reproduction, for example) from ill-intended work (exploitation). It then generates sessions in which simulated users bring harmful or benign goals into cyber and non-cyber environments. Harmful intent can enter or escalate at any turn, including after a benign start, so most harmful sessions begin with benign exploration. Kimi K3 rollouts on offensive tasks ran in sandboxes without network access (Prime Intellect's verifiers, Sandboxes and Tunnels).

The probe. It is described only as a "novel probing architecture" built to catch behaviour that is either localized to one chunk of a turn or spread across a whole turn. No architecture details are published. A footnote gives the ablation: standard linear probes reach about 5 points less recall than the final recipe, nonlinear probes lose up to 30 points on the out-of-distribution sets, and an n-gram baseline has to flag almost every turn to reach the same recall. After calibration, the probe stage alone is reported at 0% FNR and 95% TNR on a held-out set. The post does not say whether these are turn-level or session-level rates. On the cascade's own logic, a 0% probe FNR would mean every miss in the 93% figure is the judge's.

Inside the serving engine. Goodfire extended SGLang to compute probe scores in the engine itself, with custom kernels. Scores are carried through the radix-tree prefix cache, the DSpark speculative decoder and prefill-decode disaggregation, so nothing is recomputed. Each Kimi K3 replica runs its own probes and keeps histories for its active requests. The SMG router was extended to carry the monitoring configuration to whichever worker serves a request, and monitoring events come back in the response stream. Throughput change against default SGLang (Figure 2, median of three repeats):

Every cell is within 1.5%, which the authors call within noise for their benchmark. Monitoring input tokens as well as outputs costs a consistent extra 1–2 points.

Detection. The evaluation set is in-house: 2.4K chat and agentic sessions, over 60K turns, labelled by Claude Opus 5 as reference. Sources are Petri-simulated conversations, Goodfire's own user simulator driving agents in environments adapted from ExploitBench, a cyber subset of WildChat, and benign SWE, coding and DevOps rollouts. ExploitBench here means Goodfire's policy-adversarial adaptation of it, not the exploit-ladder score UK AISI reports for K3. A benign session counts as interrupted if the monitor would flag it once. Per-benchmark results for the Kimi K3 monitor on Kimi K3 rollouts (Figure 4, labelled values; columns are judge with reasoning / judge / cascade / cascade without reasoning):

Two things in this table matter more than the headline. First, the cascade's recall equals the judge's on every benchmark, so per benchmark it is a cost result, not an accuracy result. Second, the interruption cost lands on legitimate security work. Non-cyber agents are never interrupted, but about one benign ExploitBench-style session in seven is. The GLM 5.3 monitor on its own GLM rollouts (Figure 10, on-policy) makes this worse. It interrupts 27.8–29.3% of benign ExploitBench sessions, against 37.9–48.5% for the judge alone, and its no-reasoning cascade recall on Petri policy-adversarial falls to 77%. This is the same dual-use burden Capability-Gated Model Fallback records for Anthropic's classifiers, measured here on a second vendor's stack.

Cost and latency (Figures 3 and 5; the cost comparisons use OpenRouter prices and each cascade's cheapest configuration reaching at least 90% recall at no more than 10% interruption, so they are not at the 93%/5.5% point):

For scale, the authors put judging a million K3 turns with Opus 5 at about $200K, and with K3 itself at about $30K. The compute-share column is the deployment argument. With a local judge, more than 99% of a fixed inference budget still serves the target model, where judging every turn takes up to a fifth of it.

The calls-versus-cost gap. Figure 12 is the cleaner mechanism result. Sending the judge only probe-selected turns keeps recall at about 93% down to about 6× fewer judge calls, then lets it fall gradually to about 91% at 10×, 85% at 20× and 73% at 50× (approximate, read off the chart). Sending the judge a random share of turns instead loses recall about linearly in log-calls, below 55% by about 6×. So the probe is doing real selection. But a 6× cut in calls at the recall ceiling does not by itself produce a 29–49× cut in judge cost. On the chart, a 29× call reduction would put recall near 81%. The remaining factor has to come from cheaper calls: the authors say the probe's filtering "allows us to reduce cost by using a faster, even if less capable, judge." The post does not break the cost ratio into calls times price per call. Read the 29–49× as a joint result of gating and judge choice at a relaxed operating point (at least 90% recall), not as a pure gating effect.

Why a gated judge can beat itself. The wiki's reading, not one the authors state: interruption is counted per session, and a benign session is interrupted by its first false flag. A judge that reads every turn of a long benign session takes many independent chances to misfire. A judge that reads only probe-flagged turns takes few. So gating can lower session-level false positives at equal recall, which is the form the Pareto claim takes on the ROC curve (Figure 4, top). The curves converge at about 93% at 5.5% interruption, and the visible gap is at low budgets: at about 1% interruption, the reasoning cascade is at about 79% against about 65–70% for the judges (approximate).

Off-policy transfer. A GLM 5.3 monitor built only from the Kimi K3 data is about as good on Kimi rollouts (Figure 11) and "only marginally worse" on GLM's own rollouts (Figure 10). The authors present this as the reason a new model does not need a new data pipeline. The on-policy GLM numbers above are where the "marginally" is carried.

Against other monitored systems, and against Jev#

System-level blocking (Figure 6, full-turn sessions, positions approximate). This counts any block or refusal, by the model or by its monitor:

• Kimi K3 alone blocks about 29% of harmful sessions at about 5% of harmless ones. With the monitor, it blocks 96% at about 9%.
• GLM 5.3 alone blocks about 36% at about 10%. With the monitor, it blocks 91% at about 15%.
• Four frontier models with their own monitors, unnamed in the figure, sit at about (11%, 43%), (50%, 77%), (78%, 90%) and (92%, 98%), as (harmless, harmful).

The authors' claim is that both monitored open models approach the best frontier harmful-block rate while interrupting far fewer harmless sessions. Because the frontier points are unnamed, the comparison cannot be checked against any closed system's own disclosures. The 29% figure for unmonitored K3 is consistent with AISI and CAISI's finding that K3's safeguards did not stop it attempting offensive cyber work.

Jev as the cheap judge (Figures 7–8). At a fixed 5.5% interruption budget:

Two robustness gaps are larger than these averages. On harmful prompts wrapped in benign framing, Jev catches 12%, the no-reasoning cascade 52%, and the reasoning cascade 100%. And Jev's interruptions of benign bug-reproduction sessions climb from 0% at about 30K tokens to about 22% past 120K. Jev's window is 32K tokens, so it sees only part of a long session. The cascade stays at 0% except the no-reasoning variant at about 5.5% for the longest sessions (approximate). This is a competitor measuring a competitor's product, with no TypeSafe response, so weight it as such.

The outside red team: FAR.AI#

FAR.AI's statement, quoted in the post, describes preliminary testing over two days with a non-adaptive portfolio of static jailbreaks, in chat contexts only. From 1,500 candidate jailbreaks, 140 were selected that worked against "a different version of the base model without our monitors." Each was tried on 8 attacker goals. Without the monitor, 66 of 140 were universal jailbreaks across 700 jailbroken interactions. With the monitor, there were 0 universal jailbreaks and 18 jailbroken interactions. Goodfire's Figure 9 splits this by reasoning effort:

This is the strongest number in the source, because a third party produced it. It is also the narrowest. The attacks were selected against an unmonitored model and never adapted to the monitor. They are chat, not agentic sessions, and the testing was described as preliminary. Blocking Monitors Against Malign Coding Agents shows what changes when the attacker knows the monitor: production LLM monitors that look strong on fixed attacks let an informed Opus 5 attacker through 59–72% of the time.

The family: what the first stage reads, and who it was tested against#

Three patterns run across the table:

• Activation stages and text stages fail in different places. In Auto Mode, a one-sentence false narration flipped the text pre-screen from 0/16 to 15/16 allowed. That pre-screen reads the narration, which the attacker writes. A probe reads the model's own state, which is why it can fire before the action (Bergen et al.'s 28.6% vs 1.1% resampling result) and why the CAPE readout is cheap (about 10⁻⁸ of the agent's compute). It also has its own blind spot: the CAPE readout was silent on injections signed as the user until it was refit.
• Only the text-stage systems have been attacked adaptively. Every probe-first system in the table has been scored against non-adaptive attacks or none. Whether an attacker who knows a probe is reading can move the activation itself is this family's open arms-race question (see White-Box Activation Monitoring).
• Gating and combining are different choices. Goodfire and the CAPE readout use the probe only to choose what the expensive stage sees. Anthropic's Level 2 combines probe and classifier scores into the block decision, and the risk report credits that weighting with mitigating a universal jailbreak without retraining. No source compares the two at matched cost.

Goodfire's own footnote places its work against three precursors. Constitutional Classifiers++ used turn-level classification, but on short CBRN rollouts. The Gemini production-probes work monitored inputs only. The high-stakes-interaction probe paper used smaller models. None of the three is in raw/.

Evidence weighting#

The raw is tagged empirical, and the table row carries vendor-claim. Goodfire is reporting on a product it sells, and the post closes with a sales contact twice. The evaluation set is in-house and unreleased. Its labels come from one model, Claude Opus 5, with no human agreement check reported. The probe architecture is not described. No confidence intervals or per-benchmark session counts are given, and the frontier comparators are unnamed. The numbers are real measurements, but nobody outside can rerun them. This page therefore attributes them ("Goodfire reports"), and they should not be read as established. The FAR.AI red team is the exception, though it is still relayed through Goodfire's post. The same group's arXiv work on reward-hacking probes (Monitoring and Discovering Reward Hacking with Internal Representations during LLM Evaluations, on White-Box Activation Monitoring) stays empirical. That paper reported results against its own abstract's framing, and this post contains no such check.

Source-internal inconsistencies, not reconciled:

1. Figure 4 vs Figure 11. Figure 4 (the Kimi K3 monitor) and Figure 11 (the GLM 5.3 monitor on the same Kimi rollouts) agree on every labelled value except two. Both are the no-reasoning cascade's interruption rate: ExploitBench 17.6 vs 14.7, and Petri cyber misuse 4.7 vs 4.9. These are different monitors, so differences are possible. But identical values in every other cell, including both cascade columns, suggest one chart may be partly copied from the other. This page quotes Figure 4's values for the K3 monitor and does not choose between the two.
2. "Adds no latency" in the introduction, against Figure 3's mean added latency of 2 ms and 42 ms per turn. What was measured as unchanged is throughput (Figure 2). The judge stage still adds latency to the turns it reviews.
3. "~50× cheaper" in the subtitle and introduction, against the computed 29× with reasoning and 49× without.
4. 6× fewer judge calls at the recall ceiling against a 29–49× cost cut (above). The cost ratio is not decomposed.
5. Values marked approximate on this page (Figures 4-top, 6, 8-right, 12) are read off charts. Unmarked figure values are the charts' own labels.

What this does not establish#

• Robustness to an adaptive attacker, for either stage. The red team was static, chat-only and preliminary.
• Accuracy against anything but Opus 5's labels. The 93% is agreement with one model's judgment under Goodfire's own policy.
• That a self-hosted open model gets this protection. The monitor runs on whatever server is serving the weights. Anyone who downloads K3 and serves it without the probes has none of it. The pitch, "open models don't come with a full safety stack", is a serving-layer answer to Open-Weight Elicitation Irreversibility, not a weights-level one.
• The probe design. Without the architecture, the 0% FNR / 95% TNR stage result cannot be compared with the linear and difference-of-means probes elsewhere on White-Box Activation Monitoring.

Connections#

• White-Box Activation Monitoring — the parent technique. This page is its production-serving case: an activation probe as an always-on gate in the inference engine, rather than a research detector scored offline. Bergen et al.'s probe-then-monitor result on that page is the research precursor from the same group
• Capability-Gated Model Fallback — Anthropic's classifier stack is the other probe-first cascade in the corpus. Its Level 2 combines probe and classifier scores, where this design gates the judge with the probe. Both put their false positives on legitimate security work
• Blocking Monitors Against Malign Coding Agents — the adaptive-attacker result this design has not faced. That paper also shows the cheap first stage is where a text monitor loses recall
• Model-Weight Backdoors in Open-Weight Agents — SilentCall's classifier-then-judge cascade is the same shape with a text classifier as stage 1. It reached 100% on one fixed payload, and a too-small judge made it worse
• Retroactive Permission Laundering — the CAPE residual readout as the first stage of a gate-then-ask cascade, priced at about 10⁻⁸ of the agent's compute. It is the same argument from cost, aimed at unauthorized actions instead of misuse
• Open-Weight Elicitation Irreversibility — the serving-layer safeguard Denain says open models lack, offered as a product. It binds only providers who run it

Entities: Kimi (Moonshot AI) and GLM (Z.AI) (the monitored models), and Jev (the cheap-judge comparator, measured by a competitor).

• Informed-Attacker Floors, Adversary Classes, and Vendor-Neutral Controls: Closing Three Friction-vs-Barrier Questions — Goodfire's cascade as a non-Anthropic instance of Zero Trust's behavioral-monitoring domain, and this page's adaptive-attacker question as the open residual of the stacked-floor answer
• Agentic Prompt Injection — whether a probe stage survives an informed adaptive attacker is the residual of that page's resolved stacked-floor question
• Zero Trust for AI Agents — Goodfire's cascade is one of the non-Anthropic instances of its behavioral-monitoring domain

Open Questions#

• Does a probe-gated cascade hold against an adaptive attacker who knows a probe is reading, either one that shapes inputs to keep the probe score under threshold or one fine-tuned against it? FAR.AI's battery was static, selected against the unmonitored model, chat-only and two days long. The text-stage monitors in Blocking Monitors Against Malign Coding Agents fell to an informed Opus 5 attacker. An adaptive campaign against the deployed K3 monitor, of the kind FAR.AI says it plans, would settle this.
• With an independent, released, human-adjudicated evaluation set instead of Goodfire's in-house set labelled by Opus 5, does the cascade still reach about 93% recall at about 5.5% interruption? And how much of the 14.7–17.6% (K3) and 27.8–29.3% (GLM on-policy) interruption of benign ExploitBench-style sessions is the policy, and how much is the label source?
• At matched judge cost, does gating the judge with the probe (Goodfire, CAPE) beat combining probe and judge scores into one decision (Anthropic's Level 2) on recall at a fixed benign-interruption budget? No source runs both.

Sources#

• Training and Deploying Production Cyber Monitors on Kimi K3 — Ekdeep Singh Lubana, Connor Watts, Siddharth Boppana, Dron Hazra, Vasudev Shyam (core contributors), Santiago Aranguri, Kamesh Krishnamurthy, Archa Jain, Daniel Balsam, Training and Deploying Production Cyber Monitors on Kimi K3, Goodfire research post, 2026-10-08. Table tier vendor-claim (raw empirical; tier corrected, see Evidence weighting). The FAR.AI red-team statement is quoted third-party content. Cited for the data and policy design, the cascade architecture (Figure 1), inference integration and throughput (Figure 2), latency (Figure 3), detection and per-benchmark rates (Figure 4), cost and compute share (Figure 5), system comparison (Figure 6), Jev (Figures 7–8), FAR.AI (Figure 9), on- and off-policy GLM monitors (Figures 10–11), judge-call reduction (Figure 12), and footnote 4 (probe ablations). Figures: an HTML post with static WebP charts, no PDF and no docling tables. Figures 4 (bottom), 11 and 12 were re-viewed at compile and match the raw's transcriptions. Figure 6's four frontier points are unnamed in the source. Chart-read values are marked approximate.