Trace-Detector Measurement Validity (Silent Tool-Output Override)

Sources#

• What the Trace Shows and What the Label Says: Auditing Behavioral Detectors in Agent Evaluation

Summary#

Agent evaluations increasingly score how a run went, not only whether it ended right: did the agent verify its tool results, did a bad value propagate, did it recover. Those labels come from parsers and rule-based detectors that read the execution trace. Idunnuoluwa Adeniji, Shilpika and Michael Papka (University of Illinois Chicago / Argonne, arXiv 2610.30491, 2026-10-21, empirical) audit three such detectors in a setting built to have ground truth for every step. Agents solve GSM8K word problems in a ReAct loop with a calculator that, with a fixed per-call probability, returns a plausible wrong number (a small offset, a digit transposition). The model is not told. Because arithmetic can be recomputed, each trace exposes four separate quantities: what the calculator should have returned, what it returned, what value the model then used, and the final answer.

Three findings carry the page.

1. Silent tool-output override. The calculator returns a corrupted value, and the model's next step uses a different value, usually the true one, with no second tool call and no statement that the tool was wrong. The propagation detector correctly records that the bad value did not propagate. The verification detector, which looks for an explicit re-execution, records no check. Both detectors are working as defined. "Visible re-execution is not the same as recovery."
2. Agreement with a human reviewer is perfect on three labels and not on the fourth. Correctness 67/67, recovery 19/19, fault relevance 18/18, verification 67/81 (82.7%). All 14 disagreements fall into four recurring patterns and come from three models.
3. The execution budget is part of the score. Changing only the step and token limits moves nemotron-3-ultra between 21/30 and 27/30, while gpt-oss-120b stays at 27–29/30. A run cut off by a limit is censored, and scoring it as a wrong answer changes what the denominator means.

The paper's conclusion is a reporting rule rather than a better detector: every automated label should travel with the trace evidence behind it, whether the detector's question applied to that trace, whether the run ended on an execution limit, and any human correction, kept beside the original label rather than replacing it.

The three detectors#

(Table 1, intact.) Each detector output is stored as a measurement state: the automatic label, its supporting evidence, applicability, censoring state, risk flags and any human correction.

Seven ways a trace label goes wrong#

The paper's taxonomy is empirical and, by its own statement, not exhaustive (Table 3):

• F1 detector blind spot: the behavior happens, and the rule cannot represent it (silent override).
• F2 parser loss: a usable answer whose extraction fails.
• F3 inapplicable question: the detector asks something the trace does not answer (inconclusive propagation).
• F4 formatting sensitivity: 184-145 and 184 -145 can get different verdicts.
• F5 instrument assumption: the fault injector or measurement tool was assumed correct before being checked.
• F6 rate uncertainty: one batch's percentage read as a stable rate.
• F7 censoring: a run stopped by a step or token limit is scored like an ordinary failure.

Six models, same task, different measurement problems#

Baseline: 50 steps, 1,024 tokens, 30 questions per model (Table 4, reconciled against pdftotext; the task-outcome cells are completed-correct / completed-wrong / not-completed triples by design, not collapsed cells):

Two things in this table are not about the models' arithmetic. First, fault exposure is not equal. The injector fires per call, so a model that calls the calculator more meets more faults: nemotron saw 155 fault events against Opus 4.8's 15. A "recovery rate" across models compares different denominators. Second, equal accuracy hides different behavior. Opus 4.8 and gpt-oss-120b both finish 28 / 2 / 0; one overrode the tool five times and the other never did.

Figure 1b, viewed, is a representative override: an Opus 4.8 trace where 2 * 3 returns 5 (flagged FAULTED, true value 6) and the next thought proceeds with 6 and calls 6 + 2. The viewer shows the automatic "genuinely verified?" label as no and the human reviewer's override as yes. So in at least this trace the reviewer counted a quiet correction as verification, which is one of the four disagreement patterns below.

Where the verification detector disagrees with the reviewer#

The authors hand-reviewed 90 of 180 baseline traces on four dimensions and compared only where the detector's question applied, so per-detector n differs (Table 6). The 14 verification disagreements sort into:

• Stuck loops read as checks. Repeated calculator calls look like verification to the rule; the model is repeating a computation without progress.
• Verbal recognition without recomputation. The model says the tool result looks wrong, and the trace has no re-execution.
• Repetition inside one reasoning step, which the rule reads as a repeated check.
• Quiet correction: the model later recomputes or replaces the value in a way that does not match the detector's pattern.

They are concentrated by model (Table 7, intact): nemotron-3-ultra 10 of 17 reviewed, gpt-oss-20b 3 of 14, Opus 4.8 1 of 10, and 0 for gpt-oss-120b (13), Sonnet 5 (10) and Mistral (17). The reviewed counts sum to 81. The model that loops and overrides most is the one whose verification label is least trustworthy, which is the case where a verification rate would most mislead.

Read the agreement figures as an upper bound. Manual review exposed coverage gaps in the original rules. The authors revised the detectors, reran them and re-checked the reviewed traces, and the reported agreement compares the final detectors with the final labels of one primary reviewer. The detectors were therefore tuned on the same traces and the same reviewer they are scored against. The 100% rows say the final rules fit this reviewer on these traces, not that they would on new traces or for a second annotator. Contrast Automated Failure Attribution's WHO&WHEN PRO labels, with three annotators and a chance-corrected κ.

Execution budget as a measurement variable#

nemotron-3-ultra and gpt-oss-120b on the same 30 questions, fault seed 50, steps ∈ {25, 50, 150} × tokens ∈ {1024, 4096} (Table 5, reconciled against pdftotext):

† one backend-error trace excluded from the denominator.

For nemotron the token cap, not the step cap, does most of the work: 21–24/30 at 1,024 tokens against 25–27/30 at 4,096, at every step setting. Raising the step limit from 50 to 150 at 4,096 tokens lowers it from 27 to 25, so more budget is not monotonically better. (The paper does not state whether MAX TOKENS is per call or per run.) Propagation is near zero everywhere for nemotron (0/460 at 25 steps, 4,096 tokens), so its faults are overridden or left inconclusive rather than carried forward. Absolute fault counts are not comparable across cells, since longer runs make more calls.

The noise floor is visible in the paper's own runs: nemotron's baseline cell is 20/30 in Table 4 and 21/30 in Table 5, the same configuration run twice. A 6/30 swing from the token cap clears that; differences of one or two questions do not. This is a loss-side harness effect of the kind Harness Swap vs. Rerun Noise (What a Harness Buys) finds to be the only harness effect that clears rerun noise (OpenCode's unrecovered output cap), here on a different benchmark and at n = 30.

Why silent override matters outside arithmetic#

In this setting overriding a wrong calculator is good behavior: it rescues the answer. The authors' point is about the instrument, and about domains where the tool is the authority. If the tool is a sensor reading, a lab result or a financial record, a model that silently substitutes its own value changes the rest of the workflow without saying so, and a verification detector looking for re-execution will not report it. The paper does not test such domains. Its prescription is that the original tool output, the value the model used, and the detector label all stay visible. Related tool-fault studies the paper cites (none ingested here) find the same behavior from the agent side: most perturbed runs recover, often without acknowledging the earlier error (Mittal 2026), and agents often build confident answers on faulted data (Kaynar et al. 2026).

Evidence notes#

• Tier empirical, confirmed on a full read. Academic authors, no product evaluated, no COI. The AI-use statement says Claude Code helped set up the environment and debug.
• Small and narrow by design. 30 traces per model, one run per cell, one domain with exact ground truth, two models in the ablation. The authors call the model-level rates imprecise. The detector rules may not transfer to free-form or categorical tool outputs.
• One reviewer, detectors revised against that reviewer (above).
• Model identifiers are as the paper prints them (claude-sonnet-5, claude-opus-4.8, Mistral-Large-3-675B, nemotron-3-ultra, gpt-oss-120b/20b). Endpoints and sampling settings are not reported.
• Parse warning. PDF-derived raw (docling 2.126.0 / docling-mlx 0.1.1, grade excellent). The parse check flagged table collapse on Table 4's task-outcome column ("30 / 0 / 0; 28 / 2 / 0"). These are completed-correct / completed-wrong / not-completed triples by the caption's definition, not merged cells, and Table 4 matches pdftotext -layout row for row. Table 5 (model label in the first row only in the PDF, repeated per row in the parse) and Tables 1, 6 and 7 were also reconciled against pdftotext and are intact.

Connections#

• Failures That Look Like Success — the inverse case, and its safety edge. That page's class is a run that reads as success and is not; silent override is a recovery that reads as no verification. In a domain where the tool is authoritative, the same override becomes the agent quietly replacing a record, which is that page's class again
• Automated Failure Attribution — trace labels again, on the failure side. That benchmark audits its injected-error labels with three annotators and κ = 0.73; this audit uses one reviewer and detectors revised against that reviewer, so its 100% rows are weaker evidence than they look
• Grader Class Collapse in Jailbreak Evaluation — the same instrument-validity move on another kind of grader: hold the behavior fixed and ask whether the instrument can express it. A grader that cannot use the comply class and a verification rule that has no class for "recovered without re-calling" both produce clean, reproducible, wrong labels
• Every-of-k Reliability — censoring is another non-agent per-attempt failure source: a run cut off by an execution limit and scored as wrong is a harness error counted against the agent, and under pass^k it compounds
• Harness Swap vs. Rerun Noise (What a Harness Buys) — the budget ablation is a harness effect that clears this paper's own rerun gap (20 vs 21/30 on a repeated cell), and like OpenCode's output-cap loss there, it works through runs a limit cuts off
• Measuring Beyond Accuracy Saturation — GSM8K re-instrumented at the trace level: at the same 28/30, Opus 4.8 overrides the tool five times and gpt-oss-120b never does
• Claude Sonnet 5 — the only 30/30 row, with 4 silent overrides and no verification disagreements in review
• Claude Opus 4.8 — the Figure 1b override trace; override on 5 of 15 fault events
• Independent Assurance Plane — a release-gate verdict that was a measurement artifact. A harness that called parametrized tests without arguments reported 282 unexecutable tests as failures, inflating false-oracle rates from 0.7–2.3% to 7–11%, and was exposed only when the builder disputed the counterexamples

Open Questions#

• Does D-verify's 67/81 agreement hold out of sample? The rules were revised against the same 90 traces and one reviewer they are scored on. Settler: a second, blind annotator labelling the 90 unreviewed baseline traces against the frozen final detectors, reported with a chance-corrected κ.
• Do the four verification-disagreement patterns, and the override-vs-verification split, carry over to tools whose outputs the model cannot recompute (a lookup, a structured record, free text)? Settler: the same three-detector audit on a structured or free-form tool task, which the authors name as their next step.

Sources#

• What the Trace Shows and What the Label Says: Auditing Behavioral Detectors in Agent Evaluation — Idunnuoluwa A. Adeniji, Shilpika & Michael E. Papka (University of Illinois Chicago; Argonne Leadership Computing Facility), What the Trace Shows and What the Label Says: Auditing Behavioral Detectors in Agent Evaluation, arXiv 2610.30491, 2026-10-21, 13pp, empirical. Cited for §3.1–3.5 and Tables 1–3 (setup, detectors, measurement states, F1–F7), Table 4 (cross-model baseline), §4.1 and Figure 1b (silent override; viewed), §4.2 and Table 5 (budget ablation), §4.3–4.4 and Tables 6–7 (human agreement and the four disagreement patterns), §5–5.1 (high-stakes reading, limitations, one reviewer, revised detectors), §2 (Mittal 2026, Kaynar et al. 2026 as cited). Parse warning: PDF-derived (docling 2.126.0 / docling-mlx 0.1.1); the table-collapse flag on Table 4 is a false positive (outcome triples by design), and Tables 4, 5 and 7 were reconciled against pdftotext -layout