Minimal Harness for Autonomous ML Engineering (Malena)

Sources#

• Do Theorem Provers Need Theorem-Proving Agents? mini-Lean-agent: A Coding-Agent Baseline for Lean
• How Much of a Harness Does a Strong Agent Need for Autonomous ML Engineering?
• TurnCost: Turns, Not Observations: Cache-Aware Loop Engineering for Coding Agents

Summary#

Kirill Brilliantov, Alejandro Hernández-Cano and Emmanuel Abbé, How Much of a Harness Does a Strong Agent Need for Autonomous ML Engineering? (arXiv 2609.40303, EPFL and Apple, 2026-09-30, 45pp, empirical). The question is which parts of an ML-engineering (MLE) harness still help once the model works inside a coding agent. MLE harnesses have grown to include search trees, program populations, memory hierarchies and teams of specialized agents. The paper's answer: almost none of those parts help.

Backbone, hardware (1× A100 80GB, 12 CPU cores, 144GB RAM) and wall-clock budget are held fixed throughout. The paper climbs a ladder of harness interventions, all re-implemented in one OpenCode v1.15.6 codebase. The top of the ladder is Malena (machine learning engineer agent): one coding-agent session that runs for the whole budget. It has read/write/edit/bash plus three small tools, and the harness only re-prompts it with the time remaining when a turn ends. Three results:

1. Giving the model a shell and a filesystem is by far the largest effect measured. Beyond that, no intervention significantly improves performance.
2. Against four open-source MLE harnesses run as released, Malena is never significantly outperformed at a frontier backbone. The four are MLEvolve, AiScientist, Arbor and ScienceFlow; the comparison covers 17 harness×backbone pairs on MLE-bench and 6 on NatureBench. With GLM 5.2, Malena medals on 62.5% of competitions against 47.1% for the best external harness.
3. Trace analysis shows the agent running the search the harnesses hard-code. It balances rare techniques against late-run performance refinement and reuses its own checkpoints and utility code.

The authors' reading is the bitter lesson applied to test-time inference: "the leverage is in the model and in the runtime it is given, not in the scaffolding built around them." It is the MLE-domain counterpart of Harness Tax: Coding-Agent Cost Multiplies Across Harnesses While Success Barely Moves's coding-benchmark result and of Agentic Loops Overtake Bespoke Systems's formal-math one. Two qualifications travel with it, and both run against the simple reading. Weak backbones still benefit from hand-designed search. And the minimal harness is not the cheap one: Malena's never-reset session is about 6× more expensive in modeled tokens than the harness that resets into fresh threads.

The ablation ladder#

Unless noted, figures are on GLM 5.2, a 29-task MLE-bench split ("default29") and a 24h budget, with 95% bootstrap CIs and paired-by-task significance tests (Appendix A.5).

Rung 1: chat vs coding agent (§4.1, Figure 2). In Chat, one completion writes the whole solution script and the harness re-prompts it with stack traces. In Oneshot, one coding-agent session explores the data, writes, runs and debugs, then documents its solution in a design.md. Read off Figure 2, GLM 5.2's single-iteration medal rate roughly goes from ~21% to ~37%, and Kimi K3's from ~32% to ~43%. Gemma 4 31B gains too, which the authors read as "tool-use fluency matters more than raw model size." Two weak models, GPT OSS 120B and Agents A1, gain nothing. The final DeepSeek V4 releases open a clear chat-vs-agent gap that their Preview checkpoints, built on comparable base models, did not show. That observation underpins the paper's "post-training ate the harness" hypothesis (below).

Rung 2: search over coding-agent iterations (§4.2). Oneshot iterations were chained under four tree policies at a matched 24h budget: Chain (always extend the latest node), Greedy (extend the best validation score), UCB1 (a rank-normalized upper-confidence rule) and Best-of-N (independent restarts). No pair differs significantly on percentile or medal rate. The largest gap is UCB1 over Best-of-N, at +2.71 percentile points with CI (−1.44, +6.60). The authors' explanation: each node already reads its parent's design.md and workspace, so "the agent can already weigh different approaches and implicitly adapt its search."

Rung 3: autonomy (§4.3). Malena drops the orchestration altogether. It is one session for the full budget, free to produce any number of pipelines and submissions. It reaches the highest medal rate, and the paired medal-rate advantage over every manual search policy is significant: Malena vs Best-of-N is +5.09 percentile and +11.9 medal points, and vs Chain +4.58 and +7.7 (Table 5, reconciled). Against UCB1 and Greedy the percentile difference is not significant. It also has the smallest selection gap of any arm, 1.876 points between the self-selected and the oracle-best submission. Best-of-N's gap is 6.832 (see Selection below).

Rung 4: multi-agent orchestration (§4.4, Table 1, 14 harder tasks). Three coordination interventions on top of Malena:

Table 1 reconciled against pdftotext -layout; docling split the oracle medal-rate values from their CIs across two columns, but no value is altered.

Delegation and parallelism are statistically indistinguishable from base. Parallelism raises the oracle percentile ceiling (71.79 vs 69.29), but its selection gap grows enough that the submission actually picked is no better. Broadcast between peers is significantly worse: base vs +B+P3 is +7.59 percentile and +22.4 medal points, both significant (Table 10). This is the measured MLE instance of the coordination-failure literature the paper cites (Cemri et al.; Orogat et al.). The authors scope it themselves: they test simple coordination primitives, not "sophisticated inter-agent coordination protocols."

What survives inside the minimal harness#

Malena is not zero harness. Its default configuration adds three tools to OpenCode: a submission registry (no test scores are ever returned), a system tool (hardware, utilization, time remaining) and background jobs. The tool ablations (Table 9/10, 24h) sort harness parts by kind:

• Removing system + jobs costs 8.9 medal points (significant). The authors' reading is that the agent "struggles to track time correctly without it." Removing jobs alone is marginally better.
• Time nudges at 25/50/75/90% hurt significantly: base vs +N is +5.46 percentile and +10.6 medal. A resource nudge sent when utilization falls below 50% hurts further, by +7.76 percentile against +N alone. The suspected mechanism is distraction by unexpected messages.
• Eight ML-engineering skills, a HuggingFace model-search MCP, strict validation guidelines and delegation produce no significant gain in any combination tested at 12h on the 14-task split.
• The same additions do help the constrained primitive. On a single Oneshot iteration, the full stack (+S+J+N+H+D+V) beats base by 2.80 percentile and 5.0 medal points, both significant (Table 14).

Read together: state the agent cannot observe for itself (clock, hardware, a submission ledger) survives. Advice about how to work (nudges, skills, validation rules, delegation encouragement) helps only the iteration whose autonomy the harness has capped. This is the split Harness Shrinkage as Models Improve already draws between scaffolding that compensates for weakness, which shrinks, and structural machinery the model cannot perform itself, which stays. Here both halves appear in one paper.

Against production harnesses#

Each of the four external harnesses ran as released, at its backbone's maximum reasoning effort, under the same compute, budget and models. The test set was a 30-task "fixed30" split with four broken MLE-bench tasks repaired (see Benchmark hygiene).

Tables 15 and 16 reconciled against pdftotext -layout; every cited row matches.

So the headline "matches or beats every harness on every tested pair" is a no-worse-than claim, as the authors' limitations section says. At DeepSeek V4 Flash it rests on overlapping CIs with a competitor's point estimate ahead, and at Gemma 4 31B it reverses on percentile. The ordering across backbones is the paper's second bitter-lesson argument: "weaker backbones still benefit measurably from hand-designed workflow priors like MLEvolve's tree search, but as backbone agentic capability increases, Malena's plain single-session harness closes that gap."

On NatureBench (40 recent Nature-family tasks, 8h, oracle "surpassed published SOTA" metric), Malena leads at GLM 5.2 (21.7% vs AiScientist 17.5% and MLEvolve 10.8%). At Kimi K3 it ties AiScientist (26.7% vs 27.1%). The metric rewards the number of graded attempts, and MLEvolve calls the evaluation endpoint least (median 8 calls per run vs 16 and 18), which the paper flags as a mechanical disadvantage.

The time profile is shared. All four methods' quality rises fast in the first few hours, plateaus by 5–10h, and then improves only in step jumps when a new best submission lands (Figure 4, left).

The cost inversion: minimal harness, maximal cache reads#

At GLM 5.2, Malena's modeled inference cost is $12.12 per run, about 6.2× AiScientist's $1.95. Its input and output token volumes are comparable to Arbor's and AiScientist's; only cache reads are roughly an order of magnitude higher (Figure 4, right, viewed). The mechanism is structural: one ever-growing session resends its whole prior context every turn, while Arbor and AiScientist reset into fresh, shorter threads per branch or submission. At Kimi K3's much higher output price ($13.99/M vs GLM 5.2's $3.123/M), MLEvolve's uncached, output-heavy tree search becomes the most expensive of the four (Appendix D.4).

Two caveats bound this:

• The cache price is modeled, not measured. None of the compared harnesses reports real cache-hit counts, so cache reads are priced at a uniform 0.1× input. Prompt-Cache Economics measures a real cache at below that ideal, which would move the absolute figures but not the ordering.

What TurnCost adds, and what it can't (2026-10). TurnCost (arXiv 2610.15093, empirical) neither confirms nor corrects the 0.1× price by measurement. Its default cached-read price is the same assumed 0.1× (swept 0.05–0.5), and its 98.6% cached share is simulated from append-only transcript structure under an ideal cache, not read from provider billing. It confirms the structural premise. Across 600 coding-agent trajectories of 60–134 median turns, an append-only loop is 98.1–99.0% cache-eligible, and cache-aware cost grows as about T^1.23 in turn count. So a never-reset session's bill is a cache-read bill by construction, as Figure 4 shows here.

Its sensitivity ladder also says which way a real cache moves the comparison. Cache inefficiency charges the missed share of re-reads at full input price, so the penalty lands hardest on the design whose bill is mostly re-reads. If a real cache serves ~85% of eligible tokens, as Prompt-Cache Economics measures at production prefix sizes, the replay term costs about 0.85 × 0.1 + 0.15 × 1 ≈ 0.24× input rather than 0.1×. That is ~2.4× the modelled replay cost, applied mostly to Malena. This is wiki arithmetic, not a measurement, and it assumes one efficiency for every harness. On that reading the ~6× gap is a floor rather than a ceiling, which agrees with the ordering-preserving caveat above and sharpens it.

• Hardware dominates anyway. 24h of the A100 costs $24–85 depending on provider, against the ~$2–12 in tokens.

This is the opposite sign from the coding-benchmark harness comparisons. There the minimal harness is the cheap one, because cost is driven by tool-schema and system-prompt bloat on short rollouts (Harness Tax: Coding-Agent Cost Multiplies Across Harnesses While Success Barely Moves: Pi ~2.0× cheaper than Claude Code at near-equal success). The two results do not contradict each other; they differ in which term of the bill dominates. On a short task the fixed per-call overhead dominates, and a 4-tool harness wins. On a 24h task the history-replay term dominates, and whichever design resets context most often wins on tokens, whatever its tool count. Orchestration Sets Token Economics's quadratic-replay decomposition predicts exactly this. "Minimal harness" and "minimal tokens" separate once the session is long.

A dollar-capped case in Lean, where the single session is the cheap arm (2026-10). Do Theorem Provers Need Theorem-Proving Agents? mini-Lean-agent: A Coding-Agent Baseline for Lean (arXiv 2610.17331, empirical) runs the same minimal-harness-versus-specialized-harness comparison on Putnam 2025, with one model (Gemini 3.8 Flash) and a $2 per-problem dollar cap. Its loop is mini-SWE-agent with only bash. The loop verifies 7/12 against AxProverBase's 4/12 and costs $2.09 against $5.24 per verified proof. Its usage records show 91.5% of input served from cache against 8.7%, and $0.0134 against $0.0290 per call. The specialized harness's bill goes to reasoning tokens (3.64M against 0.70M) and uncached module prompts. So in a dollar-matched comparison the single session won on both solves and cost. That fits the regime reading above rather than contradicting it: sessions there are about 90 calls, not 24 hours, and the alternative is a module-per-call harness that caches almost nothing. It is a different domain and a perfect verifier, so it does not settle Malena's own dollar-matched question. Detail on Agentic Loops Overtake Bespoke Systems.

The reasoning-effort confound inside one harness#

The tuning appendix (D.3) contains a compute-control datum worth carrying separately. MLEvolve on DeepSeek V4 Flash was compared at max and medium reasoning effort on an identical 10-task subset with the same 24h timeout:

• Median reasoning per call fell from 37,124 to 7,069 characters (~5×).
• Completed search nodes per run rose from 18.5 to 62.8.
• Per-node quality was unchanged (buggy-node rate 61% vs 62%).
• The outcome moved a lot: any-medal rate 0% → 25.0% and mean percentile 42.6 → 53.1.

Under a wall-clock budget, a harness built on many short sequential calls converts per-call reasoning length directly into fewer search steps. The "max effort everywhere" convention is therefore not neutral across harness shapes. The paper adopts medium as MLEvolve's canonical setting for that backbone only. This is Compute-Controlled Benchmarking's point in a new place: the budget unit (wall-clock here, not tokens or dollars) decides which architecture a "matched" comparison favors.

Selection: the gap that eats parallelism#

Each arm is scored two ways. Self-select uses the submission with the best validation score the agent reports. Oracle uses the best hidden-test score, which no agent sees. The gap between them is the cost of choosing among one's own candidates (Table 6, reconciled):

Best-of-N has the same oracle ceiling as Malena and loses five points to selection. The authors offer a mechanism and label it a hypothesis, not a result: the gap tracks comparability of validation scores more than the number of candidates. One session scores every candidate on one validation split. Independent iterations each design their own split, and the harness then ranks numbers that were never on a common scale. The one mitigation that supplies a shared validation workspace (a "baseline iteration" that later iterations start from) has the smallest gap, 2.024, but also the lowest oracle ceiling (Appendix C.3, quoted from prose). The parallel-peers result in Table 1 has the same shape: a higher ceiling, eaten by selection. The general form is on Selection Under a Submission Budget.

Context: one long session, and what compaction costs#

Malena is the opposite of the "clear and restart" discipline: a single session on GLM 5.2's native 1M-token window for 24 hours. Appendix C.7 forces OpenCode's native summarization compaction at 64k tokens on 10 hard tasks × 4 seeds:

• Every run compacts, 9.5 times on average (3 to 63). Each compaction discards about 76% of context (65.4k → 15.6k tokens).
• Runs keep making progress through compaction: the 20 of 40 runs that medal do so after 3.25 compactions on average.
• Performance drops but stays inside overlapping CIs. Self-selected any-medal rate falls from 57.5% to 50.0%, oracle any-medal from 65.0% to 50.0%, and self-selected mean percentile from 77.1 to 73.5.

The comparison closest to a clean clear-vs-continue test is Chain vs Malena. Chain starts each iteration fresh from its parent's workspace and written design.md, which is clear-and-restart with a written record. Malena never resets. Malena wins by +7.7 medal points, which is significant. Chain is not a pure clear arm, though: it also caps each iteration at one 6h production run. This is the strongest measured counterweight in the corpus to the practitioner rule on Context Window Smart Zone.

What the trace analysis shows#

Section 5.2 and Appendix E label each run's code checkpoints with ML techniques. The labeler is a GLM 5.2 judge, which is also one of the backbones under study, and it uses a 13-category taxonomy, embedding-based deduplication and a two-pass canonicalization. Findings:

• Malena builds one general pipeline early, then spends the run on "hyperparameter search, but over code patches." On whale it settles a two-stage embed-then-retrieve architecture within 17 of 80 checkpoints and never changes it. The remaining 63 checkpoints are targeted patches: backbone swaps, resolution bumps, threshold retuning, ensembling.
• Malena and MLEvolve are consistently the top two methods in technique rarity, so the late-run shift toward ensembling, model selection and post-processing does not confine Malena to well-known recipes.
• Independent rediscovery across backbones. GLM 5.2 and Kimi K3 runs on aptos2019 each arrive at soft-label pseudo-label self-training, which no AiScientist, MLEvolve or Oneshot run on that task uses (rarity 0.9).
• The agent adapts to its stated budget. Under identical prompts that differ only in the time limit, performance at the same elapsed time differs by total budget (Appendix C.4), meaning the explore/exploit split responds to the horizon.

Two hypotheses the authors flag as hypotheses#

• "Post-training ate the harness." Coding-agent post-training substitutes for the scaffolding that turns a chat completion into an agent: search over what to try, recovery from a failed step, deciding when to stop. If so, the chat-vs-agent gap should track how much agentic post-training a model got more than its scale. The only evidence offered is the DeepSeek V4 Preview-vs-final contrast. This reframes Agent-Authored Harness Optimization: rather than each model needing a harness fitted to it at inference time, post-training may already do the fitting once, in advance.
• Coding agents as the bitter lesson for test-time inference. Hand-built structure on top of a plain coding-agent session (search trees, planner/executor splits, multi-worker coordination) has a poor and often negative return. Weak backbones gain from it, and the gain closes as backbones strengthen rather than compounding.

Benchmark hygiene#

• Four MLE-bench tasks leak or are unsolvable as shipped, and the paper repairs them with published diffs (Appendix A.2): hubmap copies the unencoded test annotation into the public test folder; smartphone-decimeter-2022 leaves the reference receiver log that ground truth is interpolated from; multi-modal-gesture-recognition ships fully labelled training data as test data; champs omits test geometry. All four are documented in MLE-bench's own Known Issues, and the maintainers deferred the fixes to avoid invalidating existing leaderboard submissions. That is the Benchmark Task Defects (Spec–Test Mismatch) pattern in a different benchmark: the defect is known and the leaderboard is kept anyway.
• Contamination was checked three ways. (i) Dolos containment, which deliberately avoids the default Jaccard similarity that under-reports small-fragment copying: 924,750 Malena-vs-real-solution pairs against a real-vs-real null, zero flagged. (ii) Bag-of-chunk embedding similarity, whose upper tail sits below the null. (iii) A behavioral argument: memorized solutions would not need the environment, yet Chat is far below Oneshot. See Benchmark Contamination and Decontamination.

Evidence and conflict of interest#

empirical, kept at compile. In the paper's favor: controlled matched-resource ablations, paired-by-task CIs, 17 external harness×backbone pairs run by the authors, a second benchmark, and a published contamination check. Limits:

• Malena is the authors' own system. Appendix D.3 documents the tuning applied to the competitors: two backbone-specific tool-calling bug fixes, worker-count and admission-policy ablations on ScienceFlow, timeout and early-stop ablations on Arbor, and the MLEvolve reasoning-effort change. None moved outcomes beyond noise except the last, which was adopted. Asymmetric effort cannot be ruled out, but it is disclosed rather than assumed away.
• Statistical power is thin at 24h: 3–4 seeds per task, versus 6–10 at 6h. The authors call their weaker no-worse-than claims well supported and their effect sizes not.
• The budget is matched on wall-clock and hardware, not tokens or dollars. Under a token budget the ordering could differ, given the 6× cache-read gap.
• Runs with no valid submission are imputed at the worst percentile, which ties two very different failure modes and gives a few large gaps outsized leverage.
• Apple co-affiliation (one author interned there) is not a harness-vendor stake. Every backbone is open-weight and served locally with vLLM, so no model provider's API pricing or harness is being promoted.

Connections#

• Harness Tax: Coding-Agent Cost Multiplies Across Harnesses While Success Barely Moves: the coding-benchmark counterpart. Both find a minimal harness does not lose on success. They disagree on cost, and the disagreement is a regime boundary (short rollouts vs 24h sessions), not a contradiction
• Harness Shrinkage as Models Improve (hub): the cleanest third-party ablation in the corpus of which harness parts shrink. Strategy priors and advice lose value as backbones strengthen; state the agent cannot observe for itself (the clock) survives
• Agentic Loops Overtake Bespoke Systems: also carries mini-Lean-agent, the dollar-capped Lean case where a bash-only single session beats a Lean-tooled harness on solves and cost (91.5% cache share against 8.7%). The same loop-vs-bespoke comparison in a noisy-verifier domain (self-measured validation scores against a hidden test), at matched wall-clock and model, with the weak-backbone reversal that page's "receding frontier" predicts
• Orchestration Sets Token Economics: the opposite slope on harness leverage. Writer's structural harness pays more as models get stronger; these search-and-workflow harnesses pay less. The replay term explains the cost inversion
• Dynamic Workflows: An Algebra for Agents: supplies the both-ways run that page's token question asked for, with a model-driven single session against hand-built harnesses. The model-authored delegation arm bought nothing
• Selection Under a Submission Budget: the selection gap measured on agentic MLE runs, with a score-comparability mechanism in place of sampler diversity
• Context Window Smart Zone: a 1M-token never-reset session beats fresh-thread designs, and forced 64k compaction costs a few points inside overlapping CIs. A measured counterweight to clear-over-compact
• The Bitter Lesson: invoked by name for test-time inference; weak backbones still gain from hand-built structure, and the gain closes as backbones strengthen
• Compute-Controlled Benchmarking: the MLEvolve reasoning-effort result. Under a wall-clock budget, "max effort" cut one harness's medal rate from 25% to 0%
• Agent-Authored Harness Optimization: "post-training ate the harness" is the authors' alternative to fitting a harness per model at inference time
• Parallel Agent Orchestration: three peers sharing one machine raise the ceiling and lose it to selection, and adding a broadcast channel drops the medal rate from 55.7% to 33.3%
• Prompt-Cache Economics: the cost inversion is a cache-read bill, priced here at an assumed 0.1× input because no harness reports real cache hits
• Cache-Aware Agent-Loop Cost (TurnCost): the trajectory-level account of why a never-reset session is a cache-read bill. It uses the same assumed 0.1× cache price, so it does not test this page's figure, but its efficiency ladder implies an imperfect cache widens the ~6× gap
• Benchmark Task Defects (Spec–Test Mismatch): four MLE-bench preparation leaks, documented upstream and left unfixed to protect the leaderboard
• Agent Loop Pattern: Malena is a loop harness whose only control signal is "time remaining" on each turn end
• GLM (Z.AI) / Kimi (Moonshot AI) / Gemma 4: the frontier and weak backbones whose ordering carries the bitter-lesson argument
• Single General Agent vs. Multi-Agent Coding Architecture: a matched single-vs-multi-agent measurement in the MLE domain, which that derived page predates

Open Questions#

• Does the chat-vs-coding-agent gap track the amount of agentic post-training more than model scale, as "post-training ate the harness" predicts? The paper's only evidence is the DeepSeek V4 Preview-vs-final contrast. Falsifiable with any model family that publishes checkpoints before and after agentic RL on a comparable base.
• Does Malena's advantage survive a token- or dollar-matched budget rather than a wall-clock one, priced from measured cache hits rather than an assumed 0.1× discount? Its modeled spend is ~6× AiScientist's, so a cost-matched arm could reverse the GLM 5.2 ordering. Sharpened, not answered (2026-10-15): TurnCost (empirical) uses the same assumed 0.1× cache price and a simulated cache share, so it measures nothing here. Its efficiency ladder implies the dollar gap widens rather than narrows under a real, imperfect cache (see the cost-inversion section), which makes a dollar-matched arm more likely to hurt Malena, not less. Partially answered by analogy (2026-10-15): Do Theorem Provers Need Theorem-Proving Agents? mini-Lean-agent: A Coding-Agent Baseline for Lean (empirical) supplies a dollar-matched minimal-vs-specialized comparison with provider-recorded cache usage, in Lean rather than MLE. There the single session wins on solves (7/12 against 4/12) and costs less per proof, because it is short (~90 calls) and the specialized arm caches almost nothing (8.7% against 91.5%). That shows a dollar cap does not by itself reverse a minimal-harness win. It does not test Malena's 24h regime, where replay dominates.
• Does the selection gap grow with the number of distinct validation methodologies in the candidate pool rather than with the number of submissions? The authors state this prediction and do not test it. A pool of N submissions under one shared split, against N under N splits, would separate the two.

Sources#

• How Much of a Harness Does a Strong Agent Need for Autonomous ML Engineering?: Kirill Brilliantov, Alejandro Hernández-Cano and Emmanuel Abbé (EPFL; Apple, where Hernández-Cano interned), arXiv 2609.40303, 2026-09-30 (dated 2026-10-01 on the PDF), 45pp, empirical. Cited: §3 the fixed resources and self-select/oracle protocol; §4.1–4.4 and Figure 2 (viewed; chat-vs-agent magnitudes read off the chart and marked approximate); Table 1; §5.1 with Tables 2, 15 and 16 and Figure 4 (viewed); §5.2 and Appendix E; Appendix A.2 (the four MLE-bench fixes); A.4 (the 0.1× cache-price assumption); C.3 (quoted from prose); C.4; C.6–C.7 with Tables 9, 10 and 12; C.8 with Table 14; D.3 with Table 17; D.6 with Table 21; F; G. Parse warning: docling split CI brackets from their values across columns in Tables 1, 6, 7 and 9, and welded neighboring cells in Table 7. The task-list Tables 3 and 4 are cell-collapsed. Tables 1, 5, 6, 9, 10, 12, 15 and 16 were reconciled against pdftotext -layout and every cited value matches. Table 7 is cited only through the C.3 prose, and Tables 3 and 4 are not cited. The title and author line spacing is garbled in the parse, which is cosmetic.
• TurnCost: Turns, Not Observations: Cache-Aware Loop Engineering for Coding Agents: Pramod Kumar & Raj Kumar Pandey (Penn State), arXiv 2610.15093, 2026-10-10, POVC '26 at ASE 2026, empirical. Cited for §4 (default 0.1× cached price, same as this paper's assumption), §5.3 (98.1–99.0% simulated cache share), §5.6 (cost ∝ T^1.23) and §5.7 (efficiency ladder). Full treatment on Cache-Aware Agent-Loop Cost (TurnCost)
• Do Theorem Provers Need Theorem-Proving Agents? mini-Lean-agent: A Coding-Agent Baseline for Lean — Ortega et al. (Princeton / Rutgers), arXiv 2610.17331, 2026-10-12, empirical. Cited for the dollar-capped Lean comparison and its Table 4 cache and per-call figures (12 problems, one run each). Full treatment on Agentic Loops Overtake Bespoke Systems