Dynamic Concurrency in Coding Agents

Sources#

• When Sub-Agents Work in Parallel: The Promises and Pitfalls of Dynamic Concurrency in Long-Horizon Coding Tasks

Summary#

Dynamic concurrency is the mode in which a coding agent's main agent decides during execution whether, when and how to spawn concurrent sub-agents, instead of following a fixed multi-agent workflow written in advance (ChatDev, MetaGPT, CodeR). The three shipped forms in this study are Codex Multi-agent V2, Claude Code's Dynamic Workflow, and Kimi Code AgentSwarm.

Li, Li, Jiang & Lou (When Sub-Agents Work in Parallel: The Promises and Pitfalls of Dynamic Concurrency in Long-Horizon Coding Tasks, Nanjing / CUFE / UIUC, arXiv 2610.10263, 2026-10-07, empirical) run each agent on the same 354 tasks with concurrency on and off. That gives 2,124 executions, 11.2B recorded tokens and $20,780 at list price. It is the first controlled single-agent-versus-fan-out comparison in this vault that uses real agents on matched tasks. OrchBench simulated its workers, and the Opus 5 card used a hand-built harness and published no per-task pairing.

The headline is that fan-out costs a lot more and buys little on average. Tokens rise 1.41–3.31×, and runtime rises in 14 of 15 agent-benchmark cells. Pooled resolution changes by −11.9 pp for Claude Code (p = 5.1×10⁻⁷, Holm-corrected McNemar), −5.4 pp for Kimi Code (p = 0.013) and +2.3 pp for Codex (not significant). The average hides a size gradient, though. Averaged over agents, the test-pass delta moves from −8.29 pp on RepoZero to −0.04 on NL2Repo, +1.24 on ProgramBench and +4.31 on LoopsBench, in order of mean implementation size. Bounded SWE-bench Verified tasks see no gain or a large loss.

The experiment#

• Agents and models (July 2026 pairings, native scaffolds, highest reasoning effort): Codex + GPT-5.4, Claude Code + Claude Opus 5, Kimi Code + Kimi K3.
• Switches. Codex toggles multi_agent / multi_agent_v2. Claude Code turns Dynamic Workflow on with ultracode, and its sequential arm disables the Workflow, Task and Agent tools. Kimi Code toggles swarm_mode. Every sequential arm guarantees zero sub-agents, so the comparison is "fan-out as shipped" against "no delegation at all", not parallel against serial delegation.
• Tasks. A 25% stratified sample of four long-horizon benchmarks: RepoZero (Py2JS 100 and C2Rust 50), ProgramBench 50, NL2Repo 26 and LoopsBench 28. Gold change sizes run from 0.8K to 54.4K lines. A 20% sample of SWE-bench Verified (100) is the bounded control. Model, prompt, environment and time budget are held fixed across modes.
• One run per cell. The authors cite LoopsBench's own repeat-stability result and the closeness of Codex's sequential LoopsBench score to the published figure as evidence that noise does not drive the trends.

What fan-out changed#

(Computed from Table 2's per-benchmark rows weighted by task count; every derived figure matches the paper's prose: 1.41–3.31×, −11.9 to +2.3 pp, and 24.12M vs 7.69M tokens per solved task for Codex.)

• Cost per solved task rises for all three agents. Even Codex, the one agent with a net gain, spends 24.12M tokens per solved task under concurrency against 7.69M sequentially, about 3.1×. That is the Cost-per-Task Over Cost-per-Token arithmetic with the parallel arm losing.
• Parallelism does not buy latency on average. Claude Code takes 27.8 min per SWE-bench task with concurrency against 17.5 without, and resolves fewer. Of the 366 tasks both modes solved, concurrency was faster on 102 (27.9%) and at least 20% faster on 46 (12.6%). The speedup exists but is the minority outcome.
• Shell calls multiply most for Claude Code. On RepoZero the count goes from 23.0 to 232.1, and on ProgramBench from 68.3 to 608.2, about 10×. Its token multiplier is still the smallest of the three.
• Concurrency produces unique wins even where it loses on average. On SWE-bench Claude Code solves five tasks only under concurrency despite a 24-point aggregate drop. Codex ties on aggregate but uniquely solves nine tasks in each mode. On LoopsBench Claude Code solves four tasks only with concurrency and none only without. The two modes have different capabilities, not one better one, so a router that picked per task could beat both. The paper does not build one.

Three delegation styles#

The three agents fan out in measurably different shapes, and each shape traces to the spawn interface the authors read in source:

• Codex delegates early and narrow. The first sub-agent starts at a median 4.0% of the trace, and mean peaks stay at 1.21–3.00. The spawn call launches one child and returns control at once, so the main agent keeps working and does part of the implementation itself. Results arrive one at a time and the main agent resumes on each message, so the last sub-agent ends at a median 83.3% of the trace, leaving a final stretch for integration and checking. 24.3% of its executions create sub-agents that never overlap with each other.
• Claude Code delegates later and sustains it. It first spawns at 22.1% because the workflow script has to declare stages and dependencies before launch. It then keeps multiple sub-agents overlapping for 52.7% of the trace at the median, the highest of the three, and adds review-stage agents up to the end (median last endpoint 100%). On hard LoopsBench tasks it creates 133.69 sub-agents per task with a mean peak of 25.46.
• Kimi Code launches one large batch and waits. It first spawns at 26.0%, has the lowest median overlap fraction (15.6%), and reaches the highest peak (37.46 on hard LoopsBench from 67.00 created), because AgentSwarm blocks the main agent until the whole batch returns.

Delegation grows with difficulty in the pooled sample: 6.69 / 7.53 / 15.84 sub-agents created on easy / medium / hard tasks, with mean peaks 3.66 / 4.61 / 6.70. Only 2 of 30 per-agent-per-benchmark Spearman tests survive Holm correction, however. The authors' explanation is that a task can get harder because one core component is harder, which adds work without adding independent subtasks.

The failure taxonomy#

Every one of the 1,062 concurrent trajectories was annotated against its sequential twin by two of three annotators (Cohen's κ = 0.77), with a senior author adjudicating disputes. 650 trajectories carry at least one failure, for 804 pattern instances, organized as 4 categories, 13 subcategories and 28 patterns. The table sums check exactly at every level.

The two largest single patterns are Fan-out Budget Exhaustion (19.2%), where investigation and repeated delegation consume the time needed to finish the implementation, and Early Sub-Agent Termination (15.9%), where the deadline kills sub-agents before the join. Both are budget failures. Fan-out is not wrong in these cases; it does not fit the wall clock it was given. The worked example is Kimi Code on ProgramBench's seqtk. The main agent hands 22 subcommands to 20 sub-agents at minute 17.5 of a 40-minute run. It assumes a two-hour sub-agent timeout and makes no further tool calls while waiting for the whole batch. It never reaches integration, and the result passes 92 of 440 tests, partly because the sub-agent that owned seq never wrote it.

The dominant failure differs by agent. Concurrent Writes are 62.9% of Codex's instances: its main agent keeps writing files while sub-agents write the same tree. Load Imbalance is 39.8% of Claude Code's: its workflow keeps launching investigators and critics until the deadline. Each agent's failure matches its delegation style above. Codex shares a workspace with its children, and Claude Code delegates in waves.

Two smaller patterns map onto failures already documented elsewhere in the vault:

• Overloaded handoff. A Claude Code workflow packs up to 60 findings per probe into an 80K-character brief, and the receiving sub-agent re-does the investigation instead of implementing. That is the real-agent counterpart of RCWT's coordination-displacement cliff, observed once (n = 3 for Bulk Handoff Overload).
• Unverified Global Completion and No Failure Takeover. The main agent declares success without checking stderr, or never recovers a sub-agent killed by a rate-limit spike. These are the orchestrator-side forms of relaying unverified subagent claims.

Where fan-out does pay#

The authors examine 131 pairs in which concurrency won: a higher test pass rate, a task solved that the sequential run missed, or a run at least 60 s and 20% faster at no quality cost. Three mechanisms recur:

1. Complementary work. Clearly separated parts run in parallel while the main agent integrates them. On justext, eight sub-agents handle delivery files and checks while the main agent writes the core algorithm, and wall time falls from 734 s to 468 s at equal tests.
2. Alternative attempts and selection. Parallel candidate solutions let the main agent pick or combine. On Halite, Codex merges a sub-agent's map logic with its own protocol handling and passes 292/391, where the sequential run fails to build (0/391). This is the SwarmResearch case, in which isolation is the point.
3. Independent validation. A sub-agent finds a fact that corrects the main agent in time to matter. On a LoopsBench security lab, a crypto sub-agent surfaces the P-256 curve-order requirement for ECDSA nonce reuse. This is a designed maker/checker split, the version Optimizer–Evaluator Decoupling endorses, not the self-verification Unproductive Self-Verification warns against.

All three require the main agent to use the parallel work, by integrating it, selecting from it or acting on it. The failure categories are the same requirement unmet.

Against the rest of the corpus#

• The Opus 5 system card's multi-agent result does not carry over to Claude Code's shipped mode. The card reports every multi-agent variant matching or beating single-agent on BrowseComp, and a 5-agent team reaching the same ProgramBench score 2.2× faster (Parallel Agent Orchestration). Here, Claude Code with Opus 5 on ProgramBench takes 38.9 min concurrent against 38.7 sequential, with test pass 0.5% against 1.8%. Overall, Claude Code is the one agent whose concurrency loss is large and highly significant. These are different harnesses, not a contradiction of the card. The card's N-agent team gives each agent its own repo checkout and shares code through Git. Dynamic Workflow writes to one shared workspace under a 40-minute budget. The weight goes to this paper for what a user gets from ultracode, and the card's figure stands as a vendor measurement of a configuration users do not get by default.
• "Few cases of agents overwriting each other's work" (Anthropic's prompting guide, vendor-claim) is consistent with Claude Code's failure mix here, where Load Imbalance dominates. It does not generalize to fan-out as a class: shared-state and merge failures are the largest category overall and 62.9% of Codex's. Overwrite risk follows the workspace design (whether sub-agents share a tree with a main agent that is still writing), not concurrency as such.
• The shipped caps were not binding, or not counting these agents. Claude Code's mean peak of 25.46 concurrent sub-agents on hard LoopsBench sits above the 20-concurrent default of v2.1.217, and 133.69 created per task sits far above the "fewer than 15 agents" workflow guideline. The paper runs "default settings" but names no Claude Code version. Either the version predates the caps, or workflow agents are not counted against the Task-subagent concurrency limit. That is the open mechanism question carried on Parallel Agent Orchestration and Dynamic Workflows: An Algebra for Agents.
• Akshay Nathan's task-shape rule now has measurements behind it. He said multi-agent suits "incredibly complicated… open explorations, or very paralyzable" work and that most tasks are neither (practitioner-opinion, on Claude Code Best Practices). The SWE-bench result is the "most tasks" half, measured, and the three benefit mechanisms are his two buckets plus independent validation.
• OrchBench's crossover agrees in shape. It found the multi-agent advantage disappearing once the working state fits one window. Here the advantage appears only as implementation size grows. Both say fan-out relieves a capacity limit and costs net when there is none to relieve. Neither paper gives a single crossover number that transfers.

How much to believe#

• Moderate conflict of interest on the strongest result. First author Han Li is also first author of LoopsBench, the benchmark that shows the clearest concurrency gains. The paper discloses the citation but not the overlap. The gain also appears on NL2Repo for two agents, and the size gradient is monotone over four benchmarks from other groups, so the direction does not rest on LoopsBench alone.
• The Claude Code arm is not a clean concurrency toggle. ultracode switches on Dynamic Workflow, and the sequential arm removes Task and Agent as well. Anything else that mode changes, such as the extra critic and review stages the paper observes, is bundled into "concurrency". Claude Code's bounded-task loss may come partly from workflow overhead on tasks too small to need it, which is what the authors' own Load Imbalance coding suggests.
• Claude Code's ProgramBench test pass (0.5–1.8%) against Codex's (53–58%) is a gap the paper reports without explanation. Kimi is also near zero. It reads as a harness or format interaction with the ProgramBench evaluator rather than a capability gap. No vault source settles it, and no cross-agent ProgramBench comparison should be drawn from this table.
• One run per cell, with 26–28 tasks on NL2Repo and LoopsBench. One LoopsBench task is 3.6 pp, so a 14.3-pp swing is four tasks.
• Strengths. Matched pairs, native scaffolds, McNemar with Holm correction, dual annotation with κ = 0.77, and a full artifact release: trajectories, codebook, review registers and scripts that regenerate every number; the text also cites archived Claude Code workflow scripts.

Connections#

• Parallel Agent Orchestration — the umbrella page. This page is its first controlled real-agent concurrency on/off measurement, and the counterweight to the Opus 5 card's multi-agent numbers
• Dynamic Workflows: An Algebra for Agents — Claude Code's half of this study. The paper cites archived workflow scripts from these runs, possibly the first public corpus of model-authored workflow programs. The wave-style delegation (133.69 created against a 25.46 peak) is that primitive's measured shape
• Orchestration-Plan Simulation — the simulated-worker predecessor. Its "missing transfer" is this taxonomy's Missing Interface Contract and Incomplete Sub-Agent Brief, and both papers find that agent count alone does not predict quality
• Orchestration Sets Token Economics — the same model with a different orchestration mode moves tokens 1.41–3.31× on long-horizon coding, here in the costly direction
• Cost-per-Task Over Cost-per-Token — 24.12M against 7.69M tokens per solved task is the cost-per-task argument applied to fan-out
• Automated Failure Attribution — this taxonomy was hand-labelled with κ = 0.77 over paired traces. WHO&WHEN PRO shows LLM judges label coordination failures as reasoning errors, which is why a human-coded concurrency taxonomy still has no automated substitute
• Context Lifecycle Management — the Overloaded Handoff pattern is RCWT's coordination-displacement cliff seen once in a real agent
• Open-Ended Discovery Harnesses — the Alternative Attempts mechanism, which works when parallel outputs are alternatives in isolated branches rather than parts written into one tree
• Optimizer–Evaluator Decoupling — the Independent Validation mechanism as a maker/checker split
• Claude Code Best Practices — answers the matched-task half of its subagent-overhead question
• Codex, Claude Code, Kimi (Moonshot AI) — the three agents studied

Open Questions#

• How much of Claude Code's bounded-task loss (SWE-bench 83 → 59) comes from parallel sub-agents, and how much from what ultracode adds (review and critic stages)? Settler: a three-arm run on the same 100 tasks with no sub-agents, the Task tool only, and full Dynamic Workflow.
• Does giving each sub-agent its own worktree, with an explicit merge step, remove the shared-state category (33.2% of instances) without losing the complementary-work and alternative-attempt gains? Settler: rerun the paper's long-horizon sample with per-sub-agent isolation, reporting D-category counts and pass rates side by side.

Sources#

• When Sub-Agents Work in Parallel: The Promises and Pitfalls of Dynamic Concurrency in Long-Horizon Coding Tasks — Han Li, HanHaoNing Li, Ziqian Jiang & Yiling Lou (Nanjing University / Central University of Finance and Economics / UIUC), When Sub-Agents Work in Parallel: The Promises and Pitfalls of Dynamic Concurrency in Long-Horizon Coding Tasks, arXiv 2610.10263, 2026-10-07, 22 pp, empirical. Cited for §3.2–3.4 (agents, switches, sampling), Table 1, Table 2 and §4 (effectiveness, efficiency, overlap), §5 (delegation scale and timing), Table 3 and §6 (taxonomy), §7 (benefit mechanisms) and §8.2 (validity). Table 2 was reconciled against the prose before citing: the pooled token ratios (1.41× / 2.36× / 3.31×), pooled resolution deltas (−11.9 / −5.4 / +2.3 pp), Codex tokens per solved task (24.12M / 7.69M) and per-benchmark test-pass averages (−8.29 / +4.31 pp) all recompute exactly from the parsed cells, so the parse is clean. Table 3's counts sum exactly at subcategory, category and total (804) level. Figures 4–6 (overlap Venn diagrams, delegation counts, activity profiles) were not opened; every figure-derived number quoted here comes from the prose