Cost-per-Task Over Cost-per-Token

Sources#

• Agent swarms and the new model economics
• Claude models explained: choosing the best model for your use case
• Codex from 0 to 10M Users: Building ChatGPT Work - Akshay Nathan, OpenAI
• Fable's judgement
• Gemini 3.5 Flash-Lite Model Card
• How is the Bun Rewrite in Rust Going?
• Recursive Self Improvement for Coding Agents
• The Harness Effect: How Orchestration Design Sets the Token Economics of Enterprise Agentic AI
• The price is wrong: AI cost calculation has to consider task completion rates, not just token costs

Summary#

Anthropic's published answer to "which model should I use for this workload?" (vendor-claim, July 2026). The default recommendation inverts the intuitive one: start with the most intelligent generally-available model and use the effort level to dial performance and cost down — not start cheap and escalate. The load-bearing claim is that the two cost metrics diverge: price-per-token is higher for stronger models, but cost-per-task is often lower, "because more capable models often take fewer turns and less thinking time to get most tasks right."

A second, non-economic argument does independent work: starting with a smaller model makes it harder to distinguish model failures from setup failures. A weak model failing your harness tells you nothing about whether the harness is wrong. Start at a capability level where failure is informative, then descend.

Anthropic documents both directions — start-smart-and-descend, and start-cheap-and-ascend-until-quality-clears — but leads with the former.

Why the two cost metrics diverge#

A task's cost is (tokens per turn) × (turns) × (price per token). Model class moves all three terms, and not in the same direction:

• Turns. A stronger model gets more tasks right on the first attempt; retries, re-planning, and repair loops are where a cheap model's savings go.
• Thinking tokens. At a fixed effort setting a stronger model reaches the answer with less deliberation.
• Price. Higher for stronger models — the only term that favors the cheap choice.

The claim is that the first two usually dominate. Note the failure mode this argument does not cover: a stronger model that over-deliberates rather than converging inverts the same arithmetic — see Unproductive Self-Verification, where Opus 5 performed worse at higher effort by re-verifying answers it had already verified. The cost-per-task argument holds only where extra capability buys convergence rather than rumination.

Effort as the second axis#

Model class and effort level are separate dials, and they overlap: "higher-class models at lower efforts can sometimes be more efficient than smaller models." So the selection space is a 2-D grid, not a ladder — Opus-at-low-effort and Sonnet-at-high-effort are different points that may cost the same. The two published curves (quality-vs-cost, quality-vs-latency) are labelled "illustrative and not plotted from benchmark data" — the shape of the tradeoff is asserted, not measured, which is unusual candor for a vendor selection guide and also means nothing here is a number you can plan against.

This is the same inference-budget-is-capability thesis as Large-Scale Test-Time Compute, surfaced as a product knob rather than a research finding.

The model classes#

Anthropic's framing: the classes do not specialize by domain ("we don't recommend one model class for finance and another for science"). Every class is trained for coding, agentic tasks, and knowledge work. The only axis is how hard a problem the class can reliably carry, and what that costs.

Class	Position	Notes
Mythos / Fable	Most capable; frontier across domains; coding, long-running agents, previously-unsolved problems	Two packages of the same underlying model: Mythos for Project Glasswing organizations doing dual-use cyber/bio work, Fable with the extra safeguards that make it public-safe. Both require limited data retention
Opus	Reasoning-intensive enterprise tasks	Benchmarks cited: GDPval-AA (knowledge work), Terminal-Bench 2.1 (agentic coding)
Sonnet	Everyday tasks; balance of performance/cost/speed	Called out for high-volume sub-agents in multi-agent orchestration
Haiku	Lowest cost, fastest	High-frequency workloads where latency and cost dominate

Opus vs. Fable: the benchmark-blind difference#

The most interesting claim in the piece. Opus and Fable have "similar benchmark scores," yet: "in real-world situations, larger models such as Fable tend to have more wisdom, creativity, and writing skills." The stated decision rule is therefore not benchmark-driven at all —

If your evals or internal testing show Opus struggling on some tasks, then Fable is the answer. If Opus already clears the quality bar, then its speed and price profile may make it the better choice.

A vendor stating that its own benchmark scores fail to separate two adjacent classes is a first-party construct-validity admission — the same gap Measuring Beyond Accuracy Saturation documents from the research side, where statistically-indistinguishable agents still differ sharply on reliability, efficiency, and cost.

The four selection questions#

1. How hard is this task? Multi-step, long-running, or previously-unsolved → higher class.
2. What are the latency needs? High-frequency customer-facing → Sonnet.
3. What are the access constraints? Mythos is Glasswing-only; not every org exposes every class to every role.
4. What are the unit economics? At high production volume, lower classes may be right if evals show the tasks complete satisfactorily.

Note that question 4 defers to evals, and question 1 defers to judgment. The framework is a scaffold for a measurement, not a substitute for one.

The advisor strategy#

The hybrid that avoids choosing: a faster, cheaper worker model calls a more intelligent advisor model to check its plan and evaluate its work — "the executor model is coached only when needed."

The one quantified result: on SWE-bench Pro, Sonnet 5 with a Fable 5 advisor lands within 10% of Fable 5's own score at 63% of the price of running Fable 5 for the whole task.

Two things make this more than a cost trick:

• It is Optimizer–Evaluator Decoupling shipped as a first-party recommendation — the thing that grades the work is structurally not the thing that produced it. Anthropic arrives at the same invariant from the cost side that the agent-quality literature arrives at from the Goodhart side.
• It is a partial answer to the standing question of where a cheap model crosses over into an expensive one (Claude Sonnet 5): the crossover isn't a point on the effort dial, it's a different topology. Selective coaching beats both "run the cheap model harder" and "run the expensive model throughout."

Caveat: 63% of the price is measured against running Fable 5 for everything, not against Sonnet 5 alone — the advisor is a markup on the cheap path, and the 10% quality gap is a real one.

The distinction, seen from the vendor side#

DeepMind's Gemini 3.5 Flash-Lite card (2026-07-21, vendor-claim) is what this page's argument looks like when a vendor publishes only half of it. Its benchmark table opens with price rows — input and output $/1M for itself, its predecessor, GPT-5.4 mini and Claude Haiku 4.5 — and its own generation-over-generation move is Anthropic's shape exactly: price-per-token up 67% on output, capability up a full tier (SWE-Bench Pro 38.3 → 54.2, OSWorld-Verified 54.3 → 74.0). If the cost-per-task argument holds, the more expensive model is the cheaper one to run, and this is the first cross-vendor instance of the pattern in the corpus.

It is also the cleanest demonstration of why the argument stays unverified. The card reports no tokens-per-task, no turn counts, and no thinking budget for any of the four models — so the one term that would decide it (does the stronger model converge in fewer turns and fewer thinking tokens?) is absent from the only artifact that prints the price. Both vendors publish the price axis and neither publishes the token axis; Anthropic at least labels its curves "illustrative", while DeepMind's numbers look precise and are missing the same multiplier. See Inference Efficiency as Capability for the per-dollar arithmetic and Compute-Controlled Benchmarking for why a price row is not a compute budget.

Four mixes, one quality bar, 8× the cost (Cursor, 2026)#

Every source above prints a price and omits the task. Cursor's swarm post (Wilson Lin, 2026-07-20, case-study) is the first in the corpus to do the reverse: hold the task, the harness and the time budget fixed, vary only which model plays which role, and publish the dollars.

The task is a from-scratch SQLite implementation in Rust graded on a held-out suite (Parallel Agent Orchestration). Four configurations under a four-hour budget: GPT-5.5 as both planner and worker; Grok 4.5 as both; Opus 4.8 planning with Composer 2.5 working; Fable 5 planning with Composer 2.5 working. Only the endpoints of the cost range are given in prose — $1,339 for the Opus 4.8 hybrid to $10,565 for GPT-5.5 alone (the per-run bars live in a chart this ingest did not capture).

Cursor's summary: "every model mix produced similar quality while the costs varied enormously." Quality at the cutoff sat between 73% and 85% across all four, and every configuration eventually passed the whole suite. So this is a rare thing — an 8× cost spread at a common quality bar, on real infrastructure, outside Anthropic.

Three findings underneath it:

• Tokens and dollars split differently, and the split is the whole argument. Workers carry at least 69% of the tokens in every run and over 90% in most. But in the Opus/Composer mix the planner produced a small fraction of the tokens and roughly two-thirds of the cost, with the worker fleet handling the vast majority of tokens for the remaining third. Frontier price is affordable precisely because frontier judgment is needed rarely: "few moments in a large task genuinely require frontier intelligence… once a frontier planner has collapsed the ambiguity into a detailed, explicit instruction, less expensive models simply have to follow it."
• The worker line is where the money is. GPT-5.5 doing both jobs spent $9,373 on workers alone; Opus-planning-with-Composer-working spent $411 on the entire worker fleet — a 23× gap on the same task at the same grade. This is the strongest evidence in the corpus for the guidance's own one-line hedge ("Sonnet for high-volume sub-agents in multi-agent orchestration"), which the guidance never develops.
• Cost-per-task held inside the role and reversed at the system level. The Fable 5 planner billed slightly less than the Opus 4.8 planner despite roughly twice the per-token price, because it emitted far fewer planning tokens — this page's thesis, confirmed, at the planner role. And the Fable run came out substantially more expensive overall, because its workers burned several times as many tokens. A better planner externalized cost downstream. Per-role cost-per-task can be right about every role and wrong about the bill.

That last point is the one to carry. The vendor guidance reasons about "a workload" as if it had one model in it; the moment a pipeline has two roles, a model's cost includes the work it causes other models to do, and no single-role measurement can see that term. It is the cost-side twin of the cross-stage coupling Client-Side Agent Optimization's combo abstraction describes for quality.

Discount appropriately. Vendor-authored, on the vendor's own harness, with the vendor's own model (Composer 2.5) as the worker in both cheap configurations — and the comparison across mixes is confounded with a simultaneous harness rebuild insofar as absolute figures go (the old harness's runs are not costed here). The solo Opus 4.8 and Fable 5 runs that would round out the frontier-cost picture were "graded only informally," so Cursor explicitly draws no quality conclusion from them.

The layer above the model menu (Writer, 2026)#

Every source above varies the model. Writer's harness-swap paper (arXiv 2607.06906, 2026-07-08, empirical) holds the model constant and varies the orchestration layer instead — 22 locked tasks, six models across five vendors, one LLM-judge panel, one pinned price table, and a single variable: a conventional production agent loop (frozen 2026-06-07) versus Writer's Agent Harness. Blended: cost per task −41% ($0.21 → $0.12), tokens −38% (14.2k → 8.8k), wall-clock −44%, quality at parity (0.78 → 0.81, reported as a wash at n = 22).

The result that bears on this page is the comparison of levers: under the baseline, moving from the most expensive model (Palmyra X6, $0.25/task) to the cheapest (Qwen 3.6, $0.16) saves 36%; keeping any model and swapping the harness saves 33–61%, with no exceptions across six models. On this workload the orchestration layer moved the bill more than the entire spread of the model menu did. Writer's framing: "teams comparing $/Mtok across vendors are comparing p; the bill is p × τ, and τ belongs to the harness."

It also supplies, for six models at once, the token axis this page keeps recording as missing — per-task tokens, turns' worth of cost, and latency, all under one price table. And on that axis the start-smart default does not hold here. Dividing the paper's per-model baseline quality means by its per-model baseline cost (arithmetic on two verified tables, not a figure the paper prints):

Model, baseline arm	mean quality	$/task	quality per $
Palmyra X6	.789	$0.25	3.16
Claude Sonnet 4.6	.785	$0.24	3.27
GLM 5.1	.752	$0.21	3.58
Gemini 3.1	.765	$0.19	4.03
Gemini Flash 3.5	.740	$0.18	4.11
Qwen 3.6	.710	$0.16	4.44

The ordering is monotone and points the wrong way for "start with the most capable model": the two strongest models are the two worst per dollar, and the capability spread that buys the premium is eight points of a capability-mean score. Three caveats before this is treated as a refutation — the arms are not matched on quality (this is a ratio, not an iso-quality comparison), the workload is an enterprise-assistant task set rather than the long-horizon agentic work where extra capability is supposed to earn its price by converging in fewer turns, and quality is LLM-judged at n = 22. But it is the first place in the corpus where per-model cost and per-model quality are published together on non-Anthropic infrastructure, and the correlation between price and cost-per-task is positive, not negative.

Discount for a total conflict of interest: all 33 authors are Writer employees, the last author is co-founder and CTO, the harness under test is Writer's, the baseline is Writer's own superseded loop, and Palmyra X6 is Writer's model. The paper self-discloses and its design is auditable (frozen baseline, locked prompts, identical judges and price tables, candidate failures scored not excluded). Full treatment, including the mechanism inventory and the "harness leverage" finding, at Orchestration Sets Token Economics.

The production bench: Databricks on its own codebase (2026)#

Writer's paper is a vendor benchmarking its own product. Its production counterpart landed five days later and, on the model axis, points the other way. Databricks reports, via The Register (Thomas Claburn, 2026-07-13, case-study, secondary reporting of Databricks' benchmark blog post and CTO Matei Zaharia's social posts), an internal coding benchmark built from real engineering tasks its own staff performed against its multi-million-line codebase — built, Zaharia says, because models are tuned to public benchmarks like SWE-Bench, which the article notes OpenAI has called "broken."

Model	$ / task	Task success
Opus 4.8	$1.94	87%
Sonnet 5	$2.09	81%
GLM 5.2 (Z.ai, open weight)	$1.28	"statistically tied with Opus 4.8 on quality"

"Cheaper per-token does not imply cheaper per-task. For example, Sonnet 5 costs less per token than Opus 4.8 but used more tokens, resulting in higher cost and lower quality." — Zaharia

The Anthropic pair is this page's thesis, measured by a third party. Sonnet 5 is "around 1.7x cheaper" per token — consistent with the published list prices ($3/$15 vs $5/$25 per Mtok = 1.67×, arithmetic here, not a figure The Register prints) — and 8% dearer per task, at six points less success. Both non-price terms move against the cheap model at once: more tokens per attempt and more attempts. This is the first instance in the corpus of the exact claim Anthropic makes about its own models being confirmed on someone else's production codebase by someone else's measurement. One ambiguity: the article never says whether $/task is per attempted or per completed task. If attempted, dividing through by the success rates widens the gap to roughly $2.58 vs $2.23 per completion (again arithmetic, not a published figure).

The GLM arm breaks the tidy version of the rule. The open-weight model lands in the top capability tier, statistically tied with Opus 4.8 on quality, at $1.28/task — 34% below Opus. (No per-token price is given for it; the article's framing is that "open weight models like Z.ai's GLM 5.2 are competitive with frontier models," and open-weight serving is the cheap end of the menu.) So the cheapest arm on offer is also the cheapest per task, at parity. "Cheaper per token implies dearer per task" is therefore not a law about price tiers; it is a claim about tokens-to-completion. Sonnet 5 lost because it burned more tokens and finished less often; GLM 5.2 does neither. Read strictly, the start-smart default would have selected the second-cheapest option available on this workload. (This figure exists in the raw only because the ingest pass rebuilt the article body from curl'd HTML — WebFetch silently dropped it.)

Two benches, one contradiction, worth keeping visible. Writer finds quality-per-dollar falling monotonically with model strength on an enterprise-assistant task set; Databricks finds the stronger Anthropic model cheaper per task on a long-horizon coding task set. Neither is Anthropic. The reconciliation the two support jointly — not proven by either — is that the convergence term this page rests on (fewer turns, fewer retries) only dominates where there are many turns to save, which is the coding regime and not the assistant regime. Note the asymmetry in what each can be trusted for: Writer is empirical with a total COI and a full methods section; Databricks is case-study, relayed second-hand, with no task counts, no variance, no n and no confidence intervals behind "statistically tied." The agreement between them is on the negative claim (listed per-token price is a bad predictor of the bill), which is the claim both sets of numbers actually support.

A third-party generalization the article cites, unread here. The Register points at an academic result (arXiv 2603.23971, March 2026): in about a third of the model comparisons the authors ran, the model with the lower listed price ended up costing more — "Gemini 3 Flash's listed price is 80 percent cheaper than GPT-5.4's, yet its actual cost across all tasks is 38 percent higher." That is the broadest statement of this page's thesis anywhere in the corpus and it is currently a pointer, not evidence: the wiki has not read the paper, the figure survives in the raw only because of the same HTML repair, and "about a third" is also the rate at which the inversion does not fire. Worth ingesting.

Interest to declare. Zaharia says these results are why Databricks built Omnigent, a wrapper for combining and swapping coding agents — so the harness half of the write-up (see Orchestration Sets Token Economics) is adjacent to a product, though no Databricks harness or model is in the comparison and Databricks sells neither of the models it prices.

Where benchmarks stop helping#

Anthropic's own guidance says public benchmarks are "helpful directional guides" that break down exactly where the choice gets expensive: at the Opus/Fable tier the models "solve almost all of the questions on the test" (saturation). The recommended replacement is a curated set of problems drawn from production, including tasks where current tooling falls short, with success criteria the team defines — Production-Sourced Evaluation as vendor advice, and the operational form of Evals as Product Spec.

The circularity is worth naming: the selection framework's two hardest questions (is this task hard? do the unit economics work?) both resolve to "build an eval," and the vendor's own benchmarks are declared insufficient for the tier where the decision matters most.

Campaign cost is not cost-to-shipped#

The largest published cost-per-task figure in the corpus is the ~$165,000 of API tokens for the Bun Zig→Rust port, set against a stated counterfactual of three engineers for a year. An independent audit of the same project (Lockwood, 2026-07-27, case-study) shows why that comparison is not apples-to-apples, and the correction generalizes well beyond Bun:

• The token figure is bounded at green, not at shipped. $165k covers the port through the May 14 merge to main. It excludes CI (a continuously-running Buildkite cluster), the employee time spent monitoring and re-prompting ~50 workflows, and the post-merge stabilization tail — which was still visibly running eleven weeks after the last release tag, with the agent PR queue growing from 1,277 to ~2,475 in eighteen days.
• The counterfactual is fully loaded; the measured side is not. "Three engineer-years" carries salary, overhead, review, and CI. "$165k" carries tokens. Comparing them favors the agent path by construction unless the same boundary is drawn on both sides.
• The extrapolation is not the correction. Lockwood's ~$800k comes from assuming the project still burns $10k/day — an assumed rate, not an observation, over a boundary he does not define. His direction holds; the number is speculation and should not be repeated as a measurement.

So the practical rule when reading any cost-per-task claim about an agent campaign: ask which cost line the number is drawn at. Cost-to-first-green is the cheapest honest boundary to report and the one least likely to be the number a buyer cares about. This is the accounting analogue of Verification as the New Bottleneck — the expensive part of the work is downstream of the part that is easy to price.

Tension: the strongest model is not always the best component#

The "start smart" default is stated for a workload, implicitly a single agent. It sits badly against the empirical multi-role result in Client-Side Agent Optimization: on HotpotQA, Opus 4.6 was the worst planner of 81 combinations (it answered from parametric knowledge instead of delegating to the solver's search tools), and a Ministral 3 8B planner paired with an Opus solver scored 74.27% vs. 31.71% for Opus-as-both. AgentOpt also measured 13–32× cost gaps between equally-accurate pipeline combinations.

Weighting by evidence: AgentOpt is empirical and multi-role; Anthropic's guidance is vendor-claim and single-role. They are not strictly contradictory — "start smart" is defensible as a first configuration precisely because it makes failures diagnostic — but "start with the most intelligent model" is not safe advice for per-role assignment inside a pipeline, and nothing in the vendor guidance flags that boundary. Anthropic does gesture at it once, obliquely: Sonnet is recommended for "high-volume sub-agents in multi-agent orchestration."

The other vendor's answer: don't publish a rule, ship a default#

Anthropic's response to "which model and how much effort?" is a published decision procedure for the user (start strong, dial down; four selection questions; build an eval). OpenAI's, as of the ChatGPT Work launch, is the opposite: be opinionated in the product and hide the axes. Akshay Nathan (Codex from 0 to 10M Users: Building ChatGPT Work - Akshay Nathan, OpenAI, practitioner-opinion):

"We want this default to be the best possible. Like, we wanna be opinionated about the default… We have for power users options under the hood. One could argue that there might be too many right now, and we're working on simplifying it… but the default should be good enough."

The mechanics of the collapse are explicit. There are "32 options" across model classes and reasoning levels; the shipped control is a one-dimensional slider — "reduce it to one dimension even though there's multiple dimensions… speed and efficiency on one side, quality and thoroughness on the other." That is this page's two axes (model class × effort) projected onto a single user-facing scalar, with the projection chosen by the vendor.

Three things worth separating:

• The escalation trigger is the same. Nathan's advice for when to leave the default — "if you're not seeing either the efficiency on the cost side or the quality on the intelligence side" — is the same two-sided test Anthropic's framework encodes. Neither vendor claims a rule for which direction to move first; both say measure your own workload.
• The exposure philosophy diverges, and only one side is falsifiable. A published rule can be wrong in public (as Client-Side Agent Optimization shows "start with the strongest model" is, per-role). A tuned default can be wrong silently, and the user has no way to know the projection is costing them. Nathan's own hedge — "there's a preference on, for you as an individual, how do you like to collaborate with the models" — concedes the projection is not user-invariant.
• Neither publishes tokens-per-task. The gap named above for DeepMind and Anthropic holds here too: no turn counts, no thinking budgets, no cost-per-task figures anywhere in the account. The default is asserted to be best "for most use cases" with nothing behind it.

The practitioner counter-current from the same episode: Vibhu reports instructing nearly every long-running task to "use sub-agents where possible," for wall-clock and to "offload to a lot of smaller, cheaper models" — the cost-per-token intuition this page argues against, running in the wild at the sub-agent layer, where Client-Side Agent Optimization suggests it may actually be right.

Who picks the model (Willison, July 2026)#

Every source above answers which model or harness is cheaper. Simon Willison (2026-07-03, practitioner-opinion) changes the question to who decides, and the answer is the model. Relaying a tip from Jesse Vincent, he prompted Claude Code:

"For all coding tasks use your judgement to decide an appropriate lower power model and run that in a subagent"

No routing table, no threshold, no per-task rule — the selection decision itself is delegated. That is a control-plane choice rather than a pricing one, and it is the general form the Claude Code team recommends: see Harness Shrinkage as Models Improve, where replacing a hard rule with "use your judgement" is the same move Anthropic made inside its own system prompt.

The policy the model then wrote into its own memory file is this page's conclusion restated unprompted — cost/efficiency, "implementation work rarely needs the top-tier model; judgment, review, and synthesis stay with the main loop," with Sonnet named for substantive implementation and Haiku for trivial edits. That is Cursor's planner/worker split reached by one developer in one sentence. It is also the arrangement Client-Side Agent Optimization identifies the failure mode for — a strong model that answers instead of delegating — which is precisely the failure this prompt asks the strong model to police in itself.

Nothing here is a measurement: "so far it seems to be working well" and a Fable allowance "shrinking less quickly than before," with no baseline, no task set, no dollars, and no check on whether the self-chosen tier was the right one. Record it as the practice in the wild — the second instance of the counter-current noted just above, differing in the one interesting place. Vibhu names the tier; Willison delegates the naming.

Connections#

• Orchestration Sets Token Economics — the lever one layer up, and the one this page never varies. Holding the model constant and swapping only the orchestration layer cuts cost per task 41% and tokens 38% across six models with no exceptions, which is a larger spread than the model menu itself (36%). It is also the first source to publish per-model cost and per-model quality together off Anthropic infrastructure — and on that data the quality-per-dollar ordering runs opposite to the start-smart default. Discount for a total vendor COI (33 Writer authors benchmarking Writer's harness against Writer's own frozen predecessor). It now also hosts the harness half of the Databricks bench whose model half is above — three shipped third-party harnesses on a real codebase, same success rate at "2x less cost" for the minimal one, and a 3.13× per-task context spread — which is the same lever measured by a party that sells none of the harnesses in it
• Prompt-Cache Economics — the token axis this page keeps calling missing, actually published — by a third party, on a $98.96 end-to-end budget reconciled against Anthropic's invoice to within 1%. It also breaks the page's framing in a useful way: per-token price is not a static constant to multiply by a token count, it is a function of cache state, prefix size, and call count, which is exactly the term every cost-aware routing paper holds fixed. And the cost-per-task thesis runs backwards in at least one measured case — query-aware prompt compression cuts tokens 3× and raises the bill 40.1% over sending nothing compressed, because the cache-write tax on the busted prefix exceeds the read savings
• Tool-Output Pruning — the two axes disagreeing inside a single system, which is the cleanest form of this page's problem. On SWE-Bench Verified, SWE-Pruner Pro posts the largest input-token reduction on one backbone (−13.5%) while running the highest API-call count of any method on both (111.8 vs 94.8; 139.8 vs 131.9) — pruning shortens each call and lengthens the trajectory, and the authors explicitly refuse to collapse the two into one efficiency number because they move in opposite directions across backbones. The reverse case is on the same table: on MiMo-V2-Flash every pruner raised per-trajectory input tokens (+6.6% to +14.9%) and every pruner improved the resolve rate, so the technique earns its keep on quality while losing on cost. Neither result is expressible in a price-times-tokens model
• Context Lifecycle Management — the cost axis this page says nobody publishes, from the context side: pruning tokens breaks the provider prefix cache, so token reduction and billed cost can move in opposite directions. Self-GC prices the commit (CommitBenefit ≈ N_future·(C−C′) − L_cache_break − L_GC) and reports a 0.3 expected-pruning break-even — the corpus's first published threshold for "is this context cut worth the cache break?" — alongside a measured 10–15% production input-token reduction that it explicitly declines to call a billed-cost saving
• Client-Side Agent Optimization — the empirical counterweight: model selection evaluated at the level of full pipeline combinations rather than per-workload, where the strongest model can be the worst component. Cursor's four mixes are the same abstraction run on a production build, and they add the cost-side coupling: a planner's bill includes the tokens it causes its workers to spend
• Parallel Agent Orchestration — the harness these cost figures were measured inside, and why the comparison is credible at all: matched task, matched models, matched time budget, held-out oracle
• Cursor — the vendor publishing the figures, and the conflicts of interest to net out of them
• Optimizer–Evaluator Decoupling — the advisor strategy is that invariant reached from the cost side; the advisor is an independent grader that also happens to be cheaper than running it as the executor
• Production-Sourced Evaluation — the vendor's own recommendation once benchmarks saturate: curate the eval from production traffic
• Evals as Product Spec — what the selection framework defers to when its two hardest questions come due
• Large-Scale Test-Time Compute — effort level is the inference-budget thesis productized as a dial; "how capable is the model?" is ill-posed without naming the budget, and the guidance concedes this by treating class and effort as one grid
• Measuring Beyond Accuracy Saturation — the research-side statement of the Opus-vs-Fable problem: benchmark scores that no longer separate models, while other measurable axes still do
• Unproductive Self-Verification — the failure mode that inverts the cost-per-task argument: more capability spent on rumination rather than convergence
• Inference Efficiency as Capability — the supply-side twin: the same price-vs-capability trade seen from the model builder, where Gemini 3.5 Flash-Lite's +67% output price buys +78% relative on one benchmark and +2% on another
• Compute-Controlled Benchmarking — the evaluation-side statement: a published price is a rate, and no vendor publishes the tokens-per-task that turns it into a bill
• Claude Code Best Practices — where this guidance is applied per session (effort defaults, context budget) and where the sub-agent-overhead question that this page's cheap-fan-out counter-current bears on lives
• Shared Harness, Differentiated Surfaces — the same choice made the other way: OpenAI collapses model class × effort onto a one-dimensional slider and hides the rest behind an opinionated default, rather than publishing a selection rule
• Harness Shrinkage as Models Improve — the third answer to the same question, and the only one that isn't a rule at all: hand the selection to the model. Replacing a hard rule with "use your judgement" is the move Anthropic ran on its own system prompt, and a practitioner has now pointed it at model routing — a published decision procedure, a hidden vendor default, and a delegated judgement are three different places the choice can live
• Anthropic — publisher of the guidance
• Claude Fable 5, Claude Opus 5, Claude Sonnet 5, Mythos Model — the classes being chosen between
• Dynamic Workflows: An Algebra for Agents — the argument at its largest published scale: ~$165k of tokens (5.9B uncached input, 690M output, 72B cached reads) against a stated counterfactual of three engineer-years, on a task the team says it would otherwise not have attempted — and, per the independent audit, a figure bounded at cost-to-green rather than cost-to-shipped
• Verification as the New Bottleneck — the accounting analogue: the work that is easy to price (getting to green) is not the work that dominates the bill (getting to shipped)
• Agent-Authored Harness Optimization — the campaign-cost question asked of an optimization campaign itself: ~$680 and ~1B tokens spent to move one benchmark run from $79 to $49.8, with no stated payback boundary — the same cost-to-green-vs-cost-to-shipped ambiguity as the Bun figure, one level up
• Knowledge-Centric Self-Improvement — a self-improvement comparison run entirely in dollars rather than tokens, for this page's reason ("so they reflect what cached and uncached tokens actually cost and remain comparable across methods with different cache profiles"), with meta-loop tokens charged to the baselines that spend them. The result is the rarer shape: solve rate up and cost down against every arm (SWE-bench Pro $208 vs DGM's $713), so cost-per-task fell without any accuracy trade to argue about

Open Questions#

• Does "cost-per-task is lower for more intelligent models" survive measurement on non-Anthropic production traffic? The claim is stated without data and the published curves are explicitly illustrative. Still open, with a near-miss (2026-07-30): DeepMind's Gemini 3.5 Flash-Lite card is a non-Anthropic instance of the shape — +67% output price, a full agentic tier of capability — but reports no tokens-per-task, so it supplies the premise and not the measurement. Partially answered (2026-08-03), and it splits: Cursor's four model mixes are the measurement — non-Anthropic infrastructure, a real four-hour workload, matched time budgets, matched quality, published dollars. Within the planner role the claim holds: the more expensive Fable 5 planner billed slightly less than Opus 4.8 at roughly twice the per-token price, because it emitted far fewer planning tokens. At the level of the whole run it fails: the same Fable configuration came out substantially more expensive because its workers burned several times the tokens, and the most expensive run of all was the strongest model used throughout ($10,565 versus $1,339). So the thesis appears to be a claim about a role, not about a system, and no source yet measures it on a single-agent workload outside Anthropic. Sharpened, with the first counter-datum (2026-08-03): Writer's harness swap publishes per-model cost and per-model quality for six models on non-Anthropic infrastructure under one pinned price table, and cost per task rises monotonically with model strength on that workload — quality per dollar is worst for the two strongest models (Palmyra X6 3.16, Sonnet 4.6 3.27) and best for the cheapest (Qwen 3.6 4.44). It is a controlled bench rather than production traffic, the arms are not iso-quality, and the capability spread is only eight points, so it does not close the question — but the sign is wrong for the vendor guidance and the paper's own conclusion is that the model menu is the smaller lever anyway. Closest yet, and it splits again (2026-08-04): Databricks' internal coding bench — real engineering tasks on its own multi-million-line codebase, measured by neither Anthropic nor a model vendor — puts Opus 4.8 at $1.94/task and 87% success against Sonnet 5 at $2.09 and 81%, on tokens ~1.7× cheaper. Within Anthropic's own line the claim therefore holds, in the long-horizon coding regime where its mechanism should be strongest, and this is the first time it holds on a third party's real codebase. Against the wider menu it fails: open-weight GLM 5.2 is statistically tied with Opus 4.8 on quality at $1.28/task. So the surviving form of the rule is about tokens-to-completion, not about price tier — a cheaper model is dearer per task when it burns more tokens and finishes less often, which is contingent, not structural. Still not closed: case-study secondary reporting, production-derived tasks rather than production traffic, and no n, variance or per-arm methodology behind "statistically tied." The article's own cited generalization (arXiv 2603.23971 — a third of comparisons invert; Gemini 3 Flash 80% cheaper listed, 38% dearer in practice) is the paper most likely to settle this and is not yet ingested.
• Does the advisor strategy's result (within 10% of the advisor's score at 63% of its price) generalize beyond SWE-bench Pro and the Sonnet-5/Fable-5 pairing — and where is the crossover at which advisor calls cost more than they save?
• Is "start with the strongest model" safe inside multi-role pipelines, given AgentOpt's finding that the strongest model was the worst planner? Anthropic's Sonnet-for-sub-agents note hints at a boundary it never states. Partially answered (2026-08-03): Cursor's production swarm says the safe form of the rule is positional — strongest model as planner, cheapest capable model as worker — and that running the strongest model in every role is the single most expensive way to reach the same grade. It also dissolves the apparent conflict with AgentOpt: Opus was the worst HotpotQA planner because it answered from parametric knowledge instead of delegating, and Cursor's architecture makes that impossible ("a planner never implements"). The failure is a property of harnesses that let a planner execute, not of strong models in the planner seat. Still unsettled: whether the ordering survives on tasks where the worker's job is judgment-heavy rather than instruction-following, which is the regime Cursor's own framing exempts.

Sources#

• Claude models explained: choosing the best model for your use case — Anthropic, July 2026 (vendor-claim): the start-smart default, class taxonomy, four selection questions, advisor strategy, saturation → custom evals
• Gemini 3.5 Flash-Lite Model Card — Google DeepMind, 2026-07-21 (vendor-claim): price rows inside the benchmark table, a +67% output-price rise across one generation of the efficiency tier, and no token or turn counts to price a task with
• Codex from 0 to 10M Users: Building ChatGPT Work - Akshay Nathan, OpenAI — Latent Space, 2026-07-28 (practitioner-opinion): OpenAI's opposite exposure philosophy — "32 options" collapsed onto a one-dimensional slider behind an opinionated default, with the same escalation trigger and the same absence of tokens-per-task
• How is the Bun Rewrite in Rust Going? — Tom Lockwood, lockwood.dev, 2026-07-27 (case-study, independent): the outside view of the Bun port's public artifacts, and the argument that a campaign's headline token cost omits CI, employee time, and the post-merge tail
• The Harness Effect: How Orchestration Design Sets the Token Economics of Enterprise Agentic AI — Sayed Ali et al. (33 authors, all Writer, Inc.), arXiv 2607.06906, 2026-07-08 (empirical, total vendor COI — Writer's harness, Writer's baseline, Writer's model in the panel, last author is co-founder/CTO): §6.2 and Table 4 for the per-model baseline→harness cost figures and the 36%-model-menu-versus-33–61%-harness comparison, Table 6 for the per-model capability means used in the quality-per-dollar column above, §7.1 for the "$/Mtok compares p; the bill is p × τ" framing. Table 2 is cell-collapsed and Table 7 row-shifted in the raw parse — neither is cited here; the model roster comes from §5.3 prose. Full treatment and parse warnings at Orchestration Sets Token Economics
• The price is wrong: AI cost calculation has to consider task completion rates, not just token costs — Thomas Claburn, "The price is wrong," The Register, 2026-07-13 (case-study, secondary reporting: a news article relaying Databricks' benchmark blog post and Matei Zaharia's social posts; the primary at databricks.com is not in the corpus). Supplies the $1.94 / $2.09 / $1.28 per-task figures, the 87% / 81% success rates, the "around 1.7x cheaper" per-token ratio, the SWE-Bench-is-tuned-for motivation, and the cited arXiv 2603.23971 result. 644 words, no tables. Ingest hazard worth remembering: WebFetch silently dropped the GLM 5.2 figure and the Gemini 3 Flash example, and both exist in the raw only because the body was rebuilt from curl'd HTML — the two figures most load-bearing against the vendor guidance are the two that nearly vanished
• Fable's judgement — Simon Willison, "Fable's judgement," 2026-07-03 (practitioner-opinion, 460 words): the self-delegated model-routing prompt, the auto-saved memory file's stated rationale (implementation to a cheaper model, judgment/review/synthesis in the main loop), and the unquantified outcome. The tip is second-hand from Jesse Vincent and the underlying judgement-over-rules advice second-hand from Cat Wu and Thariq Shihipar; no measurement of any kind
• Agent swarms and the new model economics — Wilson Lin, cursor.com, 2026-07-20 (case-study, vendor-authored): "Results across model mixes" and "Model economics" — the four planner/worker configurations, the $1,339–$10,565 total range at matched quality, the ≥69%-of-tokens/one-third-of-cost worker split, the $9,373 → $411 worker-spend comparison, and the Fable-versus-Opus planner inversion. Footnote 1 records that the solo Opus 4.8 and Fable 5 runs (hatched bars in the cost chart) were graded only informally, so no quality claim attaches to them