Scale-Dependent Prompt Sensitivity

Sources#

• Agent swarms and the new model economics
• Brevity Constraints Reverse Performance Hierarchies in Language Models
• Prompt Design at Scale: How Format, Instruction Count, and Context Length Shape Instruction Adherence and Hallucination in Large Language Models

Summary#

An empirical finding by Hakim (2026) that reframes documented "inverse scaling" cases as a prompt-engineering problem rather than a capability problem. On 7.7% of standard benchmark problems (115 of 1,485 across GSM8K, BoolQ, ARC-Easy, CommonsenseQA, MMLU-STEM), larger language models underperform smaller ones by 28.4 percentage points — despite 10–100× more parameters. Causal intervention shows brevity constraints recover 26pp of accuracy on large models and completely reverse the hierarchy on mathematical and scientific reasoning benchmarks. The mechanism — spontaneous scale-dependent verbosity ("overthinking") — implies large models possess superior latent capability that universal prompting masks. The deployment consequence: optimal prompting strategies must be scale-aware, not uniform.

Details#

Scope of the Study#

31 models from 0.5B to 405B parameters spanning Llama, Qwen, Gemma, and Mistral families, evaluated across 1,485 problems from five benchmarks → 46,035 individual evaluations. Greedy decoding (do_sample=False) for reproducibility. Small/large split at ≤10B vs. >70B parameters.

Three Problem Categories#

Problem-level analysis reveals benchmark evaluation is more informationally sparse than aggregate scores suggest:

• Non-discriminative (27.1%) — ceiling effects (17.3%, all models succeed) or floor effects (9.8%, all models fail). Roughly one-third of evaluation effort yields no signal about relative capability.
• Normal scaling (48.1%) — larger models outperform smaller ones as expected.
• Inverse scaling (7.7%) — smaller models systematically beat larger ones.

Inverse Scaling: Not Adversarial, Not Rare#

Prior inverse-scaling work (Inverse Scaling Prize, BIG-Bench) focused on constructed tasks designed to expose failure modes — memorization of rare patterns, distractor reasoning, spurious correlations. Hakim's contribution is that inverse scaling shows up at meaningful rates on standard capability benchmarks: BoolQ 11.3%, CommonsenseQA 9.7%, ARC-Easy 9.3%, GSM8K 4.3%, MMLU-STEM 3.9%.

Effect size is categorical rather than marginal: Cohen's $d = 1.34$ (conventional "large" threshold is 0.8). Mean gap 28.4pp favoring small models. Mann-Whitney U yields $p < 0.001$ on every dataset.

Within-family analysis rules out architecture artifacts:

• Llama: smaller variants (2B–13B) hit 48–68% vs. larger (70B–405B) at 41–54%
• Qwen: 0.5B–7B at 62–83% vs. 32B at 40%
• Pearson $r = -0.58$ between family size and accuracy on inverse problems ($p = 0.029$)

Overthinking as Causal Mechanism#

Hypothesis: large models generate excessively verbose responses that obscure correct reasoning. Supported by both correlational and causal evidence:

Correlational — response length correlates negatively with large-model accuracy on inverse problems ($r = -0.43$). Notably, large models don't generate more explicit reasoning steps (9.1 vs. 10.5 for small models) but produce 59% longer total output (202 vs. 127 tokens). They elaborate within steps rather than taking more steps.

Causal — intervention on all 115 inverse problems with three conditions (control, brief, direct) on seven models:

• Brief constraints: <50 words for math, <10 words for reading comp
• Direct: final answer only, no reasoning
• Result: large-model accuracy +26.3pp under brief; small-model accuracy −3.1pp. Gap reduction 67% (44.2pp → 14.8pp). Paired $t = 7.80$, $p < 0.0001$.
• Direct format: gap compresses to 7.8pp (82.3% reduction) but both sizes lose accuracy, suggesting some reasoning is beneficial.

Token generation fell from median 197 → 78 under brief (60% reduction) — intervention manipulated the hypothesized mechanism.

Complete Hierarchy Reversals#

The strongest claim: on two datasets, brevity constraints don't just close the gap — they flip it.

• GSM8K: +13.1pp favoring small → −7.7pp favoring large
• MMLU-STEM: +27.3pp favoring small → −15.9pp favoring large

These reversals are the argument that standard evaluation masks rather than measures large-model capability. Llama-3.1-405B goes from 41.5% (control) to 67.2% (brief) on inverse problems — a 25.7pp unlock.

Where Brevity Hurts: BoolQ#

Dataset heterogeneity is critical: BoolQ's gap widens slightly under brevity (23.5pp → 24.3pp). The explanation: BoolQ requires cross-sentence passage integration where elaboration is functional rather than excessive. Brevity constraints are not a universal prescription — they help on self-contained problems (math, science) where overelaboration accumulates errors, and hurt on problems where explicit reasoning is load-bearing.

Contamination Ruled Out#

Three independent tests confirm inverse scaling reflects genuine capability differences, not memorization artifacts:

• Response diversity: 89–100% unique responses across datasets (contradicts template memorization)
• Length variability: CV 0.31–1.21, all exceeding the memorization threshold (CV < 0.15)
• Error patterns: 40–81% over-reasoning failures vs. 13–23% memorization avoidance
• Fisher's exact test: no association between contamination indicators and inverse scaling ($p = 0.23$)

RLHF Length-Bias Hypothesis#

Speculated origin: RLHF reward models exhibit length bias — annotators conflate thoroughness with quality. Larger models have greater capacity to satisfy length-reward signals during training and internalize verbose generation more deeply. Consistent with verbosity differences being larger in instruction-tuned than base variants. Suggests a tractable mitigation at training time: reward model calibration that penalizes overelaboration on concise-answer problem types.

Practical Implications#

1. Aggregate benchmarks systematically underestimate large-model capability on a predictable subset — differences comparable to an entire model generation separate standard from optimized prompting for frontier models.
2. Problem-aware routing + scale-specific prompting is the deployment pattern: detect overthinking-prone problem types and apply brevity selectively.
3. Cost–capability improves simultaneously — brevity both raises accuracy on inverse problems and reduces tokens (smaller spend).

A production consequence: a model excluded from a comparison for being too prompt-sensitive#

The page's claim that prompting must be scale-aware is usually a tuning recommendation. Cursor (Agent swarms and the new model economics, 2026-07-20, case-study) supplies the version where the cost is paid instead of the tuning: they wanted GPT-5.6 Sol as the frontier arm of their agent-swarm comparison and dropped it, reporting that "the new model appears more sensitive to literal and emphasized wording than the others we tested, and we encountered runaway spirals unlike anything the other models produced." With no time to tune prompts for a just-released model — and with the observation that tuning for one model while leaving the rest untouched would invalidate the comparison — they fell back to GPT-5.5.

Two things worth carrying. Prompt sensitivity is a per-model property that resets on release, so a harness tuned across a model generation carries an implicit incumbency advantage that no benchmark reports. And the runaway-spiral failure is this page's overthinking mechanism at agent scale: not a longer answer, a non-terminating loop. case-study, one vendor, one unreleased-recently model, and reported only as a footnote — but it is a rare public record of a frontier model being benched for prompt fragility rather than capability.

The controlled version, and the lever with no sign (Eliav, July 2026)#

Cursor's exclusion is case-study, one vendor, one model, one footnote. Eliav 2026 (arXiv 2607.19257, empirical) is the controlled version of the same claim, and it crosses prompt format against both scale axes — instruction count (10→160) and context length (2k→512k) — on one held corpus across five models. Its headline is not about markdown at all: neither a format's effect nor its sign can be predicted without specifying the scale at which it was measured.

• No reliable format winner, anywhere. Markdown-minus-plain adherence deltas stay within 2.1pp and are not consistently signed across N for four of five models. The one clean directional signal runs against convention: Qwen 35B favors plain text at five of six instruction counts, widening to 4.8pp at N=160.
• Both pre-registered structure hypotheses fail. "Any structure beats plain" and "tables specifically beat everything" were registered as the plausible outcomes; the data supports neither. Prose — the format both hypotheses predicted would lose — is best or tied-best at three of the five long-context cells with genuine spread. Plain text is simultaneously the worst performer at one cell (Claude Haiku @ 128k, 0.383 against 0.817–0.867 for the other three) and the single best at another (Sonnet 5 @ 512k, 0.867 against 0.530–0.667). The generalization is the absence of a ranking, not a ranking.
• Rankings reverse between adjacent rungs of the same model, verified against the paper's fixed anchor set so it is not a question-composition artifact.
• Placement is a lever of comparable size that nobody tests. Moving an identical instruction block between system prompt and user turn changes adherence by up to 8.7pp at N=160 — larger than the format effect for four of five models — and its direction is model-specific: it helps two, hurts two, and does nothing for the fifth. See Instruction Compounding.
• A format effect can be a plumbing failure rather than a comprehension one. Gemini Flash alone collapses on prose and table at N≥40 (−18.1pp and −13.7pp against plain, where every other model stays within 2.5pp). Manual transcript inspection found the visible response often contains no essay at all but a tail fragment of an internal rule-verification pass — hidden-reasoning leakage, not a scoring artifact and not a retrieval deficit. A pooled "format effects are small and inconsistent" summary would have hidden it entirely.

Why this sharpens rather than repeats this page. Hakim's brevity result is a prompt lever with a predictable sign — constrain length, and large-model accuracy rises on overthinking-prone problems, reliably enough to deploy behind a router. Format is a prompt lever with no sign at all: the same rendering is the best performer for one model at one scale and the worst for another. Both findings say the prompt surface is not cosmetic; only one of them yields a rule you can apply without re-measuring. That also upgrades the Cursor observation above from a vendor anecdote to a measured property — prompt sensitivity is per-model, and now demonstrably per-scale-point within a model, so a harness tuned at one instruction count or context length carries no guarantee at another.

Two priors the paper bounds rather than contradicts: Oh et al. (2024) report tables yielding ~40% relative accuracy gain over natural text, and Liu et al. (2025) find a table-first format-preference hierarchy — both measured at fixed, moderate context. Eliav's result is that whatever advantage exists there need not survive as context grows toward a model's own effective ceiling.

Same caveats as everywhere this paper is cited: sole author, single lab, arXiv preprint without peer review, 20 trials per instruction cell and 60 questions per long-context cell, and every instruction tested is a hard, exactly-checkable output constraint.

Limitations#

• Greedy decoding only; unknown whether temperature sampling changes the 7.7% rate.
• Knowledge/reasoning tasks only; no generative task evaluation.
• Doesn't establish why large models overthink (training dynamics? architecture? emergent?).
• The causal sample selected models partly for their stronger overthinking tendency (44.2pp gap vs. 28.4pp in the full analysis), so 67% reduction is an upper-bound estimate.

Connections#

• Agentic Loops Overtake Bespoke Systems — smaller Gemini models solved nothing — a scale-sensitivity result
• Jagged Intelligence (Ghosts, Not Animals) — overthinking on easy tasks is a jagged-capability failure mode
• Client-Side Agent Optimization — AgentOpt's HotpotQA finding (Claude Opus 4.6 is the worst planner, bypasses solver via parametric knowledge) is this paper's overthinking mechanism surfaced as a routing failure. Together, the two papers imply systematic large-model misuse with two available mitigations: route around it (combo selection) or constrain output (brevity)
• Claude Code Best Practices — the context-window-as-primary-constraint framing pairs naturally with brevity: shorter completions also preserve more of the context budget. Claude Code's emphasis on verification-driven development becomes especially important when large-model output is systematically verbose in ways that can mask errors
• Agent Harness Engineering — enforcing output-length invariants at the harness level (via system prompts, structured output schemas, or response validators) is a mechanical-enforcement pattern that directly addresses scale-dependent overthinking. Falls under "enforce invariants, not implementations"
• LLM-Driven Vulnerability Research — the vuln-research scaffold's paragraph-level prompt ("find a security vulnerability in this program") succeeded partly because the task rewards thoroughness, which is the behavior larger models over-produce. This is a case where large-model verbosity is aligned with task utility rather than adversarial to it
• Claude Opus 4.7 — Hakim's findings were measured on Opus 4.6. 4.7's literal instruction following may make brevity constraints more effective (the model obeys word caps) while its higher default effort and extra per-turn thinking may increase baseline verbosity. Net direction is an open empirical question
• Interactivity Benchmarks — another case of a paper inventing its own evaluation framing (FD-bench extensions, TimeSpeak/CueSpeak, visual-proactivity benchmarks) to surface a phenomenon standard benchmarks miss; same epistemic strength-and-soft-spot as this paper's BoolQ-exception framing
• Hermes Agent — /verbose modes and bounded memory implicitly cap output length; the brevity-constraint findings predict accuracy gains from that
• The Verifiability Thesis — Karpathy's "jagged because of what the labs trained on" is the model-capability story behind why scale doesn't uniformly help
• AI-Driven Formal Proof Search — a sharp threshold case: smaller Gemini variants solved zero open problems while Gemini 3.1 Pro succeeded — capability here is gated, not graded
• Large-Scale Test-Time Compute — the overthinking result is that thesis's counter-case: more test-time compute (longer generations) reduces accuracy on ~7.7% of problems, so inference budget is a resource to allocate rather than maximize; brevity is budget-down, and it unlocks latent large-model capability the default masks
• Parallel Agent Orchestration — where the swarm that benched GPT-5.6 Sol is described; the runaway-spiral failure is this page's mechanism showing up as non-termination inside a long-running agent rather than as a long answer
• Cursor — the vendor reporting the exclusion, in a footnote to its own model comparison
• Agent Context Files — where the format finding bites hardest in practice: every CLAUDE.md / AGENTS.md / SKILL.md in the ecosystem is markdown injected into the system prompt, and neither the rendering nor the slot has any measured justification
• Instruction Compounding — the instruction-count half of the same experiment, and the reason format is not the lever to reach for: adherence floors by ~80 simultaneous rules whatever the rendering, so reformatting a prompt past that point is polish on a dead metric
• Context Window Smart Zone — where the same paper's long-context half lives: recall holds to 64–128k, then degrades in proportion to each model's own effective ceiling, and refusal rather than fabrication is what rises
• Benchmark Score Redundancy — the matrix-level companion to this item-level result: this page finds ~27% of benchmark problems are non-discriminative (ceiling/floor); BenchPress finds the whole 84×133 benchmark matrix is rank-2. Both quantify how much less independent signal aggregate benchmarks carry than their count implies

Derived#

• When to Use Claude Opus 4.6 for Work — the brevity-intervention and BoolQ-exception findings feed directly into deployment rules for Opus 4.6
• Opus 4.6 → 4.7 Changes and Multi-Agent Coding Considerations — brevity constraints and harness-level length enforcement applied to an Opus 4.7 multi-agent coding team

Open Questions#

• Does the RLHF length-bias hypothesis replicate when tested against base (non-instruct) model variants directly? If verbose generation were primarily pretrained, base-model verbosity differences should match instruct-model differences.
• What problem characteristics predict prompt sensitivity? An automated classifier would make scale-specific prompting deployable.
• How does the overthinking effect interact with tool-using agents? If brevity helps large models but tools require structured reasoning, the optimal prompt is not uniformly brief.
• Do reasoning models (o1, DeepSeek-R1 style) exhibit different overthinking dynamics than instruct models? Their trained behavior is explicitly to generate long CoT — does brevity intervention hurt them?
• Is BoolQ's functional-elaboration exception a clean taxonomy boundary, or does every task type have a context-dependent optimal length?

Sources#

• Brevity Constraints Reverse Performance Hierarchies in Language Models
• Prompt Design at Scale: How Format, Instruction Count, and Context Length Shape Instruction Adherence and Hallucination in Large Language Models — Netanel Eliav, arXiv 2607.19257, 2026-07-21 (empirical, sole author, single lab, not peer reviewed): §4.3 and Table 5 the markdown-minus-plain deltas, §4.4 the placement effect, §4.5 and Table 6 the Gemini Flash prose/table collapse and its hidden-reasoning-leakage diagnosis, §5.3 and §6 the two failed pre-registered orderings, Table 7 the non-ceiling recall cells, §2.1/§2.4 the positioning against Sclar et al. 2024, He et al. 2024, Oh et al. 2024 and Liu et al. 2025. Tables 5, 6 and 7 were reconciled and are clean; the bold best-performer markers in Table 7 are lost in the raw parse but the surrounding prose names the winner. The paper's Table 1 model roster is cell-collapsed and is not cited here — see the Sources note on Instruction Compounding
• Agent swarms and the new model economics — Wilson Lin, cursor.com, 2026-07-20 (case-study, vendor-authored): footnote 2 — GPT-5.6 Sol dropped from the swarm comparison for sensitivity to "literal and emphasized wording" and runaway spirals, with GPT-5.5 substituted. This footnote was silently dropped by the initial fetch and restored at ingest against the raw HTML