Effective Compute Scaling

Sources#

• From AGI to ASI

Summary#

The "From AGI to ASI" report anchors its forecasting in effective compute — a single growth rate that multiplies three independently-improving factors. Epoch estimates it at ≈ 10× per year (one order of magnitude annually), which the authors call a conservative lower-end figure. This is the only pathway to ASI with historic data to fit forecasting models on, which is why it is "business as usual" scaling and the report's most-tractable quantitative handle.

The three multiplicative factors#

Factor	Rate	Note
Hardware manufacturing (Moore's law & related)	~1.5×/yr	compute-per-dollar, sustained for six decades — least uncertain factor
Compute investment growth	~2.5×/yr	growing hardware spend over the last decade
Algorithmic efficiency	~3×/yr (Epoch: up to ~6×)	FLOPs to hit a fixed performance threshold (e.g. AlexNet-on-ImageNet) falling ~2× the rate of Moore's law; mostly many incremental gains stacking, not rare breakthroughs

Hardware × investment ≈ 4× per year in compute spent on the largest training runs. Folding in algorithmic efficiency (the "as if the hardware fleet grew" effect) gives ~10×/yr effective compute (1.5 × 2.5 × 3 ≈ 11.25, rounded down). Sustained for a decade → a 10,000× increase over today. Uncertainty compounds across factors, so the true rate could be substantially higher or lower — and may be accelerating.

The decisive open question: does compute become capability?#

Compute growth is tractable to forecast; how it translates into new capabilities is not. Three regimes are possible: diminishing returns (slow progress), proportional (exponential), or — under recursive improvement — super-exponential. The report's key nuance: even if individual-model progress plateaus, continued compute growth still raises aggregate capability by running more instances, faster, thinking longer. "Mere" quantitative scaling can thus unlock what looks like qualitative advance — e.g. 1,000 AGI instances → 10,000 in a year → 100 million in five years (or 1M instances at 100× speed). Whether that constitutes ASI is the spine of the scaling debate (see The Bitter Lesson: "more compute → more search → more intelligence", with the catch that naive brute-force search fails outside toy domains; gains come from better priors/heuristics).

The data wall#

The first major friction: running out of high-quality data to pretrain ever-larger models, estimated to bite later this decade (Villalobos et al. 2024). Model size is outpacing the production of novel human text. Counters the report weighs:

• Synthetic / self-generated data — risks degeneration on naive iterated training (Shumailov et al. 2024), but test-time-search outputs distilled back (AlphaZero-style) can produce "just-beyond-frontier" data; with billions of users spending test-time compute, this could be a real recursive-improvement engine.
• Simulation & interaction data (RL, multi-agent, generative agent-based models) — scales straightforwardly with compute where good simulators exist; e.g. DeepMind's Adaptive Agent.
• Other modalities (image/audio/video) extend the runway but can't grow fast enough on human production alone.

Verdict: likely a friction, not a fundamental blocker — if ASI is driven by scaling, data generation can plausibly scale at a similar pace via compute.

Economic & resource frictions#

If progress relies mainly on scaling, the binding question is whether the economic cost of scaling over many orders of magnitude is sustainable — which depends circularly on the economic returns AI produces. Adjacent constraints: energy build-out, land/water, rare earths, and the environmental footprint (with exotic proposals like orbital datacenters carrying their own risks). Even with raw FLOPs available, memory bandwidth and interconnect bottlenecks can cap effective utilization. If instead progress comes from algorithmic innovation / self-improvement / paradigm shifts, required economic inputs scale more slowly and this is only a marginal friction.

Connections#

• AGI-to-ASI Pathways — scaling is pathway 1; this page is its quantitative engine and its two headline frictions (data wall, economics)
• Intelligence Explosion Dynamics — compute growth is the substrate a recursive loop accelerates; whether returns are proportional or hyperbolic decides the regime
• Task Time-Horizon Scaling — METR's time-horizon trendline is the capability-side complement to this compute-side curve (Whitfill et al. model time-horizon growth under compute projections)
• The Bitter Lesson — "is scaling enough?" is the bitter lesson as a forecasting question; search needs good priors, not just more FLOPs
• Multi-Agent Collective Intelligence — the "plateaued model but more instances" argument routes scaling into collective capability
• Fundamental Limits of ASI — why capability forecasting must be empirical-first: theory yields only vacuous negatives
• Advantages of Digital Intelligence — these are precisely the AI properties that scale with compute, so more effective compute widens the human–AI gap
• Universal AI (AIXI) — AIXI approximations are guaranteed to improve with compute, but brute-force versions need prohibitively fast growth for linear intelligence gains — the theoretical backstop to "is scaling enough?"

Open Questions#

• When does more compute reliably yield more intelligence — only for some problem classes, or generally? Can quantitative and qualitative scaling be traded off?
• Can data generation (synthetic, simulated, interactive) actually keep pace with model-size growth, or does the data wall bind first?
• When (if ever) does scaling become economically unviable, and how do hardware/software-efficiency trends move that point?

Sources#

• From AGI to ASI — Section 2 (effective-compute growth factors), Section 5.1 (scaling pathway), Section 5.5 (data wall, economics), Table 4