Effective Compute Scaling

Sources#

From AGI to ASI

Summary#

The "From AGI to ASI" report anchors its forecasting in effective compute — a single growth rate that multiplies three independently-improving factors. Epoch estimates it at ≈ 10× per year (one order of magnitude annually), which the authors call a conservative lower-end figure. This is the only pathway to ASI with historic data to fit forecasting models on, which is why it is "business as usual" scaling and the report's most-tractable quantitative handle.

The three multiplicative factors#

Factor	Rate	Note
Hardware manufacturing (Moore's law & related)	~1.5×/yr	compute-per-dollar, sustained for six decades — least uncertain factor
Compute investment growth	~2.5×/yr	growing hardware spend over the last decade
Algorithmic efficiency	~3×/yr (Epoch: up to ~6×)	FLOPs to hit a fixed performance threshold (e.g. AlexNet-on-ImageNet) falling ~2× the rate of Moore's law; mostly many incremental gains stacking, not rare breakthroughs

Hardware × investment ≈ 4× per year in compute spent on the largest training runs. Folding in algorithmic efficiency (the "as if the hardware fleet grew" effect) gives ~10×/yr effective compute (1.5 × 2.5 × 3 ≈ 11.25, rounded down). Sustained for a decade → a 10,000× increase over today. Uncertainty compounds across factors, so the true rate could be substantially higher or lower — and may be accelerating.

The decisive open question: does compute become capability?#

Compute growth is tractable to forecast; how it translates into new capabilities is not. Three regimes are possible: diminishing returns (slow progress), proportional (exponential), or — under recursive improvement — super-exponential. The report's key nuance: even if individual-model progress plateaus, continued compute growth still raises aggregate capability by running more instances, faster, thinking longer. "Mere" quantitative scaling can thus unlock what looks like qualitative advance — e.g. 1,000 AGI instances → 10,000 in a year → 100 million in five years (or 1M instances at 100× speed). Whether that constitutes ASI is the spine of the scaling debate (see The Bitter Lesson: "more compute → more search → more intelligence", with the catch that naive brute-force search fails outside toy domains; gains come from better priors/heuristics).

The data wall#

The first major friction: running out of high-quality data to pretrain ever-larger models, estimated to bite later this decade (Villalobos et al. 2024). Model size is outpacing the production of novel human text. Counters the report weighs:

Synthetic / self-generated data — risks degeneration on naive iterated training (Shumailov et al. 2024), but test-time-search outputs distilled back (AlphaZero-style) can produce "just-beyond-frontier" data; with billions of users spending test-time compute, this could be a real recursive-improvement engine.
Simulation & interaction data (RL, multi-agent, generative agent-based models) — scales straightforwardly with compute where good simulators exist; e.g. DeepMind's Adaptive Agent.
Other modalities (image/audio/video) extend the runway but can't grow fast enough on human production alone.

Verdict: likely a friction, not a fundamental blocker — if ASI is driven by scaling, data generation can plausibly scale at a similar pace via compute.

Economic & resource frictions#

If progress relies mainly on scaling, the binding question is whether the economic cost of scaling over many orders of magnitude is sustainable — which depends circularly on the economic returns AI produces. Adjacent constraints: energy build-out, land/water, rare earths, and the environmental footprint (with exotic proposals like orbital datacenters carrying their own risks). Even with raw FLOPs available, memory bandwidth and interconnect bottlenecks can cap effective utilization. If instead progress comes from algorithmic innovation / self-improvement / paradigm shifts, required economic inputs scale more slowly and this is only a marginal friction.

Which constraint binds, and where (Musk, July 2026)#

Musk's Economist interview sharpens the energy friction above into a claim about geography, and it is worth recording because the report treats energy as one undifferentiated constraint (prediction tier — an unmeasured assertion by an interested party, stated here as his claim):

Outside China the binding constraint is electricity, not chips. "The rate at which AI chips are being made exceeds the rate at which new electricity is coming online" — and he puts the pinch specifically on power and cooling, since "the power demands of the AI chips are very very high." He separately dismisses the water-footprint concern as "negligible, almost nothing," which cuts against the land/water framing above.
Inside China the binding constraint is chips, because of US export controls — but China is "closer than most people realize to solving the lithography problem," and Chinese labs are already competitive on far less compute (Kimi K3).
The electricity gap is the structural asymmetry. China "has more electricity than the United States, Europe and India combined already," and he guesses it reaches ~4× US production. On his analysis this means whoever solves their own constraint first leads: "if they had a lot of compute there's a good chance that they would be the leaders, and at some point they probably will have a lot of compute."
Orbital datacenters are the named workaround, and he states the consequence precisely: "once we address the power constraint with AI data centers in space, then the constraint will once again be chips outside of China" — i.e. the exotic proposal this section mentions is, in his framing, a constraint swap rather than a removal.

Read against the section above, the useful contribution is the ordering claim: the report lists energy among several adjacent constraints, while Musk asserts it is currently the binding one for everyone except China. Nothing in the corpus measures this either way, and he has a direct commercial interest in both the power build-out and the orbital-datacenter answer.

Connections#

AGI-to-ASI Pathways — scaling is pathway 1; this page is its quantitative engine and its two headline frictions (data wall, economics)
Intelligence Explosion Dynamics — compute growth is the substrate a recursive loop accelerates; whether returns are proportional or hyperbolic decides the regime
Task Time-Horizon Scaling — METR's time-horizon trendline is the capability-side complement to this compute-side curve (Whitfill et al. model time-horizon growth under compute projections)
The Bitter Lesson — "is scaling enough?" is the bitter lesson as a forecasting question; search needs good priors, not just more FLOPs
Multi-Agent Collective Intelligence — the "plateaued model but more instances" argument routes scaling into collective capability
Fundamental Limits of ASI — why capability forecasting must be empirical-first: theory yields only vacuous negatives
Advantages of Digital Intelligence — these are precisely the AI properties that scale with compute, so more effective compute widens the human–AI gap
Universal AI (AIXI) — AIXI approximations are guaranteed to improve with compute, but brute-force versions need prohibitively fast growth for linear intelligence gains — the theoretical backstop to "is scaling enough?"
Inference Efficiency as Capability — the algorithmic-efficiency term seen from the inference side: KV-cache, quantization, and speculative-decoding gains raise effective compute per dollar at serving time, not just at training time
Cross-Lab Pre-Release Review — why the compute geography above is a governance fact: if the power constraint binds outside China and the chip constraint inside it, both are temporary, and a US-only review club governs a shrinking share of frontier releases
Researcher Uplift from Code Output — the compute-side term in the labor-vs-compute R&D decomposition: Kwa notes compute tripling yearly already grows research input ~1.6×/yr (via compute's ~0.45 exponent) independent of any labor uplift, so both inputs compound
Balance-of-Power Superintelligence — Zuckerberg's RSI section is a runaway in this page's algorithmic-efficiency term (100x per gigawatt), and his proposed remedy is to grow the denominator: labs and clouds collectively building enough compute that the majority stays pointed at human-chosen goals

Open Questions#

When does more compute reliably yield more intelligence — only for some problem classes, or generally? Can quantitative and qualitative scaling be traded off?
Can data generation (synthetic, simulated, interactive) actually keep pace with model-size growth, or does the data wall bind first?
When (if ever) does scaling become economically unviable, and how do hardware/software-efficiency trends move that point?

Sources#

From AGI to ASI — Section 2 (effective-compute growth factors), Section 5.1 (scaling pathway), Section 5.5 (data wall, economics), Table 4