Jeff Dean

Sources#

• Jeff Dean: The 1% Rule for Building in AI

Summary#

Jeff Dean is Chief Scientist at Google, where he has been since 1999 (employee-era: "a 20-person startup"). His build record is the systems substrate most of modern ML runs on or was trained with: MapReduce, BigTable, TensorFlow, the TPU, and Gemini. He also wrote Latency Numbers Every Engineer Should Know, the canonical order-of-magnitude table for distributed-systems reasoning, and co-authored the 2014 knowledge-distillation paper with Geoff Hinton and Oriol Vinyals — the one NeurIPS rejected as "unlikely to have significant impact," and which now produces Gemini's Flash models from the Pro line.

In this corpus he appears once, in a 57-minute YC Startup School 2026 conversation with Diana Hu (practitioner-opinion; note the standing COI — he is Google's Chief Scientist describing Google's hardware, and the TPU efficiency figures below are his own recollection, not a cited measurement).

The method: napkin math against a bottleneck#

Dean's two most-cited origin stories are the same move run twice — take a bottleneck, compute its order of magnitude from first principles, and ask whether a different design point moves it 10–100×.

• 2001, search in RAM. He and Sanjay Ghemawat computed that Google's whole search index would fit in the aggregate RAM of the machines they already had, and shipped the hard-drive→RAM version "in a few days." That is why Google search is fast.
• 2013, the TPU. Deep-learning speech models had just halved the word error rate — "the equivalent of 20 years of advances in speech recognition in just a few months" — but were far more expensive than the old system. The napkin calculation: if every user talked to their phone for three minutes a day, Google would have to double its server fleet. The answer was a chip specialized for low-precision dense linear algebra and nothing else — "it can't run Chrome or Word" — which shipped two years later at 30–80× the energy efficiency and 20–30× lower latency than contemporaneous CPUs and GPUs.

The TPU story carries a design lesson he states explicitly and which is easy to miss: the chip was built before the Transformer existed, so it was deliberately specialized only down to general-purpose linear algebra. "We knew ML algorithms were still evolving, and you didn't want to overspecialize, but you wanted to specialize enough." The bet was on the shape of the arithmetic, not on the architecture that would use it.

His prescription for founders is the method, not the answer: "squint at a problem and think about not necessarily being anchored on exactly how that problem is solved today, but how you would solve it from first principles… get an order of magnitude or two orders of magnitude better."

What he thinks the next bottleneck is#

• Inference hardware, not training hardware. His stated answer to "what's the it fits in memory moment of 2026": high-performance, low-energy inference silicon. "Imagine what you could do with something where the latency is 50× better." Specific directions: minimize data movement; commit to a small number of very low precisions and build only those into the hardware.
• Energy is the unit, and data movement dominates it. A multiply costs about a picojoule; moving the operand in from HBM costs roughly 1000× that. Batching exists only to amortize that ratio — "instead pay a thousand X divided by batch size" — which is why batching and low latency are structurally opposed. Developed on Inference Efficiency as Capability.
• The AI edition of the latency-numbers table. Asked to update his own list, he named: bandwidth from accelerator main memory → on-chip memory → the multiplier unit; energy per multiply; inter-chip interconnect bandwidth; how many chips that bandwidth reaches; and the falloff when you need 10,000 chips instead of 500.

Predictions on record (2026)#

• 2027: automated ML experimentation. ML systems improving themselves by decomposing a goal into sub-problems, running each in a tight automated experiment loop, and reassembling the results — "anything where you can have a measurable objective." See Recursive Self-Improvement.
• Agents running for days or weeks. His nominated widely-held-but-false belief: people underestimate agent run length. His example of what already works — reimplementing whole systems in a different language for better safety or performance properties, which works precisely because the existing code is the specification.
• Grading his own prior prediction. In May 2025 he said AI was at junior-engineer level; in 2026 he calls that "pretty spot on" and names what he underestimated: the rate at which task complexity grew, and how much of the gain would land outside coding.

The 1% rule#

His most quotable contribution here, and the one with its own page: test the general models on your candidate problem, and build where they succeed 0–1% of the time, not 20% — partial success is the signal that the capability is arriving. Full treatment, including its direct collision with Build for the Next Model, on The 1% Rule for Wedge Selection.

Practices worth stealing#

• Turn your own optimization loop into a skill. He and Ghemawat wrote a skill teaching a model their microbenchmark workflow — measure, modify, re-run, widen the benchmark set, measure cache footprint, iterate — so the agent runs the loop unattended. "It really just is us giving the approach we would use as people to the model in a form that it could use." The human-readable ancestor is public: a ~30-page Performance Hints document he and Ghemawat wrote, which people have summarized into model context to improve performance reasoning.
• Build taste by scoring your own forecasts. "Write down a bunch of things you think might be important in the next 12 months… go back and evaluate in 12 months which of these actually seemed important, or which ones did other people in the world go out and create." A deliberate sample-generating protocol for a skill usually treated as innate — see Research Taste as the Human Bottleneck.
• Run the thought experiment that violates the industry's founding assumption. His current example: 60 years of fabrication have assumed every chip of a design is identical and no bit flips. Large-scale distributed systems make the opposite assumption and build reliability above unreliable parts (replication, Reed–Solomon). So: what would you build out of transistors with 20 errors per day instead of one per million years? Redundant signal paths, brain-like. He is careful to add that most such experiments fail for good reasons — "but it's good to revisit those every so often."
• Collect tools, not titles. Career framing: an "amazing tool belt of techniques," always adding, because "you never know when you might come across a problem where you need these four specialized tools rather than these three."

Connections#

• The 1% Rule for Wedge Selection — his 0-1%-not-20% heuristic for what a startup should build; the page it anchors
• Google DeepMind — the lab whose Gemini, AlphaFold, AlphaEvolve and AlphaChip work he cites as the shape of things; his employer's research arm
• Inference Efficiency as Capability — his energy-and-data-movement argument sits underneath this page's architecture-level levers: the picojoule-vs-1000× ratio is why every lever there exists
• Recursive Self-Improvement — his 2027 prediction (automated experiment loops) and his mechanism for making them affordable (learned surrogate validators)
• Research Taste as the Human Bottleneck — he names taste as the scarce skill once agents write everything, and supplies a protocol for building it
• Build for the Next Model — the strategy his 1% rule contradicts when read as market selection rather than feature selection
• Large-Scale Test-Time Compute — his multi-agent-plus-evaluator prescription for long-running agents is inference-time search, stated by a hardware architect
• The Bitter Lesson — his TPU design rule (specialize to the arithmetic, not to the architecture) is the bitter lesson applied to silicon rather than to models

Sources#

• Jeff Dean: The 1% Rule for Building in AI — The 1% Rule for Building in AI, YC Startup School 2026, interviewed by Diana Hu (published 2026-07-30, 57 min, practitioner-opinion). Official YC transcript, not machine-transcribed. COI: Google's Chief Scientist on Google's hardware and models; the TPU efficiency and latency multiples are first-party recollection with no citation