Azalia Mirhoseini

Sources#

• CS329A Self-Improving AI Agents — Part 1: Course Overview
• CS329A Self-Improving AI Agents — Part 2: Test-Time Compute Scaling
• CS329A Self-Improving AI Agents — Part 3: Robust Verification
• CS329A Self-Improving AI Agents — Part 5: Planning and Multi-Step Reasoning
• CS329A Self-Improving AI Agents — Part 9: Future Research Areas

Summary#

Azalia Mirhoseini is an assistant professor in Stanford's CS department and co-instructor of CS329A: Self-Improving AI Agents. Her stated path before Stanford: Google Brain, then Anthropic (on Claude) and Google DeepMind (on Gemini) — she is also a co-author on Anthropic's Constitutional AI paper (Bai et al. 2022), which appears in the bibliography of several sources here.

In this wiki her name arrives twice: as the voice teaching the inference-scaling half of CS329A, and — earlier and more often — as the senior author of the papers other sources cite when they need a citation for repeated sampling.

Large Language Monkeys, and why it keeps getting cited#

Large Language Monkeys: Scaling Inference Compute with Repeated Sampling (Brown, Juravsky, Ehrlich, Clark, Le, Ré & Mirhoseini, arXiv 2407.21787, 2024) is the lab result she teaches as the origin of inference-time scaling: sample a fixed model many times, keep a verifier or selector to pick among the samples, and coverage — the fraction of problems solved by at least one sample — keeps climbing log-linearly out to 10,000 samples. Its consequence, in her own framing in lecture: models "already know a whole lot more than what you get out of them when you just ask them once." The wiki carries the result and its numbers on Latent Capability Overhang.

It is cited in the bibliographies of at least four other sources here (Rethinking the Evaluation of Harness Evolution for Agents, SwarmResearch: Orchestrating Coding Agents for Open-Ended Discovery, Life After Benchmark Saturation: A Case Study of CORE-Bench, Ouroboros: A Self-Developing Frontier Coding Agent with Reviewed Core Evolution), which is the practical reason this page exists: the citation was load-bearing across the corpus before the person had a page. CodeMonkeys (Ehrlich, Brown, Juravsky, Clark, Ré & Mirhoseini, arXiv 2501.14723, 2025) is the software-engineering sequel she references in lecture — have the model generate unit tests for the code it generated and use them as the selector, the verifier closing the loop on itself.

The rest of the lab's line: Archon and AI-as-compiler#

Lecture 2 adds two more of her lab's directions to the Monkeys line above.

Archon (Inference-Time Architecture Search) is the third step: having established that sampling produces coverage the selector can't keep, treat the whole inference pipeline as an architecture and search for it. Its ITAS optimizer composes generate / fuse / critic / rank / verify / unit-test operations across a pool of models under an inference-call budget, and reports open-source stacks beating GPT-4o and Claude 3.5 Sonnet by 14.1% average pass@1. The result she emphasizes is that fusion beats oracle selection — the one thing in her line of work that routes around the verification bottleneck rather than pushing on it. The course TA is a co-author.

AI as a compiler is the direction she names as personally most exciting: generate low-level hardware-aware code (CUDA) from a high-level source (PyTorch), where verification is free because you can compare outputs on arbitrary inputs. KernelBench is the benchmark; coverage scales with samples there as elsewhere. The generalization is any language-to-language translation, where the reference implementation is the verifier — see The Verifiability Thesis and LLM-as-Compiler Knowledge Base.

Weaver — weakly-supervised verifier ensembling — is the lab answer to the bottleneck she names below: not thousands of verifiers but "10 or 20 of them," combined under weak supervision, with the honest caveat that it is "very costly from a compute perspective." Previewed in lecture 2 and delivered in lecture 3, where it is the fourth and final paper. The recipe is score → weight → select over a heterogeneous pool of ORMs, PRMs and LLM judges, with a quality floor that excludes weak members and per-verifier weights fitted on ~1% of labels; the reported result is an open-weight stack matching o3-mini, and a follow-up distills the whole pool into a ~400M scorer keeping ~97% of the accuracy. Full treatment, including the independence assumption the wiki's empirical judge measurements contradict, on Weak-Verifier Ensembling. Same standing COI: she is the senior author of the work she teaches.

The planning line: SPRINT and SWiRL#

Lecture 5 adds two more lab papers, and they are the first in the series that are not about verification at all. Both build their training data the same way — have one model annotate another model's trajectories, then train on the annotation — and both report the same kind of generalization, which she flags explicitly as a cross-project pattern.

SPRINT (Intra-Trace Parallel Planning (SPRINT)) takes DeepSeek-R1 reasoning traces, has GPT-4o segment them into steps and tag each step's plan and execution spans, infers the dependency DAG, repacks independent steps into concurrent rounds, and supervised-fine-tunes a 7B distill on the result so it emits independent plans together. Reported: ~3.5 points of accuracy it was not aiming for, fewer sequential tokens than a 32B model, and transfer from math to countdown and GPQA-diamond.

SWiRL (Offline Multi-Step Tool-Use RL (SWiRL), a COLM 2025 preview at the time of the lecture) trains multi-step tool use with every tool call moved out of the RL loop: generate trajectories offline by iterative prompting, have an LLM judge score each proposed action — the query, never the result — and optimize per-step expected reward against frozen context. Its two portable findings are the process-versus-outcome filtering inversion between RL and SFT, and cross-tool transfer (GSM8K with a calculator lifting HotpotQA with a search engine).

Disclosure pattern. She narrates both entirely in the first person — "when we started the project", "we did not need to train an LLM as a judge for this" — which makes the authorship obvious to anyone listening and leaves it unstated for anyone reading a slide. Same shape as Archon in lecture 2; unlike Weaver in lecture 3, where she said so.

The efficiency line: intelligence per watt (lecture 9)#

Her half of the course's closing lecture (delivered 2025-12-05, co-presented) is the only part of the series that is not about capability, and it is her third distinct research line here after verification and planning. The work is joint with Christopher Ré and John Hennessy at Stanford, and it proposes a metric — intelligence per watt = average task accuracy ÷ average power draw — plus the study behind it: >20 models at ≤20B active parameters, enterprise and consumer accelerators, ~1M chat and reasoning queries, everything open-sourced. Headline as taught: local-model coverage up 3.1× since 2023, an Apple M4 Max at ~1.5× lower intelligence per watt than a B200, and a 5.3× two-year gain in the metric decomposing into 3.1× model × 1.7× hardware. Full treatment, including the forward claims graded against this wiki's 2026 sources, on Inference Efficiency as Capability.

The thesis underneath it is a routing claim: essentially all inference goes to cloud accelerators today, most single-turn queries do not need a frontier model, and consumer memory has grown enough to hold a quantized one — so hybrid local/cloud serving engines that route by query complexity are the direction she names, alongside energy-efficient architectures and kernels, and energy as "the most valuable resource we have going forward."

She also names her lab's inference-systems work — Hydragen and Tokasaurus — as the infrastructure bet: test-time scaling is a throughput and latency workload structurally unlike single-turn chat, so it needs its own serving optimizations, and "these things are going to matter a lot more in the future."

Disclosure, and it is her best in the series. She introduces the metric work as "a recent work that we did, in collaboration with Professor Ré and Professor John Hennessy" and the systems work as "my lab did some of this work" — named authorship, not the pronoun-only pattern of Archon (lecture 2) and SPRINT/SWiRL (lecture 5). The difference tracks how the work is framed: current lab activity gets named, a canonical citation in a survey slot gets a pronoun. The one residual runs the other way — she discusses SWiRL in this lecture as course material ("we learned about SWiRL") without restating that it is hers.

Positions she stated in lecture (late 2025, practitioner-opinion)#

• Verification is the field's bottleneck. She has a lab paper on combining verifiers, and treats "robust verification is hard" as the reason the self-improvement flywheel does not simply run forever.
• Models prefer their own traces. "Even if the traces are coming from a better model, they tend to like their own generated traces more" — offered as an observation without a citation. See the note on Same-Model Review Blindness.
• Temperature has a usable ceiling. Past roughly 1.2 the diversity that makes repeated sampling work turns into gibberish; getting diversity above that needs other tricks.
• Continual learning is the gap she names last. Humans get more skilful as they solve problems; models get an offline batch of trajectories and a fine-tuning run some time later. "Can you learn a new skill? Can you learn how to learn — is an important capability that we basically don't have right now." Her framing of the idealized solution is worth keeping because it prices the alternative: infinite perfect context would be continual learning, and the reason it is not the answer is that reasoning quality over in-context data decays well before the window does, "even a million or a few million." Cartridges (Ré's lab) is the intermediate she points at — put the learning in the KV cache rather than the weights.
• But weights beat memory stores for skill transfer. Asked whether updating weights or updating a memory store is easier in practice, she concedes memory is easier for knowledge — "the simplest and naive version is a database the LLM could learn to look at" — and then draws the line: what you are usually trying to teach is reasoning over a new domain, and "having a side memory system doesn't quite achieve that." Her example is cross-embodiment generalization in robotics, which no amount of retrieval delivers. A rare explicit statement of where the memory-versus-fine-tuning boundary sits, from someone who works on neither.
• Environments are proxies, and self-created ones are the easy half. On a student's reading that Absolute Zero implies agents self-creating environments: simulations were always constructible for games and toy tasks; what makes environments hard now is that they must be reasonable proxies for real-world feedback, and "whether you use agents to create that or software to create that" is the part that does not matter.
• Difficulty-adaptive sampling was unpublished. Asked whether sample count can be made a function of problem difficulty, she and Chowdhery said no published work did it, only reward-model-guided follow-ups — a gap 2026 sources have since filled (Large-Scale Test-Time Compute).

Connections#

• CS329A: Self-Improving AI Agents (Stanford) — co-instructor; the inference-scaling and verification lectures are hers
• Latent Capability Overhang — Large Language Monkeys is that page's 2024 empirical ancestor
• Large-Scale Test-Time Compute — her repeated-sampling line is the parallel-sampling arm of this hub
• The Verifiability Thesis — her stated bottleneck; CodeMonkeys is verifier-first agent design
• Inference-Time Architecture Search — Archon, her lab's third step in the line: search over composed inference architectures, and the fusion-beats-oracle result
• Weak-Verifier Ensembling — Weaver, her lab's fourth: combine imperfect verifiers rather than train a better one, and distill the pool into a tiny scorer
• Process vs Outcome Reward Models — the four-year verifier literature she teaches Weaver as the successor to
• Aakanksha Chowdhery — co-instructor; they met at Google Brain, and lecture 4 is hers
• Alignment Fine-Tuning (AFT) — where Constitutional AI (Bai et al. 2022, on which she is a co-author) is carried; her co-instructor teaches it in lecture 4 without disclosing the connection
• Intra-Trace Parallel Planning (SPRINT) — SPRINT, her lab's fifth entry here: the reasoning trace's hidden DAG, recovered by annotation and trained back into a 7B model
• Offline Multi-Step Tool-Use RL (SWiRL) — SWiRL, her lab's sixth: multi-step tool-use RL with the tools taken out of the loop, and the filtering inversion between RL and SFT
• Tree Search over Agent Trajectories (LATS) — the one paper in lecture 5 that is not hers, and the only one she critiques structurally
• Inference Efficiency as Capability — her third line and the course's closing note: intelligence per watt, the local-inference thesis, and the Hydragen/Tokasaurus bet that test-time scaling needs its own serving stack

Sources#

• CS329A Self-Improving AI Agents — Part 1: Course Overview — CS329A lecture 1 (delivered 2025-09-22, published 2026-08-03, practitioner-opinion): her self-introduction, the Large Language Monkeys walkthrough and results, temperature limits, verifier positions
• CS329A Self-Improving AI Agents — Part 2: Test-Time Compute Scaling — CS329A lecture 2 (delivered 2025-09-26, published 2026-08-03, practitioner-opinion), her solo lecture: the coverage power law and its long-tail condition, the generation–verification gap, Archon, the AI-as-compiler / KernelBench direction, and the verifier-ensembling preview. She is a co-author of every lab result she teaches here — a standing COI
• CS329A Self-Improving AI Agents — Part 5: Planning and Multi-Step Reasoning — CS329A lecture 5 (delivered 2025-10-06, published 2026-08-03, practitioner-opinion), her solo planning lecture: LATS, SPRINT and SWiRL, the last two her own lab's and narrated in the first person without a named disclosure. Also the Claude Sonnet 4.5 system-card aside on prompting for 100+ tool calls, and the position that "basically all LLMs are now inherently reasoning models"
• CS329A Self-Improving AI Agents — Part 3: Robust Verification — CS329A lecture 3 (delivered 2025-09-29, published 2026-08-03, practitioner-opinion), her solo verification lecture: the four-paper arc from OpenAI's GSM8K verifier to Weaver, her positions on PRM gaming as a training-time-only problem, the unsolved generator-vs-verifier budget split, and the closing view that repeated sampling should migrate to training time at the cost of solution diversity. Total COI on the fourth paper — she is its senior author, and says so
• CS329A Self-Improving AI Agents — Part 9: Future Research Areas — CS329A lecture 9 (delivered 2025-12-05, published 2026-08-03, practitioner-opinion with the forward directions at prediction grade), co-presented: her half is the intelligence-per-watt work with Ré and Hennessy — the metric, the demand and local-memory trends, the ≤20B-active study design, the 3.1× / 1.5× / 5.3× findings and the model-versus-hardware decomposition — plus the four forward directions (hybrid local–cloud routing, energy-efficient architectures and kernels, test-time-scaling serving infrastructure via Hydragen and Tokasaurus, energy as the scarce resource) and the closing Q&A on continual learning, infinite context, Cartridges, weights-versus-memory-stores, and environments as proxies. Also the taxonomy of non-verifiable domains (The Verifiability Thesis). COI disclosed by name — the strongest disclosure in the series, and the only lecture where she names a collaborator rather than using the first-person plural alone; one residual, SWiRL discussed as course material without restating her authorship