Aakanksha Chowdhery

Sources#

• CS329A Self-Improving AI Agents — Part 1: Course Overview
• CS329A Self-Improving AI Agents — Part 4: Learning from Feedback with Tools and Code
• CS329A Self-Improving AI Agents — Part 6: Train-Time Scaling and Scaling RL
• CS329A Self-Improving AI Agents — Part 7: Self-Improvement and Deep Research Agents
• CS329A Self-Improving AI Agents — Part 8: Agentic Evaluations and Long-Horizon Tasks
• CS329A Self-Improving AI Agents — Part 9: Future Research Areas

Summary#

Aakanksha Chowdhery is an adjunct professor at Stanford and co-instructor of CS329A, and works in research at Reflection AI. She was previously at Google Brain — where she met co-instructor Azalia Mirhoseini — and is lead author of the PaLM paper, a co-author on Pathways, and a co-author on self-consistency (Wang et al. 2022), a citation that recurs in this corpus's bibliographies whenever sampling-then-aggregating is invoked.

She teaches the pre-training/scaling half of CS329A lecture 1 and the whole chatbot→agent half.

The framing that earns her a page: coverage is not capability#

Asked whether reasoning models are really doing anything beyond what a base model could produce, Chowdhery draws the distinction the wiki otherwise states as a benchmarking rule:

"If you had just a base model right after pre-training, it would be able to do some reasoning. But at the same time it will generate different kinds of reasoning chains and it doesn't quite know which one is correct. So a lot of what we will cover in train time or test time scaling really comes down to it learns which is correct. So pass@1 accuracy goes up… as opposed to pass@k or coverage, which is what you were seeing in the repeated sampling results."

That is the mechanism side of Compute-Controlled Benchmarking's reporting rule ("a gain visible only in pass@k is a gain in attempts, not capability"): the training loop is precisely what converts one into the other. Developed on Large-Scale Test-Time Compute.

She is also careful about what this does not settle, and says so under audience pressure: whether the jump comes from RL or from diverse pretraining data has "no single point of consensus," "both processes help," and the loop "is not completely well understood — it's the first signs of life and it starts to get commercialized." An unusually explicit refusal to over-claim from an instructor teaching the material.

The generator–verifier gap#

Her name for the course's bottleneck: models generate cheaply — "a whole bunch of nonsense or sensible set of reasoning traces" — and the binding constraint is the feedback that tells you which. Where verification is mechanical (math, code, rule-based domains) the loop closes; where it is not (her example: creative writing) human feedback becomes the bottleneck, and that is where model improvement stalls. See The Verifiability Thesis and Verification as the New Bottleneck.

On agents (late 2025, practitioner-opinion)#

• Agents differ from chatbots by holding a goal: plan, act on an environment, read feedback, correct, and decide when to stop — or come back and say the goal is unreachable. Tools and memory follow from the goal, not the other way round.
• Real deployments were still hand-built static workflow graphs; open-ended loops were "signs of life" in coding and deep research only (Agent Loop Pattern).
• Clarifying user intent is a first-class agent step, not politeness: "oftentimes the users will not specify the problem well enough," and the agent must fix that before it can know what to verify against (Unknowns as the Agentic Bottleneck).
• Coding agents got reliable through better models plus RL with verifiable rewards, not a new architecture — "the paradigm is very much the same."

Lecture 4: sorting self-improvement by where the feedback comes from#

Her first solo lecture (delivered 2025-10-03) is the course's pivot from how good is the verifier to what is producing the signal at all, and it is organized as three papers with three different answers: the environment (ReAct), the interpreter (RLEF), and the model itself (Constitutional AI, on Alignment Fine-Tuning (AFT)). The taxonomy is hers and it is the useful part — the papers are all recounted rather than measured.

Three positions worth attributing to her:

• Grounding is the workaround for bad calibration, not a solution to it. Pressed on how a model decides when to search, she declines the framing: models asked to rate their own confidence "are not well calibrated… that's a research problem that is still being solved," so an application should route around it by making tool use the default rather than gating it on self-assessment. See Confident But Unsure and Trained Calibration.
• Self-critique is the weak version of AI feedback. Volunteered while summarizing the Constitutional AI follow-on literature: "often times getting the model to critique itself can be harder. So having a consensus of other models to critique the models sometimes works better, because the models might be overconfident." Her co-instructor's stated instinct runs the other way (models "like their own generations"), and this wiki's measurement agrees with Chowdhery.
• The workflow is the domain-specific part, and that is why ReAct does not generalize. Asked whether RL post-training makes the ReAct abstraction obsolete: yes if the search space can be defined, no because "defining the search space for all tasks is not well defined" — an accountant agent, a legal agent and a finance agent each follow a different sequence of steps, and it is the sequence, not the token generation, that resists automation. The same distinction she drew in lecture 1 between an agent and a workflow graph, now stated as the reason the graph is still hand-built.

COI, undisclosed and small but real. The ReAct results she teaches run on PaLM, of which she is lead author; the enumerated-valid-action technique she recommends comes from "another paper that we worked on in collaboration at Google with robotics folks," i.e. her own; and Constitutional AI is her co-instructor's paper. None is flagged in lecture.

Lecture 6: the loop closes, and she bounds it#

Her second solo lecture (delivered 2025-10-10) is the one the course exists for — train-time scaling, defined as "take what the model output, filter after applying this test time scaling, and then use that to fine-tune further" — taught as a three-rung ladder: STaR, then DeepSeekMath/GRPO, then DAPO. The full arc is on CS329A: Self-Improving AI Agents (Stanford); what belongs to her personally is that she is the instructor who states the ceiling.

Her lecture-1 framing above says the training loop converts coverage into pass@1. Lecture 6 supplies the other half of that sentence and it is less flattering to the course's thesis: DeepSeekMath's RL improved majority@K and not pass@K, so "the model actually became more consistent, not fundamentally smarter" — and she generalizes it across all three papers she has just taught: "none of these will yet improve the fundamental capability or teach the model to solve new problems or generalize a lot out of domain," with step changes in capability attributed instead to "a breakthrough or scaling." Developed on Large-Scale Test-Time Compute.

Two more positions worth attributing to her:

• The RL share of a training run, with a vendor claim graded against it. Asked what fraction of frontier training is now RL rather than next-token prediction on scraped text, she gives roughly 1% a year earlier, perhaps ~5% now — explicitly not describing Anthropic, which has not published it — and notes that Grok 4 claimed 50% RL while "it actually did not quite improve" proportionally, because "your rewards are not strong enough or there's noise in rewards." One of the few public numbers on the axis, and she prices its limit in the same breath. See Effective Compute Scaling.
• SFT versus RL, as a decision rule. RL hill-climbs from fewer examples wherever a strong reward signal exists and takes real work to get right; SFT is the faster path when you already have high-quality data, and "doesn't bring reasoning capabilities or boost them."

No COI on this lecture's papers — the first of the six compiled. STaR is another Stanford lab's, DeepSeekMath and DAPO are industry labs'. The only self-reference is oblique and cuts against her: PaLM, of which she is lead author, appears as the model Minerva improved and DeepSeekMath beat.

Lecture 7: search as curation, in two settings#

Her third solo lecture (delivered 2025-10-17) is the homework lecture, and its opening line is the tightest statement anyone in the course gives of what test-time compute leaves unsolved: "we kind of know that the solutions lie in the search space of the models, but how do you curate the answer out of the search space of what the model outputs." Two halves — AlphaCode / AlphaCode 2 (Selection Under a Submission Budget) and the RAG → agentic RAG → Search-o1 ladder (Retrieval Inside the Reasoning Chain) — with the full arc on CS329A: Self-Improving AI Agents (Stanford).

Three positions that belong to her:

• A better base model is a cheaper axis than a bigger sampling budget. "Making the model better is a slightly easier axis than scaling the number of samples" — offered when a student proposes extrapolating a log-linear curve to a trillion samples, and evidenced by AlphaCode 2 matching AlphaCode's solve rate at 100 samples instead of 1,000,000. Note this is the same course that, five lectures earlier, endorsed the opposite tradeoff on easy and medium problems; she is talking about hard ones, and does not say so.
• The scaling law is bounded by diversity, not by compute. "You kind of assume that if we sample more the diversity continues to increase… if you sample 10× more but you don't end up with more diverse solutions, then you're actually not going to improve." The clearest statement in the corpus of why clustering earns its place twice — it selects submissions and it measures whether the extra budget bought anything (Latent Capability Overhang).
• Overconfidence, with the mechanism named. Asked whether output probabilities correlate with correctness, she reports that aggregated log-probabilities are systematically overconfident — a model 50% correct sounding 80% sure (illustrative, not a measurement) — and that it shows up behaviourally as the model "not chang[ing] its mind" under correction. She calls knowing-what-they-know an active research area and offers it as a project. Consistent with her lecture-4 refusal to gate tool use on self-assessed confidence; see Confident But Unsure.

No authorship COI on this lecture's papers either — AlphaCode and AlphaCode 2 are DeepMind's, Search-o1 is Renmin University's. Two undisclosed residuals: she narrates AlphaCode 2 as "this work was at Google" without noting she was at Google Brain, and credits Large Language Monkeys to "Azalia covered this" — her co-instructor's lecture rather than her co-instructor's authorship.

Lecture 8: three rulers for agentic evaluation, and the deflationary one#

Her fourth and final solo lecture (delivered 2025-11-17) surveys three benchmarks — METR's time horizons (Task Time-Horizon Scaling), OpenAI's GDPval, and DeepScholar-Bench (Deep Research Agents) — and its structure is an argument that no single number does the job: duration, economic value, and synthesis quality are three axes and "you still need to validate them against humans who can solve these tasks."

Two positions are hers rather than the papers':

• She uses GDPval to block the extrapolation her own field runs on. "When you see an exponential trend, people often assume that okay so now the now AI can do one hour so next it will be able to do couple of hours" — and GDPval's roughly linear win-rate curve is what she offers against it. Coming from the instructor of a course whose thesis is a self-improvement flywheel, that is a deliberate brake.
• The residual human job is architecting the problem, not doing it. "Humans are basically architecting what is the set of problems and then the model can go solve it" — supported by three independent findings in one lecture: models track low-context contractor times rather than maintainer times, underspecified GDPval prompts leave models unable to work out what to work on, and DeepScholar-Bench systems retrieve relevant papers while missing the foundational ones an expert would know. See Context Advantage, Not Taste.

Asked what her own job looks like in two years, she declines the AGI framing and offers a definition instead: "when AI can build the next generation of models themselves then we don't have to even be in the room" — and says the current state is AI co-scientist, a strong brainstorming partner, not that. Her stated hardest problem is the long tail of reliability, and she attributes it to "a mix of both a data problem and a fundamental model capabilities" problem, with distributed systems named as a domain where models still struggle.

No authorship COI, third lecture running — METR is an independent evaluator, GDPval is OpenAI's, and DeepScholar-Bench is a Stanford/Berkeley group's rather than either instructor's. The only residual is that a Stanford adjunct describes the third paper as "out of Stanford" without stating she is not an author, which cuts in the harmless direction.

Lecture 9: the three bottlenecks she leaves the class with#

The closing lecture (delivered 2025-12-05) is co-presented — her half surveys three 2025 papers, Mirhoseini's covers efficiency — and her selection is organized, unusually for a "future directions" talk, as three named inputs to the self-improvement loop that are still human-supplied or unreliable, one paper each:

Bottleneck	Paper	The move
Reasoning chains are not diverse enough to keep the loop running	Multiagent Finetuning — Rationale Bootstrapping (STaR)	manufacture specialists from one base model
Verifiers are unreliable and can invent their errors	DeepSeek-Math V2 — Process vs Outcome Reward Models	a meta-verifier that grades the verifier's analysis
Training prompts are statically curated by human experts	Absolute Zero — Rationale Bootstrapping (STaR)	the model proposes its own tasks under a learnability reward

Her own summary of the three, and it is the course's closing thesis in one sentence: "to achieve true generalization and reasoning, we really need that notion of diversity in the reasoning chain … we need better verification … and then the third is the selection of data itself." All three are statements about the inputs to the loop rather than about the loop's algorithm, which is a notable place for a course on self-improvement to end.

Two positions belong to her personally:

• Verifiability is a property of a decomposition, not of a domain. Her KernelBench example is the sharpest thing in the Q&A: a compiler and an execution check give you a free verifier for one GPU kernel, but reading a performance profile across a large concatenated program does not — so the working practice is to break the problem into parts the model can handle and hand it a reference knowledge base for the rest. She notes mid-sentence that she is "literally describing a class project right now." See The Verifiability Thesis.
• The scale term, stated twice. Both Absolute Zero's own result and Mirhoseini's SWiRL experience say larger models absorb the synthetic-data flywheel better — which makes self-improvement complementary to scaling rather than a route around it. Neither instructor draws that conclusion out loud.

COI: instructor-authored work returns after three clean lectures — but on the other half of the session. Her three papers are MIT/Google's, DeepSeek's and Tsinghua's respectively, so her half is clean for the fourth consecutive lecture; the conflict sits on Mirhoseini's intelligence-per-watt segment, and is disclosed there by name. Her one residual runs in the harmless direction: she credits KernelBench to "the Stanford stuff … in your [lab]" — attributing it to her co-instructor in the second person, which discloses the connection to the room and not to a reader.

Connections#

• CS329A: Self-Improving AI Agents (Stanford) — co-instructor; the scaling-history and agent lectures are hers
• Process vs Outcome Reward Models — lecture 9's middle paper: DeepSeek-Math V2's meta-verifier, the rung the course's own verification arc had been implying since lecture 3
• GDPval Benchmark — lecture 8's middle third and the wiki's anchor for the benchmark; she is the only source in the corpus who describes it rather than quoting its Elo derivative
• Task Time-Horizon Scaling — lecture 8's first third: the only account in the corpus of how METR's horizon is actually built, and the three limitations she attaches to it
• Context Advantage, Not Taste — the thesis her lecture-8 synthesis lands on, from three benchmarks that were not designed to test it
• Selection Under a Submission Budget — lecture 7's code half, and where her better-model-over-more-samples rule and the diversity ceiling are developed
• Retrieval Inside the Reasoning Chain — lecture 7's retrieval half: the ladder from RAG to Search-o1, and hedging tokens as the knowledge-gap trigger
• Confident But Unsure — she states the calibration failure twice across two lectures, and routes around it both times rather than proposing a fix
• Rationale Bootstrapping (STaR) — lecture 6's first paper, and the wiki's earliest self-improvement loop in which the weights actually move
• Group Relative Policy Optimization (GRPO) — lecture 6's second and third: where GRPO came from, and DAPO's four fixes for long-chain-of-thought RL
• Effective Compute Scaling — where her RL-share estimate and the Grok 4 grading are filed
• Reasoning–Acting Interleaving (ReAct) — her lecture-4 walkthrough is this wiki's definition of the pattern
• RL from Execution Feedback (RLEF) — the coding half of the same lecture
• Same-Model Review Blindness — she states the cross-model-critique preference her co-instructor's self-affinity claim contradicts
• Azalia Mirhoseini — co-instructor; they met at Google Brain
• Compute-Controlled Benchmarking — her coverage-vs-pass@1 distinction is that page's rule stated from the training side
• The Verifiability Thesis — her generator–verifier gap is the same claim in the course's vocabulary
• Large-Scale Test-Time Compute — where the coverage→pass@1 conversion is developed

Sources#

• CS329A Self-Improving AI Agents — Part 1: Course Overview — CS329A lecture 1 (delivered 2025-09-22, published 2026-08-03, practitioner-opinion): scaling-law history, ChatGPT post-training stack, coverage vs pass@1, generator–verifier gap, chatbot→agent transition and workflow patterns
• CS329A Self-Improving AI Agents — Part 4: Learning from Feedback with Tools and Code — CS329A lecture 4 (delivered 2025-10-03, published 2026-08-03, practitioner-opinion), her first solo lecture: the feedback-source taxonomy over ReAct / RLEF / Constitutional AI, the calibration and self-critique positions above, and the closing exchange on why the workflow rather than the token generation is the domain-specific part. Undisclosed COI on PaLM and on the robotics collaboration she cites
• CS329A Self-Improving AI Agents — Part 6: Train-Time Scaling and Scaling RL — CS329A lecture 6 (delivered 2025-10-10, published 2026-08-03, practitioner-opinion), her second solo lecture: the train-time-scaling ladder over STaR / DeepSeekMath / DAPO, the majority@K-not-pass@K bound on all three, the SFT-versus-RL decision rule, the late-2025 RL-share estimate with the Grok 4 comparison, and the closing open-problems list. Slide-read figures in an auto-caption transcript; none of the three papers is in raw/. The first of her lectures with no authorship conflict on the papers taught
• CS329A Self-Improving AI Agents — Part 8: Agentic Evaluations and Long-Horizon Tasks — CS329A lecture 8 (delivered 2025-11-17, published 2026-08-03, practitioner-opinion), her fourth solo lecture: the METR time-horizon methodology and its limitations, GDPval's design and win-rate trend with the explicit linear-versus-exponential contrast, DeepScholar-Bench's three axes and sub-19% ceiling, the cross-benchmark synthesis, and the closing Q&A on the reliability long tail and her own two-year horizon. Slide-read figures in an auto-caption transcript; none of the three papers is in raw/. No authorship conflict — the third such lecture in a row
• CS329A Self-Improving AI Agents — Part 7: Self-Improvement and Deep Research Agents — CS329A lecture 7 (delivered 2025-10-17, published 2026-08-03, practitioner-opinion), her third solo lecture: the curation framing, the AlphaCode / AlphaCode 2 walkthrough with the 10@k-versus-pass@k metric and the selection-bottleneck gap, the better-model-over-more-samples rule and the diversity ceiling on extrapolating log-linear scaling, the RAG → agentic RAG → Search-o1 ladder with the hedging-token trigger and per-document compression, and the closing log-probability calibration discussion. Slide-read figures in an auto-caption transcript; none of the papers is in raw/. No authorship conflict, with two undisclosed residuals (AlphaCode 2 as "this work was at Google"; Large Language Monkeys credited to her co-instructor's lecture rather than her authorship)
• CS329A Self-Improving AI Agents — Part 9: Future Research Areas — CS329A lecture 9 (delivered 2025-12-05, published 2026-08-03, practitioner-opinion), co-presented with Azalia Mirhoseini: her half is the course recap and the three-bottleneck survey over Multiagent Finetuning, DeepSeek-Math V2 and Absolute Zero, plus the closing Q&A on non-verifiable domains and the KernelBench decomposition argument. Slide-read figures in an auto-caption transcript; none of the three papers is in raw/. No authorship conflict on her papers — the fourth in a row; the session's COI sits on her co-instructor's efficiency half and is disclosed there. One residual: KernelBench attributed to her co-instructor in the second person