Research Taste as the Human Bottleneck

Sources#

• When AI builds itself

Summary#

As AI absorbs the execution of AI development (AI Accelerating AI Development), the human role contracts toward a residue the essay When AI builds itself calls research taste and judgment: "choosing which problems matter, which results to trust, and when an approach is a dead end." This is the top rung of the autonomy ladder and the single capability that, if it stays human, keeps recursive self-improvement from closing the loop. Whether it stays human is the essay's load-bearing uncertainty.

The narrowing role#

The essay's clearest statement of the dynamic: "The human role is narrowing at each step in the AI development process." Two concrete narrowings:

• Writing → reviewing. "Once human- and AI-authored code quality reach parity, humans will stop writing code entirely, and shift to only reviewing it." (Parity is reported as roughly now; see AI Accelerating AI Development.) This is the harness-shrinkage story told from the human side.
• Running → choosing experiments. "Once Claude can run experiments, the question shifts towards 'Which of these experiments is worth running?'" The doing — writing the code, running the experiment, producing the result — "now costs almost nothing in human time, even if it still has costs in compute."

The residual human comparative advantage, "for now," is taste: deciding what's worth the compute. Put in the terms of Compute Allocator, the human becomes a compute allocator at the level of an entire research program rather than a single invocation.

Why it might not stay human — "just another capability"#

The essay refuses to treat research taste as a permanent human moat. Two arguments push against it:

1. Perspiration is automatable, and that's most of the work. Genius is "1% inspiration and 99% perspiration"; the 99% — scale it up, see what breaks, fix it, retry — is exactly what Claude excels at. Large-scale research progress "is mostly a function of tools and resources." (See The Bitter Lesson, Recursive Self-Improvement.)
2. Taste shows the same capability curve as everything else. Early evidence of improving research judgment — Claude beating the human next-step choice 51%→64% on hard detour moments (AI Accelerating AI Development) — suggests "research taste might be just another AI capability that AI systems fail at for a time, then get good at." The precedent: AI was once bad at, then good at, explaining why a joke is funny, demonstrating theory of mind, and solving linguistic riddles — qualitative skills assumed to be human-shaped. This is the optimistic face of Jagged Intelligence (Ghosts, Not Animals): today's valley is tomorrow's peak.

The honest counter, in an Anthropic employee's words: "The comparative advantage of humans as of right now is still in seeing the bigger picture and thinking beyond the confines of the immediate task." The question is whether "for right now" is a stable equilibrium or a receding frontier.

The bottleneck either way#

Even if taste stays human, it becomes the binding constraint — the Amdahl's-law bottleneck of the whole pipeline. If humans can't review code as fast as Claude generates it, "human review will become the bottleneck to AI development." And if humans spend most of their time on the single-digit fraction of work that is direction-setting, each human steers vastly more work — so the quality and throughput of human judgment becomes the scarce resource, exactly as Verification as the New Bottleneck predicts for verification and the HBR accountability-redesign work predicts for oversight. The risk is that the human nominally decides but actually rubber-stamps (the "not close to substituting for senior researchers" judgment quietly eroding into a formality).

The human cost (the quieter thread)#

The essay includes unusually candid employee quotes about what the narrowing feels like — worth recording because they name a cost the productivity charts don't:

• The collapse of the gift economy of small favors: asking a colleague "can you help me get this script running?" "created a little debt, a little mutual awareness. [Claude is] faster, it creates zero debt, but each of these is a lost bid for human collaboration."
• The vertigo of contingent relevance: "On days where everything works well, I can't help but think nothing I do matters … But then there are days where everything breaks and I don't understand why and I realize I have no idea what I've been up to anymore."

Connections#

• Recursive Self-Improvement — whether taste stays human decides which of the three futures obtains; the essay's central uncertainty
• AI Accelerating AI Development — the evidence that execution is already absorbed, leaving taste as the residue
• AI R&D Autonomy Evaluation (AECI) — "not close to substituting for senior Research Scientists/Engineers" is the formal version of "taste is still human"
• Jagged Intelligence (Ghosts, Not Animals) — the joke/theory-of-mind precedent for "a capability AI fails at, then masters"; taste as the current valley
• Autonomous Scientific Discovery — Mythos 5's autonomous hypothesis-generation (~80% preferred over Opus-class) and "only high-level human input" genomics are concrete chips at the taste moat
• Verification as the New Bottleneck — if taste/review can't keep pace with generation, judgment becomes the binding constraint
• Harness Shrinkage as Models Improve — the same role-narrowing dynamic from the harness side; what's left after the model-facing harness dissolves
• Compute Allocator — taste exercised as allocation: deciding which experiments are worth the compute
• The Bitter Lesson — "research progress is mostly tools and resources" is the bitter lesson aimed at taste itself
• The Abstraction Barrier — DeepMind's theoretical case that taste/novel-concept-discovery is a real ceiling, not just the next capability to fall: AI may be bounded by human conceptual frameworks
• Transformative Creativity — Boden level-3 (creating new conceptual spaces) is taste exercised at the frontier of concept-space; the "Einstein test" version of the moat
• Multi-Agent Collective Intelligence — steering large superhuman-speed agent groups (whose output humans can't fully consume) is the form taste takes under the collective pathway
• Instrumental Convergence — a knowledge-seeking objective is an alternative framing of judgment as positive-sum information gain rather than scarce human taste
• Artificial Superintelligence (ASI) — taste/judgment is the candidate for what (if anything) stays human as systems cross from AGI into ASI
• Returns to Expertise in Agentic Coding — the labor-data instance of the same "is taste durable?" question: Anthropic frames a decrease in the returns to domain expertise over time as the signal that "models are starting to supply the essential judgment users currently bring" — taste-as-just-another-capability, made empirically trackable in usage
• Planning / Execution Division of Labor — the rubber-stamping risk made concrete: humans nominally own ~70% of planning decisions, but whether that control is real judgment or approval-by-default is exactly the "how do you measure rubber-stamping?" question
• Implementation Abundance Inverts Product Work — the product-work cousin: as implementation cheapens, curation/taste becomes the expensive step, the same residue named here for AI research
• Why AI Lags at Design — design taste as a currently-durable pocket of the human-as-reward-function; a concrete domain where the taste moat holds longer
• Role Averaging, Not Role Elimination — "taste-makers guide from inception" carries the same rubber-stamping risk: guidance is only real if it isn't approval-by-default

Open questions#

• Is research taste a genuine ceiling (an architectural capability scaling can't reach) or the next jagged valley to fill? The essay calls this the decisive unknown.
• If taste is automatable, what — if anything — remains a durable human comparative advantage in AI development?
• How do you measure rubber-stamping? "Humans set direction" can be true on paper while real judgment quietly transfers to the model.

Sources#

• When AI builds itself — §"What might the future of work at Anthropic look like?" and §"What if we're wrong?"