Returns to Expertise in Agentic Coding

Sources#

• Agentic coding and persistent returns to expertise
• AI and Job Postings: From Destruction to Creation?
• Helping People Choose Careers in the Age of AI
• Voice AI in Firms: A Natural Field Experiment on Automated Job Interviews

Summary#

The headline finding of Anthropic's economic-research report Agentic coding and persistent returns to expertise (Hitzig, Massenkoff, Lyubich, Heller, McCrory, June 2026): what amplifies an AI coding agent is the user's domain expertise, not their coding proficiency. Across ~400,000 Claude Code sessions, the more a person understands the problem they are solving, the more work the agent does per instruction, the more often the session succeeds, and the more readily it recovers from trouble. Coding background, by contrast, barely matters: in code-producing sessions every major occupation lands within seven percentage points of software engineers. The report's one-line thesis — "coding agents are not substituting for domain expertise; the more understanding a worker brings to an agent, the more quality work the agent is able to do" — is the empirical confirmation of "you can outsource your thinking but not your understanding".

Evidence note. empirical — measured from a privacy-preserving (Clio) analysis of ~400,000 interactive sessions from ~235,000 people, Oct 2025–Apr 2026, with classifiers (Claude Sonnet 4.6) validated against automatic telemetry and regressions with controls and confidence intervals. Two honest caveats keep it short of a clean external benchmark: it is first-party — Anthropic measuring its own product, on its own telemetry, with its own classifiers (not independently reproducible) — and it excludes headless (claude -p), SDK, and third-party-IDE usage, a "substantial share" of real activity. Outcomes are transcript-inferred proxies, not observed real-world results.

Expertise is task-specific, not a résumé#

The expertise rating (a five-point novice→expert scale) is not job title or general ability. "A senior engineer asking their first Rust question is a beginner at Rust. An accountant who has never used Python, but tells Claude exactly which reconciliation rules a script must enforce and catches the edge case it mishandles at month-end close, is an expert at that task." The classifier reads three signals:

1. Precision of framing — how specifically the user directs the work.
2. What they ask Claude to verify — experts specify the checks that define "done."
3. Who corrects whom — does the user correct Claude, or does Claude correct the user.

This is exactly the understanding residue made measurable: the signals are the surface evidence of an internal model good enough to direct, judge, and verify.

The amplification: experts get more agent per prompt#

Expertise scales how much autonomous work each human prompt sets off (the action chain):

User expertise	Actions per prompt	Output per prompt
Novice	~5	~600 words
Expert	~12	~3,200 words

More than twice the actions, five times the output. The gap holds within every work mode and every task-value band, and survives a regression controlling for work mode, task value, month, occupation, and model family: +9% actions and +13% output per expertise level (p < 0.001 at each adjacent step). An expert isn't just luckier — each instruction they give safely unlocks a longer leash.

The success gradient#

The more expertise a session exhibits, the more likely it succeeds, on every measure. Success is defined two ways: judged success (a classifier reads the transcript and decides whether the person got what they set out to do) and the stricter verified success (judged-successful and at least one hard external signal — a matching commit/PR, passing tests, or explicit user affirmation).

Expertise	Verified success	At least partial success
Novice	15%	77%
Intermediate–Expert	28–33%	91–92%

The curve is concave: most of the gain is novice→intermediate; intermediate→expert is modest. A working grasp of the domain captures most of the benefit; deep mastery adds only a little. (These are adjusted rates — comparing sessions of the same work mode, value band, month, subject, and occupation type.)

Two corollaries about recovery, which is where expertise earns its keep:

• Recovering from trouble. Among sessions that "hit trouble" (verified failure signals — errors, failed tests, repeated attempts, user frustration), verified success rises from 4% (novice) to 15% (expert); partial success from 60% to 80–81%. Part of the value of expertise is the ability to steer a struggling agent back on course. (Caveat: experts hit trouble less often, so their troubled sessions are on harder problems — estimated task value of a troubled session roughly doubles from novice to expert — so some of the recovery gap reflects novices stuck on routine problems vs. experts stuck on genuinely hard ones.)
• Abandonment. A troubled session is abandoned when it is judged failed and zero lines of code were written. 19% of novice sessions end abandoned, against 5–7% for everyone else. The least experienced give up when stuck.

Occupation matters less than expertise#

The complementary half of the story, and the strongest evidence for software democratization: a coding background is becoming less relevant to coding success. Occupation is inferred (the classifier is explicitly told not to treat the act of coding as evidence of a coding job — a lawyer scripting contract-clause checks is mapped to Legal, not Software).

• Software-related occupations reach verified success ~30% of sessions overall; other professions ~26%. In code-producing sessions, 34% vs 29% — and partial success 89% vs 88%.
• Every one of the ten largest occupations lands within seven points of software engineers on verified success in code-producing sessions; the five-point software/non-software gap has neither widened nor narrowed over seven months.
• Management occupations edge out software engineers on verified success. The report's reading: management skills (delegating, specifying, confirming) transfer to directing an agent — "perhaps acting like a manager confers greater success." (Measurement caveat: verified success partly rests on explicit in-transcript confirmation, and managers may simply say when they got what they asked for.) This is the constructive counterpart to HBR's accountability critique — the skill of bounded delegation helps; the org-chart framing of agents-as-employees is what backfires.

What it means for the labor market#

The report frames itself as an early read on knowledge-work transitions. Two readings, in tension only superficially:

• Substituting for coding skill. Implementation-heavy work that used to require a coding background is being absorbed; "a coding background [is becoming] less relevant to successful programming." This is the floor rising — "a person with command of a domain, in any field, may now be able to do technical work they previously could not."
• Rewarding domain understanding. Simultaneously, the gains accrue to whoever brings the firmer grasp of the problem. "A person without any such expertise will get far less from the same tool." This is the residual human comparative advantage showing up in usage data: not coding, but knowing what to build and being able to verify it.

The report names the metric to watch: if the returns to expertise begin to decrease over time, that signals models are starting to supply the judgment users currently bring — i.e., taste becoming "just another capability". As of this data, the returns are persistent.

The premium, priced in vacancies. Indeed Hiring Lab (Gallacher, July 2026) finds the same shape on the hiring side rather than inside sessions. US software-development postings rose ~15% from Claude Code's February 2025 launch through mid-2026 while overall postings fell 7% — but the rebound is concentrated: 71% of the May 2025 → May 2026 increase came from senior roles and 37% from postings whose title mentions AI (the two overlap). Gallacher's reading is this page's thesis in an employer's words — "demand is growing for experienced professionals who can work with AI, not necessarily a broad-based recovery across all software roles." Two things keep it a corroborating signal rather than proof. It is Indeed analyzing its own job board in a blog post, so the sample is one platform's vacancy flow and the causal story (agentic tooling raised demand for expertise) is asserted from a coincidence of timing, not identified — see Firm AI-Spend Intensity and Headcount Growth for the full evidence note. And "senior role" is a title, not the task-specific expertise this study measures: the classifier here rates a senior engineer asking their first Rust question as a beginner, while a job posting cannot. Seniority-titled demand is the closest labor-market proxy for the expertise premium currently available, and it is a coarse one.

The composition check: experts using AI on their inexpert tasks#

Google ATLAS (July 2026) supplies a cross-lab observation that looks like a contradiction and resolves into a mechanism. Two facts hold simultaneously in its Gemini data:

• The users skew expert. A 1% increase in an occupation's median earnings is associated with >2.5% higher AI usage intensity (1.86 controlling for education); weighting US median earnings by conversation volume moves it from $62,252 to $82,919.
• The tasks skew inexpert. Sorting ~19,000 O*NET tasks into expertise quartiles by the Autor–Thompson word-rarity measure, usage is most over-represented on the lowest-expertise non-routine cognitive tasks — 2.6× baseline for Q1 against a flat 1.6–1.8× for Q2–Q4.

So the modal work interaction is a high-expertise worker pointing AI at the least expert-demanding parts of their job. That is this page's thesis observed from the outside: the human keeps the judgment and offloads what doesn't need it, which is why the expertise premium persists rather than dissolving. Autor & Thompson's model says which way it cuts — automating an occupation's inexpert supporting tasks raises the scarcity of the remaining human expertise (wages up, employment down), while automating its expert tasks erodes entry barriers and depresses wages. ATLAS's snapshot points at the first.

The caveats are real: ATLAS measures consumer surfaces and free API only (no enterprise, no agentic coding at scale), it is a two-week snapshot with no time dimension, and its expertise measure is a lexical proxy — task-statement word rarity — not the behavioral three-signal classifier this study uses. The two are measuring "expertise" on different objects: ATLAS rates the task, Anthropic rates the user's handling of it. See Task Saturation: Broad but Shallow AI Diffusion.

Connections#

• Agent Review Comment Resolution — the same premium on the receiving end of review. Across 341 repos, core developers (top 20% by authored-plus-reviewed PRs, computed per repository) resolve 78.1% of Copilot's agent review comments and 70-80% of every comment category — with their share highest on solution approach, documentation, naming and code organization, and lowest on functional defects (29.5% peripheral). Acting on an agent's design feedback takes project knowledge; fixing a defect it found does not. The mirror finding is the sharper one: rejecting a wrong agent suggestion is the one behaviour that leans peripheral (35 vs 32 cases), so the expertise premium is on knowing what the project intended, not on catching the model out
• Controlled Variance: AI's Edge as Reduced Dispersion — the counter-case, and it cuts two ways. In a randomized hiring experiment, 131 experienced recruiters forecast the effect's direction wrong on the task they were expert in (36% expected the AI arm to receive lower offer rates, 48% lower retention, 61% lower interview quality; the AI arm got 12% more offers). Expertise did not confer foresight about its own automation. But the paper's own reading preserves this page's thesis on the other margin: interviewing automated cleanly while evaluation — the judgment half — stayed human and became the process bottleneck, "redirecting recruiter expertise toward evaluation"
• The Solo-Authorship Rebound — a levelling signature on a different outcome, and not a contradiction. Across 300M+ OpenAlex works the post-2022 switch into solo authorship is largest among the least prolific authors (Δβ +1.16 for 1–5 lifetime works vs +0.21 for 6–20) and only weakly largest among the most senior. That is AI cutting the fixed cost of producing output at all, sitting beside this page's amplification of output per prompt — and neither measures whether the low-output authors' solo papers are any good
• The Tragedy of the Cognitive Commons — the regeneration question this page's cross-section cannot see: expertise measurably amplifies an agent today, and Lovett argues the entry-level work that builds that expertise is what AI removes first
• Task Crossover — the unreconciled tension: expertise is what amplifies an agent here, yet crossover finds a large share of AI work happening outside the user's expertise. Both rest on usage telemetry, and nothing yet says whether borrowed work is done as well
• Implementation Abundance Inverts Product Work — the product-process face of "judgment outlasts cheap execution": as implementation cheapens, curation/taste becomes the expensive step
• Task Saturation: Broad but Shallow AI Diffusion — the cross-lab composition check: ATLAS finds high-earning workers over-using AI on the lowest-expertise cognitive tasks, which is this page's mechanism (offload the inexpert, keep the judgment) seen in Google's usage data rather than Anthropic's
• Role Averaging, Not Role Elimination — the empirical backbone of "specialties don't disappear": domain expertise still decides success even as roles average
• Outsource Your Thinking, Not Your Understanding — this is the empirical proof of Karpathy's thesis: success tracks understanding of the problem, not the ability to type code; the non-delegable residue, now measured
• Printing Press Software Democratization — Cherny's "the best person to write accounting software is a good accountant, because coding is the easy part" is exactly the every-occupation-within-7pp finding; this is the hard data the analogy was waiting for
• Vibe Coding vs. Agentic Engineering — "floor up, ceiling held": occupation-doesn't-matter is the floor rising; expertise-still-decides is the bar that stays
• Research Taste as the Human Bottleneck — the "if returns to expertise decrease, the model is supplying judgment" test is the labor-data version of "is taste a durable moat or the next jagged valley?"
• Planning / Execution Division of Labor — the mechanism of amplification: expert framing safely lengthens the action chain Claude runs per prompt (5→12 actions)
• Agentic Coding Work-Composition Shift — the companion finding from the same study: what the work is and how it shifts over the seven months
• AI Employee Framing — managers' edge here (delegation skill transfers) is the constructive flip side of HBR's warning (employee framing diffuses accountability); skill helps, org-chart symbolism hurts
• Verification as the New Bottleneck — "what they ask Claude to verify" is one of the three expertise signals; the expert is the one who can specify and check, which is exactly the bottleneck role
• Engineer PM Convergence — domain/product understanding as the bottleneck skill, seen in the success data
• Jagged Intelligence (Ghosts, Not Animals) — experts recover from the agent's spiky failures; novices abandon — staying in the loop pays measurable dividends
• Claude Code — the product the entire study measures
• METR — the report cites METR's time-horizon ceiling as the capability frontier this usage sits below
• Conversation-to-Delegation Shift — OpenAI's Codex study cites this report (Hitzig et al. 2026) and reaches the same conclusion from usage data: as work becomes delegation, the binding skill is domain understanding + supervision, not execution
• Organizational Complements to AI — the cited Hitzig et al. argument restated as economics: supervision/verification/coordination and domain expertise are the binding complements that gate AI's value
• Exposure Taxonomy: Observed, Theoretical, Reported, Anticipated — the AEI Cadences survey confirms this from the worker's mouth: 15+-year workers report AI can do ~10pp less of their work, naming judgment, context, and relational work as what AI can't touch — tacit expertise as the residual. That page also carries the augmentation-pays finding: ranking 872 occupations by share of augmented vs. automated Claude queries, jobs in the ninth augmentation decile out-earn jobs in the ninth automation decile by about $7,000/yr (Steele & Cruz 2026). The correlational, 2022-salary, work-and-personal-use-mixed version of this page's thesis — pay tracks the work that is hard to hand over
• The Automation–Optimism Link — the mirror gradient: experienced workers are more skeptical of AI's reach, yet heavy delegators are the most optimistic — expertise and enthusiasm pull in opposite directions
• Experimental Learning Impact of Generative AI — the learning-task echo: AI's gains skew to higher-ability students and to augmentation use (AI deepening understanding), the same "benefit accrues to whoever brings/builds understanding" pattern this study measures in agentic coding
• Conversation Artifacts — "the human stays involved in high-value work" (more turns and more Claude output as wages rise) is the augmentation reading of expertise amplifying the agent
• AI Usage Cadences — off-hours Claude work skewing to higher-wage occupations is consistent with expertise-heavy work being where AI lands first
• Anthropic Economic Index — the research program this study belongs to; Cadences is its next report
• Context Advantage, Not Taste — the instrument. If the human's role is a closable information asymmetry rather than a faculty, the measured decline in the expertise premium is that gap closing, tracked in usage data; the two framings disagree about what the decline means
• Unknowns as the Agentic Bottleneck — the mechanism behind the expertise premium: Thariq Shihipar observes that "the best agentic coders have relatively few unknowns" — the expert's map already matches the territory, and they assume unknowns rather than believing the map complete
• Review as the Control Point — the same claim on the review side: reviewer expertise + disposition is the first of the three moderators that decide whether a coding agent helps or harms software. Expertise amplifies (and protects) at the review keyboard as it does at the authoring one — the CMU theory is the mechanism story for why this study's expertise premium should persist
• Market-Priced AI Exposure (the AI Premium) — the asset-pricing echo: the AI premium loads on the intensive margin (paid/core and seasoned users, long prompts price AI risk; casual/new use does not), and the market-implied skill map rewards interactive/relational/communication work and penalizes analytical/scientific — the market's version of this page's "depth of understanding amplifies the agent" and the survey's "experienced workers name relational judgment as the residual AI can't touch"
• AI-Native Organization — the practitioner-side amplification claim to weigh against this data: Tan's self-measured ~400x (self-deflated to 8x–80x) vs. the measured 2× actions / 5× output premium; his "2x people and 100x people use the exact same Claude" attributes the spread to wiring, not expertise — a complements story this study doesn't test
• Firm AI-Spend Intensity and Headcount Growth — the two labor-demand instruments that bracket this page's premium, and they disagree on composition: Indeed's postings rebound is 71% senior (the expertise premium priced in vacancies), while Ramp's spend-linked firm panel finds entry-level headcount growing fastest (+12.0%) at intensive adopters. Stock vs. flow, adopting firms vs. one job board — reconcilable, unreconciled
• Owning Your Externalized Cognition — the premium this page measures, proposed as portable property: if domain judgment is what amplifies an agent, writing it down as skill files turns it into a transferable asset. This page's concave curve complicates that — gains concentrate novice→intermediate and mastery adds little, so the library of a true expert may be worth less as career capital than the doctrine assumes
• AI and Market Power — the firm-level analogue of this page's individual premium, with the same double edge: OECD find the benefits of GenAI "unlocked more easily by firms that have higher productivity and capabilities," measured as a monotone gradient across GenAI-exposure quintiles in productivity, markups and tertiary-educated workforce share (0.04 → 0.46). Where this page finds expertise amplifying an agent within a session, that one finds firm capability amplifying adoption across a whole economy — and draws the same worry, that the capable gain most and the gaps widen
• Systems Thinking Over Specialization — the hiring-side reading of both halves: Elizabeth Stone's "great engineering is scarce" is the persistent premium, and her "specialists can learn that quickly now" bets on exactly this study's concave curve (working grasp is cheap to acquire; mastery isn't)

Open Questions#

• The forward test the report itself names: do the returns to expertise persist, narrow, or invert as models improve? A decrease would mean models are absorbing the judgment users currently supply.
• Outcomes are transcript-inferred (verified success leans on git activity + explicit affirmation). How much of the management edge — and the whole success gradient — is real outcome vs. who-narrates-success-in-the-transcript?
• The study excludes headless / SDK / IDE usage (a "substantial share"). Does the returns-to-expertise pattern hold in non-interactive and pipeline use, where there is no human steering mid-session at all?
• Is "intermediate captures most of the benefit" stable, or an artifact of current model capability — i.e., will the concave curve flatten further (everyone converges) or steepen (mastery starts to separate again) as models get better?

Derived#

• Learning to Co-Work with AI: A Software Engineer's Field Guide — the field guide's "domain depth + taste" skill thesis is what this study measures directly
• Is Breadth Cheap Now? Specialist Ramp Speed and Domain-Expert-as-Builder at Scale — reads this study's concave curve, within-7pp parity, and management edge as the case that performance breadth is cheap while retained-capability breadth remains unproven

Sources#

• Agentic coding and persistent returns to expertise — Anthropic Economic Research, June 2026; §"The returns to expertise", §"Occupation may matter less than expertise", §"Looking ahead"
• Voice AI in Firms: A Natural Field Experiment on Automated Job Interviews — Jabarian & Henkel, Voice AI in Firms (arXiv 2607.28222, 2026-07-30; empirical, pre-registered RCT): §3 intro (the recruiter forecasts), fn. 5 and §8 (recruiter expertise redirected toward evaluation), §7.1 (the evaluation stage as the new bottleneck). Parse warnings and full treatment at Controlled Variance: AI's Edge as Reduced Dispersion.
• AI and Job Postings: From Destruction to Creation? — Guillermo Gallacher, AI and Job Postings: From Destruction to Creation? (Indeed Hiring Lab, 2026-07-08; empirical administrative postings data, but a blog post by the platform whose data it is). Key points list and §"A senior, AI-fluent rebound" — the 15% / 7% / 71% / 37% figures, all stated in prose as well as in chart captions. The international comparison's country identities and both scatter plots' correlation statistics are chart-image-only and are cited nowhere in this vault. Full evidence note and the contradicting entry-level result at Firm AI-Spend Intensity and Headcount Growth.
• Helping People Choose Careers in the Age of AI — Steele & Cruz, arXiv 2607.15506 (2026-07-16), empirical; §5.6 + Fig. 17 (mean salary by augmented vs. automated Claude usage decile). The $7,000 ninth-decile gap is stated in prose; the authors attach two caveats (2022 salary data, and the September 2025 AEI release cannot separate work from personal use). Parse warnings for this source are recorded on Exposure Taxonomy: Observed, Theoretical, Reported, Anticipated.