Efficiency Debt of AI-Generated Code

Sources#

Characterizing the Quality Profile of AI-Generated C++ in Production

Summary#

Tran, Lewis, Yang, Thakur, Kini, Patil, Hashemi & Ranganathan (Google, arXiv 2608.06640, August 2026) run the corpus's first production-scale, human-controlled measurement of AI-authored code quality: 3.52 million submitted changes over twelve months (2025-04-01 to 2026-04-01) in one enterprise C++ monorepo, with authoring-time byte-level provenance and a human-written cohort as the comparison.

The finding this page is named for is the one nothing else in the vault hosts. AI-generated C++ carries a source-level imperative bias — it does the work locally instead of delegating it — and that bias is expensive after deployment. The measured chain runs end to end in one dataset:

Upstream (submitted code): roughly 2.0x the loop constructs, 30-40% fewer standard-library/API calls, ~2.0x the map-access and container-insertion warnings, 1.39x the missing-move warnings.
Downstream (production profiles): AI-heavy functions shift +3.46% of total CPU cycles toward direct on-CPU execution and -1.08% away from calls into shared optimized libraries (absl/gtl).
Cost: median normalized compute for AI-heavy functions grows to 1.31x its baseline against 1.25x for human functions (a ~5% relative increase), and memory to 1.36x against 1.25x (~8%).

The mechanism matters more than the percentages. IPC and MIPS show no notable degradation — the code is not executing slowly, it is executing more. This is a work-volume tax, not a compilation or scheduling problem, which is why it is invisible to every instrument that watches throughput rather than allocation.

Evidence note. empirical, and the measurement is exceptional in scale and instrumentation — authoring-time provenance rather than post-hoc attribution, a genuine human cohort, and outcomes joined across review, CI, sanitizers, reverts and production profiling. Four limits travel with every number. (1) COI is structural: Google engineers measuring the output of Google's AI coding tools inside Google's monorepo, publishing a result that is on balance reassuring, with a discussion that attributes the weaknesses to "historical default system configurations" rather than to the models. The paper anonymizes the company in-text for double-anonymous review while the byline names it. (2) Observational, explicitly not causal — the authors say so: AI-generation share can correlate with task difficulty, repo context, author experience or review norms, and the stratified cohort comparisons "do not convert the study into a causal estimate." (3) Model mix is masked. Individual model identifiers were aggregated away at collection, so the paper cannot say whether the issues are uniform or driven by early, weaker models — and developer prompting proficiency grew across the same window, so early commits confound model quality with human steering quality. (4) One language, one monorepo, review-gated, with mature static analysis and function-level compute observability. The authors state the transfer boundary themselves.

The upstream profile: where the excess actually sits#

The static taxonomy classifies line-level findings into five quality attributes. The AI/human rate ratios are not uniform, and two of the five run below parity:

Quality attribute	AI/human rate ratio
Efficiency and Resource Use	1.23
Maintainability and Readability	1.08
Modernity and API Evolution	1.04
Correctness and Safety	0.94
Policy, Portability, and Environment Fit	0.92

Two categories — Interface and Coupling Burden and Copy and Allocation Overhead — account for 82.21% of the total positive absolute rate gap. Everything else is near parity or negative. And the concrete mechanisms are narrow enough to target: misc-include-cleaner plus misc-definitions-in-headers are 98.95% of Interface and Coupling Burden, and runtime-missing-move is 51.60% of Copy and Allocation Overhead.

Read the composition column separately from the rate column, because the paper reports both and they answer different questions. By rate contrast, the excess is in efficiency and coupling. By share of what a reviewer actually encounters, Interface and Coupling Burden is 43.96% of AI weighted findings against 40.36% of human, and API Misuse and Invalid Calls is 29.49% against 30.05% — the two together are 73.45% of AI findings and 70.41% of human. A category can be disproportionate without dominating review burden, and vice versa.

The change-structure control. AI changes are bigger but their functions are smaller: median 89 lines changed [IQR 27-230] against 33 [8-122] for human changes (Cliff's delta 0.284), 3 files touched against 2, new-code ratio 0.83 against 0.60 — while median function LOC runs 11 [5-26] for AI against 15 [7-31] for human (delta -0.118) at identical median complexity. Larger, newer changes made of shorter functions. The downstream comparisons stratify on change size among other covariates, so the outcome ratios below are not simply the size effect. (Table 2 reconciled line-by-line against page 9 of the PDF; Table 4 against page 10.)

The reliability result splits, and the paper partly obscures it#

This is the finding that complicates the vault's existing narrative, so it is worth stating precisely rather than in the paper's own summary language.

Build failure rate: AI/human ratio stays above parity all twelve months, median ~1.3x (Figure 4's box is tight, roughly 1.17-1.35 — a stable effect, not a noisy one).
Sanitizer findings: median ~1.3x, but with a much wider monthly spread (roughly 1.07-1.84).
Revert rate: median ~0.9x, below parity — AI-generated code is reverted less often than human-written code. The monthly box does cross 1.0 at its top whisker (~1.07), so this is a median claim, not a universal one.
Review friction, all above parity: blocking threads 1.92x, total comments 1.39x, reviewer iterations 1.24x, time to merge 1.19x, submit attempts 1.07x.

The paper's own summary understates its own measurement. Section 4.3 reports the build-failure ratio at "roughly 1.3x"; the RQ3 summary two paragraphs later calls it a "comparable build failure rate" while calling the 0.9x revert ratio "superior." By the paper's own yardstick a 0.3 deviation from parity is a finding in one direction and a rounding error in the other. Cite the measurement, not the gloss.

The defensible reading is the one the discussion actually earns: the burden is not acute breakage. Existing automated gates and human reviewers catch fatal errors before or during deployment — that is what the extra build failures and the lower revert rate jointly say — and what survives is chronic: coupling burden, allocation overhead, and compute footprint.

The null result that relocates the fix#

The authors went looking for the reviewer-fatigue story and did not find it. They tested whether standard review metrics — prolonged review time, higher iteration counts — correlate with the survival of inefficient AI-generated C++, hypothesizing reviewer fatigue or over-trust. No clear correlation.

Their conclusion is the load-bearing one for this vault: conventional proxies for human effort "do not adequately capture the cognitive friction of evaluating AI-generated code," and because "human reviewers struggle to consistently intercept these localized inefficiencies regardless of review depth," upstream automated intervention is necessary, not merely cheaper.

Two things follow. First, this is a production-scale null against the simplest reading of Review as the Control Point's P1 (load lowers review depth lowers quality) — for this outcome class the control point isn't review at all, because a missing std::move in a correct function is not a thing more attention reliably finds. Second, it is the strongest non-vendor evidence in the corpus for Faros's "authoring problem, not a review problem" thesis, arrived at from the opposite direction: Faros argues review shouldn't be the fix, this measures that for one defect class it can't be.

RQ4: the mitigation works, and it does not close the gap#

The intervention takes the RQ2 profile and feeds it back to the generator. Fifty C++ functions each carrying at least one target-category finding are stripped to their signatures and reimplemented under three prompt stages (base / general feedback / category-specific feedback), three independent runs each, 450 implementations, scored on targeted static findings and on R_eff — a benchmark score where 1 = improvement, 0.5 = parity, 0 = regression against the original implementation, with instruction count weighted 2x memory.

Stage	R_eff	Targeted findings	vs baseline
Baseline (original human code)	—	1.28	—
Stage 1 (base prompt)	0.294 (±0.041)	1.26	-1.56%
Stage 2 (general feedback)	0.342 (±0.041)	1.22	-4.69%
Stage 3 (category-specific)	0.385 (±0.043)	1.12	-12.50%

The headline 11.1% reduction is Stage 3 against Stage 1; the table's own Reduction column is against baseline (12.50%). Both are correct and they are different denominators. All six figures are arithmetically self-consistent and restated in prose.

The find the paper does not state: even the best-mitigated output is, on average, a regression. Mean R_eff at Stage 3 is 0.385 on a scale where 0.5 is parity with the original human implementation. Taxonomy-informed feedback moves regenerated functions from clearly regressive (0.294) to still net-regressive (0.385) — a real 31% relative gain that never reaches break-even. The discussion nonetheless concludes that "the models' tendency toward an imperative loop bias vanishes when the system design provides adequate architectural guidance." An 11.1% reduction in findings and a sub-parity efficiency score do not support "vanishes." The measurement supports mitigable; the framing claims solved.

Why this is a fourth debt axis, not a restatement of the others#

The vault already holds three characterizations of what agent-authored code leaves behind. This is the fourth, and it is the only one whose cost is denominated in money after deployment rather than in risk or in engineer-hours:

Axis	Where the debt sits	How it surfaces
Agentic Technical Debt	Architecture — intent re-derived each session	A forced rewrite
Security Debt of Agent-Generated Code	CI/container plumbing (87.6% of smells)	A credential exploited
Agent-Generated Test Quality	Test environment and the gap between test and diff	CI reliability decays; regressions ship unexercised
Efficiency debt (this page)	Function bodies — local work over library delegation	A compute and memory bill that grows ~5-8% faster

They converge on one characterization: the agent handles the code under consideration competently and mishandles the environment it runs in — the CI sandbox, the container, the surrounding architecture, and now the machine. The paper's own hypothesis for the cause is the same one the other three reach: "generalist LLMs lacking the highly specific context of internal enterprise monorepo structures," with the proposed fix being a knowledge base that injects relevant context into the prompt — which is the CLAUDE.md remedy and Faros's "context engine" arrived at independently, from profiling data.

Adoption, for the record#

RQ1 is not this page's subject but supplies the denominator, and it is the corpus's largest non-vendor adoption series. AI-generated share of submitted code across major languages rose 28.99% (April 2025) to 68.62% (March 2026); C++ majority-AI changes went 27.65% to 59.69% while majority-human fell 72.10% to 40.04%; the overall C++ monthly share ran 28.56% to 62.80%. Adoption is uneven by org — Machine Learning & AI reaches 70.19%, Consumer Products/Apps/Devices 67.45%, with some slices plateauing after initial uptake. Full treatment at AI as Primary Author.

Connections#

Acceleration Whiplash — the contradiction and the corroboration in one source. It corroborates the review-friction half with non-vendor production telemetry (blocking threads 1.92x) but at far smaller magnitude (time to merge 1.19x against Faros's +441.5% median time-in-review), and it contradicts the production half: Faros measures incidents/PR +243% while this measures revert rate ~0.9x, below parity. See the contradiction section on that page
Security Debt of Agent-Generated Code — the human control cohort that page's first open question asks for, on a different path set: Correctness and Safety findings run 0.94x in AI-generated C++, which is direct evidence against a general "AI code is less safe" prior and against the uncorroborated 2.7x vulnerability figure — though it measures clang-tidy categories in application C++, not security smells in CI/IaC, so it counterweights rather than settles
Agent-Generated Test Quality — the sibling axis with the opposite instrument problem: that study has a human cohort broken by a parse gap, this one has a human cohort at production scale but no ability to separate model generations. Both land on debt in the environment rather than the logic. Its coverage cut also bears directly on this page's second open question — whether the sub-parity revert rate is a property of the code or of the gates around it. Outside a review-gated monorepo the most-assumed gate is the test suite, and on agentic PRs in open source it executes 27.0% of changed lines in Python and none at all in 64.8% of them, which is a concrete reason to expect the reliability picture to look worse where presubmit gates are weaker
Agentic Technical Debt — efficiency debt is this page's compute-denominated register of the same missing-context mechanism, and the paper's proposed fix (a knowledge base injecting monorepo context at prompt time) is the CLAUDE.md remedy reached from profiling data
AI as Primary Author — the non-vendor adoption series behind the authorship shift, and the interaction-mode composition that Faros's PR-level framing cannot see
Review as the Control Point — a production null against the simplest P1 reading: review time and iteration counts do not predict which inefficiencies survive, so for this defect class the control point is upstream of review entirely
Telemetry vs. Survey Measurement — a new instrument shape: first-party engineering telemetry with a control cohort, which is what Faros's cross-customer adoption-depth comparison lacks — and with the vendor incentive pointing the opposite way, since the org measuring the code is the org that built the tools that wrote it. That page now also carries the live counterexample this study is: DX (vendor-claim) announced in July 2026 that with adoption above 90% "comparing AI users against a non-user control group is no longer a viable measurement strategy" — while this human-controlled cohort was running inside a population where adoption is effectively total. What saturated is who uses AI; what did not is what wrote this change, and roughly three in ten submissions here were still human-authored at the end of the window. The distinguishing ingredient is authoring-time provenance, an instrumentation decision taken before the code exists, not an analysis choice available afterward
Agent Review Comment Resolution — the same missing-context diagnosis, one loop over. This paper blames AI's imperative bias on "generalist LLMs lacking the highly specific context of internal enterprise monorepo structures" and prescribes injecting repository context at prompt time; that study finds the dominant reason agent review comments get rejected is the identical gap — an agent flagging as a defect what the team decided on purpose (23.8% of 470 card-sorted argued discussions, against 4 outright hallucinations). One context deficit, symptoms on both the authoring and the reviewing side, one prescription. It is also the evidence-class counterweight: this page has a human control cohort and that study has none, so "is an agent reviewer worse than a human reviewer" stays unmeasured
Same-Model Review Blindness — two reasons the review dial does not work, with opposite prescriptions. This page's null says review time and iteration count do not predict which AI inefficiencies survive, and concludes the class is invisible to a reader at any depth, so the fix moves upstream to authoring. Greptile (case-study) finds the same review pass is worth 6–12 points more high-severity recall when a model from a different family runs it, so the fix is lateral — change who looks, not how hard or how early. The two also give competing accounts of the same asymmetry: Tran et al. attribute AI's authoring defects to generalist models lacking monorepo-specific context and prescribe context injection, Caridad attributes the review gap to each model's own design instincts and prescribes cross-vendor routing. Only the first has a control cohort, and the two prescriptions compose rather than conflict — nothing about routing the reviewer helps with a defect class no reviewer can see
Verification as the New Bottleneck — the class of defect the bottleneck cannot absorb: not a matter of insufficient review capacity but of review being the wrong instrument, since attention does not find a missing move constructor
Post-Acceptance Edit Behavior — the filter this paper names and cannot measure. RQ1's own caveat is that developers "substantially filter generated text before it reaches submitted-code analysis," which is why 68.62% is a share of what survived a human. DECODE instruments that survival step directly: 53.6K in-IDE edits of accepted completions, bimodal retention, a median 63% of the completion surviving, and 31% of trajectories carrying a removal edit. It bounds the caveat in the direction the caveat implies without closing it — different population (opt-in extension users, not one monorepo), different granularity (a 9-line completion, not a submitted change), older completion models. The mirror-image methodological note is the useful one: this paper cannot separate model generations at all, and over there 20 models separate the outcome by eta-squared 0.002 to 0.007, which is weak evidence that the masking costs less than it looks

Open Questions#

Does the imperative-bias and library-avoidance pattern generalize beyond C++? The paper's own closing question names Rust, Go and Java, and the answer decides whether this is a property of generated code or of C++'s particular idiom space (where the "right" call is a specific absl/std API a generalist model has weak priors on).
Is the sub-parity revert rate a property of AI-generated code or of the gates around it? Every reliability figure here comes from a review-gated monorepo with mature static analysis and presubmit CI, and the paper's own reading is that the gates catch the fatal errors. The discriminator is the same measurement in an org with weaker presubmit gates — where Faros's incident numbers come from.
Stage 3's mean R_eff of 0.385 sits below the 0.5 parity mark, so the best-mitigated regeneration is still on average worse than the human original it replaced. Does any feedback regime — the RLEF pipeline the authors propose, or context injection from the monorepo — push mean R_eff above 0.5, or is sub-parity efficiency a floor for generalist models on performance-sensitive code?

Sources#

Characterizing the Quality Profile of AI-Generated C++ in Production — Tran, Lewis, Yang, Thakur, Kini, Patil, Hashemi & Ranganathan (Google, arXiv 2608.06640, 2026-08-06), empirical. §3 (dataset scope, provenance and projection, static taxonomy, metrics), §4.1 + Figure 3 (adoption trends and interaction-mode mix), §4.2 + Tables 2-4 (change structure, static issue profile, source-level efficiency measures), §4.3 + Figure 4 + Table 5 (review, reliability and compute outcomes, one-year CPU shift), §4.4 + Table 6 (the taxonomy-informed feedback intervention), §5 (discussion, the IPC/MIPS observation), §6 (threats to validity, including the human-in-the-loop null). Parse status: docling verify: ok, no collapse or shift flags. Tables 2 and 4 were nonetheless reconciled against the local PDF (pdftotext -f 9/-f 10 -layout) and match cell-for-cell; Table 5 and Table 6 are arithmetically self-consistent and fully restated in prose. One internal discrepancy, in the source rather than the parse: Table 4 gives standard-library use as ~0.4x, while the prose states "30% to 40% less often" (~0.6-0.7x) in both §4.2 and §7. This page quotes the prose figure as the conservative one and does not cite the 0.4x cell. Figures 3 and 4 were viewed directly under the image two-pass rule; the interaction-mode shares in this page and on AI as Primary Author are read off Figure 3's stacked-area panel and are approximate (+/- ~3pp) — no numeric mode shares appear anywhere in the prose