Loop Engineering

Sources#

• Boris Cherny: We Cut 80% of Claude Code's Prompt
• Driving the Agent Quality Flywheel from Your Coding Agent- Google Developers Blog
• Loop Engineering
• Thread by @AndrewYNg
• Verify, Repair, Repeat, or Stop? Robust Stopping for Noisy Verify-Repair Loops in LLM Agents

Summary#

Loop engineering is replacing yourself as the person who prompts the agent — you design the system that does it instead. Karpathy-era agentic coding was a human holding the tool one turn at a time: type, read, type again. Loop engineering inverts that: you build a small system that finds the work, hands it out, checks it, records what's done, and decides the next thing — then let that system poke the agents. Addy Osmani's June 2026 essay gave the practice its name and an anatomy: five primitives plus one place to remember. Its sharpest, most surprising claim is that this is "not really a tool thing anymore" — a year ago a loop was a private pile of bash you maintained forever; now the pieces ship inside the products, and the same loop works in the Codex app or in Claude Code because the primitives are the same primitives.

The thesis: stop prompting, design the loop#

The essay is built on two quotes the field converged on independently:

• Peter Steinberger (Peter Steinberger): "You shouldn't be prompting coding agents anymore. You should be designing loops that prompt your agents."
• Boris Cherny (head of Claude Code): "I don't prompt Claude anymore. I have loops running that prompt Claude and figuring out what to do. My job is to write loops."

A loop, in this framing, is a recursive goal: you define a purpose and the agent iterates until it's complete. At the center of every loop is the same four-step cycle — act, observe, reason, repeat — the agent does something, reads what came back, decides what that means against the goal, and decides whether to go again.

Loop engineering sits one floor above the harness (Agent Harness Engineering): a harness is the environment a single agent runs inside; a loop is that harness on a timer, spawning helpers, and feeding itself. Osmani is openly skeptical — "it's still early" — and stresses the cost caveat: token usage varies wildly depending on whether you're "token rich or poor."

The five primitives, plus memory#

A loop needs five things and then one place to remember stuff. Each maps onto a primitive both the Codex app and Claude Code now ship:

1. Automations — scheduled discovery + triage that run by themselves. The heartbeat that makes a loop a loop and not a one-off run. → Agent Loop Pattern
2. Worktrees — isolated parallel checkouts so two agents don't collide on the same file (the agentic version of two engineers committing to the same lines). → Agent Harness Engineering
3. Skills — SKILL.md files that codify the project knowledge the agent would otherwise guess; the format is the same in both tools, and the matching description is what triggers implicit invocation. → Agent Context Files
4. Plugins / connectors — MCP-based integration that lets the loop touch your real tools (issue tracker, database, staging API, Slack) instead of only the filesystem. → MCP and Computer Use
5. Sub-agents — one agent has the idea, a different one checks it; the maker is too generous grading its own homework. → Verification as the New Bottleneck

Then the sixth thing, memory: a markdown file, a Linear board — anything that lives outside the single conversation and holds what's done and what's next. "The agent forgets, the repo doesn't." This is the same on-disk-not-in-context trick every long-running agent depends on (see Agent Harness Engineering's repository-as-system-of-record and Agent Context Files's state-vs-policy distinction; Ticket-Driven Agent Orchestration is the Linear-board form).

The sixth primitive is the one with a controlled measurement behind it. Knowledge-Centric Self-Improvement runs a loop whose agents are deliberately disposable — fresh context every attempt, nothing inherited but the state file — and shows the external memory doing the improving on its own: generic agents plus a curated store beat DGM, HyperAgents, GEPA and OpenEvolve on five benchmarks at lower dollar cost, and the frozen store keeps working after the model family that wrote it is swapped out. Two design notes transfer directly to a MEMORY.md-shaped state file: write scope conditions, not conclusions (each entry carries when it applies and when it does not), and bound what gets delivered — their transfer adapter caps every field at 0-3 items and returns empty lists when the prior is weakly relevant, because handing over a fixed quantity made memory "noisy or detrimental." A state file that only grows is the failure mode.

Tool-agnostic: Codex app ≈ Claude Code#

The essay's central structural observation is that both products now have all five primitives, with different names for the same capability:

Primitive	Job in the loop	Codex app	Claude Code
Automations	discovery + triage on a schedule	Automations tab → Triage inbox; /goal for run-until-done	scheduled tasks / cron, /loop, /goal, hooks, GitHub Actions
Worktrees	isolate parallel features	worktree per thread	git worktree, --worktree, isolation: worktree on a subagent
Skills	codify project knowledge	Agent Skills (SKILL.md), $name or implicit	Agent Skills (SKILL.md)
Connectors	connect your tools	Connectors (MCP) + plugins	MCP servers + plugins
Sub-agents	ideate + verify	TOML agents in .codex/agents/	subagents in .claude/agents/, agent teams
State	track what's done	markdown or Linear connector	markdown (AGENTS.md, progress files) or Linear MCP

The conclusion: "once you notice the shape is the same you stop arguing about which tool — you just design a loop that works no matter which one you're sitting in." This is the deployment-side evidence for Harness Shrinkage as Models Improve: capability that used to live in a hand-maintained bash harness is being absorbed into the products as named primitives. One clean detail Osmani flags: a skill is the authoring format, a plugin is how you ship it — bundle skills + connectors as a plugin to share across repos.

Two weeks after Osmani's essay, Google supplied the strongest confirmation yet that the loop is tool-agnostic: its Agent Quality Flywheel ships an entire eval-fix loop (synthesize scenarios → grade → analyze → propose fix → compare baselines) as an installable skill driven by whatever coding agent you already use — a vendor packaging a pre-designed loop for other vendors' agents to run, with the maker/checker split built in as Optimizer–Evaluator Decoupling.

/goal: the maker–checker split applied to "done"#

The in-session primitive closest to the whole idea: /loop re-runs on a cadence, but /goal keeps going until a condition you wrote is actually true — and after every turn a separate small model checks whether you're done, so the agent that wrote the code isn't the one grading it. You give it "all tests in test/auth pass and lint is clean" and walk away. Codex ships the same /goal (verifiable stop condition, pause/resume/clear). This is the maker/checker separation applied to the stop condition itself — the reason you can trust the loop to halt unattended.

When the stop-checker is noisy, running the loop is worse than not running it#

/goal's separate stop-checker answers who decides you're done. It does not answer what to do when that decider is unreliable — and Wu et al. (2026) (empirical) measure what that costs. In a verify-repair loop where both the checker and the repairer are noisy, a fixed five-round repair budget ends at 0.116 true validity where committing the first draft ends at 0.700, and it degrades monotonically with the budget (0.246 / 0.122 / 0.116 at K = 1/3/5). Reported acceptance keeps rising the whole time, because a verifier's pass rate is ρ₀ + J·Q — mostly its own false-accept rate once discrimination J is low.

Three things transfer to loop design directly:

• A round cap is the failure mode, not the safety net. "Repair up to K times" is the arm that collapses; every acceptance-threshold heuristic (majority vote, confidence threshold) merely approaches never-repairing and never beats it. The only arm that wins compares the expected benefit of one more round against the expected damage, per instance.
• The stop boundary is a property of the repairer, not the checker. It sits at b* = α/(α+β) — repair-fixes over repair-fixes-plus-repair-breaks — and the measured value ranges from 0.954 to 0.289 across settings, so no fixed budget or universal confidence threshold ports between loops.
• Below some checker quality, calibrate nothing. At near-zero discrimination the label-free calibration degenerates harder with more data, and the recovery is an estimation-free keep-best rule (commit the incumbent unless a challenger clears a hard vote margin) that recovers 0.223 → 0.793.

The scope caveat matters for how much to import: the damage rates driving this are largely manufactured by injecting a corrupted problem statement into the repairer, and the task is GSM8K-style math, not a repo. What generalizes is the shape — an unattended loop with a weak checker has a negative-return regime, and neither a round budget nor a pass-rate threshold can find its edge.

What one loop looks like#

Osmani's worked shape: an automation runs every morning, calling a triage skill that reads yesterday's CI failures, open issues, and recent commits, and writes findings to a markdown file or Linear board. For each finding worth doing, the thread opens an isolated worktree, sends a sub-agent to draft the fix, and a second sub-agent reviews that draft against the project skills and existing tests. Connectors open the PR and update the ticket; anything the loop can't handle lands in a triage inbox for the human. The state file is the spine — it remembers what was tried, what passed, what's still open, so tomorrow's run resumes where today stopped. "You designed it one time. You did not prompt any of those steps."

What the loop still doesn't do for you#

The loop changes the work; it doesn't delete the human from it. Three problems get sharper as the loop gets better, not easier — Osmani names each (these are his blog-series terms; the wiki's homes for the underlying ideas are linked):

• Verification is still yours. "A loop running unattended is also a loop making mistakes unattended." Even with a verifier sub-agent, "done" is a claim, not a proof — "your job is to ship code you confirmed works." → Verification as the New Bottleneck
• Comprehension rots if you let it. The faster the loop ships code you didn't write, the bigger the gap between what exists and what you understand — what Osmani calls comprehension debt, the cognitive sibling of Agentic Technical Debt. The antidote is the non-delegable bottleneck of understanding: read what the loop made.
• The comfortable posture is the dangerous one. When the loop runs itself, it's tempting to stop having an opinion and take whatever it returns — cognitive surrender. "Designing the loop is the cure when you do it with judgement and the accelerant when you do it to avoid thinking — same action, opposite result." This is "stay in the loop, treat them as tools" restated for the unattended case.

A fourth thread runs through the skills primitive: without skills the loop re-derives your whole project from zero every cycle — Osmani's intent debt. A skill is intent "written down on the outside" so it compounds instead of being re-guessed (see Agentic Technical Debt, Agent Context Files). And the human review ceiling is real: worktrees remove the mechanical collision, but "your review bandwidth decides how many you can actually run, not the tool" — the oversight-fatigue / span-of-control limit on unattended fan-out.

The deployed endpoint: codebases that maintain themselves (Anthropic, July 2026)#

Boris Cherny's YC interview describes Osmani's worked shape running in production at Anthropic, at fleet scale: "we actually have Claude maintaining itself now." A Slack channel is where the team had Claude start a set of routines (cloud-side loops — same primitive, survives laptop closure) against its own codebases — CLI, iOS, Android, desktop. Now 20–30 routines per day run across all the codebases, each a one-sentence prompt:

• Dead-code cleanup — daily static + dynamic analysis sweep, a deletion PR every day ("we didn't prompt that. It just kind of figured it out")
• Ship finished experiments — an experiment at 100% rollout gets its flag deleted and shipped
• Write missing tests — coverage gaps filled; and its inverse, delete useless tests "added by older models or added by people at some point"
• "Abstraction police" — find near-duplicate abstractions that drifted apart across a big codebase and unify them

Cherny's claims for it: hundreds to thousands of agents daily, "the work of dozens or hundreds of engineers," and "we're on the path to fully automating the maintenance of our apps" — freeing engineers for new product and user contact. His taxonomy distinguishes this from dynamic workflows: a workflow is one task broken into orchestrated chunks; a loop/routine is one repetitive task that "doesn't share context, but it might share memory." All practitioner-opinion, first-party, no published metrics — but it is the most concrete account yet of the essay's thesis deployed as standing infrastructure, maker and maintainer the same model.

Which loop is this, exactly?#

Two weeks after Osmani's essay, Andrew Ng responded to loop engineering "becoming a hot buzzphrase after mentions of it by Boris Cherny and Peter Steinberger went viral" — and answered a question the essay never asks: which loop? Ng's three-loop taxonomy places everything on this page inside the innermost loop, the one the agent closes alone at a cadence of minutes. Two slower loops sit outside it: the developer feedback loop (a human reviews the build and re-steers, tens of minutes to hours) and the external feedback loop (friends, alpha testers, A/B tests — hours to weeks), which is the only one that revises the vision rather than the spec.

The reframing is useful because it bounds the discipline. The five primitives make the inner loop faster; they do nothing to the outer two, and a product moves at the speed of its slowest loop. It also predicts the shape of the argument with Ambrosino's "loops are so last week" (Vibe Coding vs. Agentic Engineering): the inner loop is harness and will be absorbed by capability (Harness Shrinkage as Models Improve); the outer loops are the structure of product development and won't be.

The leverage point moved#

Two people can build the identical loop and get opposite results — one moves faster on work they understand deeply, the other avoids understanding the work at all; "the loop doesn't know the difference, you do." Cherny's point isn't that the work got easier — it's that the leverage point moved from prompt-crafting to loop-design, which is harder than prompt engineering, not easier. Osmani's closing balance: set up your loops, but don't forget direct prompting is still effective — "build it like someone who intends to stay the engineer, not just the person who presses go."

Connections#

• Vibe Coding vs. Agentic Engineering — Ambrosino's "loops are so last week" marks the frontier moving past orchestrated loops toward autonomous, supervised-vs-unsupervised development
• Agent Loop Pattern — the loop primitive (/loop, routines, Ralph Wiggum) that loop-engineering is the system-design discipline above; automations are this primitive on a schedule
• Agent Harness Engineering — Osmani: "loop engineering sits one floor above the harness"; the harness on a timer that spawns helpers and feeds itself; supplies the worktree-isolation and external-memory primitives
• Harness Shrinkage as Models Improve — the deployment-side evidence: bash-pile loops are being absorbed into products as named primitives; the harness shrinks as the pieces ship inside the tools
• Verification as the New Bottleneck — the maker/checker sub-agent split and /goal's fresh-model stop-check; "ship code you confirmed works"; review bandwidth as the unattended-fan-out ceiling
• Agent Context Files — skills as intent "written down outside"; the skill-is-authoring-format / plugin-is-distribution distinction; state files as memory
• MCP and Computer Use — connectors/plugins (MCP) as the primitive that lets the loop act inside your real tools, not just the filesystem
• Outsource Your Thinking, Not Your Understanding — comprehension debt and cognitive surrender are this thesis stressed by loop speed; understanding stays non-delegable
• Agentic Technical Debt — intent debt (loop re-derives the project each cycle) is the same compounding-drift failure; skills are the persistent-context antidote
• Jagged Intelligence (Ghosts, Not Animals) — "stay in the loop" is the cure for cognitive surrender
• Ticket-Driven Agent Orchestration — Linear-board-as-state is the durable work-graph form of the memory primitive
• AI Brain Fry / Human-AI Accountability Redesign — the human-oversight limit on unattended loops; span-of-control redesign is the missing partner
• Boris Cherny — "my job is to write loops"; primary practitioner; the self-maintaining-codebase fleet is his team's deployment
• Dynamic Workflows: An Algebra for Agents — the neighboring primitive: a workflow structures one task across staged agents; a loop/routine repeats one task on a schedule without shared context
• Peter Steinberger — originated the "design loops that prompt your agents" framing the essay is built on
• Claude Code / Symphony — the tool surfaces that now ship all five primitives (Claude Code and the Codex-side orchestration stack)
• Agentic Work Systematization — the empirical, adoption-curve counterpart to the skills primitive: OpenAI's Codex study measures ad-hoc→reusable-routine systematization at scale (skill use 5.4%→26.6%, 96.2% at OpenAI)
• Parallel Agent Orchestration — the measured fan-out the loop produces: "your review bandwidth decides how many you can run" is the ceiling on the 5+-concurrent-agent workflow OpenAI's data documents
• Agent Quality Flywheel — a vendor-packaged loop: Google ships the eval-fix cycle as a skill any coding agent drives; loop-design sold as product rather than hand-built
• Optimizer–Evaluator Decoupling — the maker/checker split and /goal's separate stop-checker, stated as an architectural invariant of improvement loops
• The Three Loops of AI-Native Building — Andrew Ng's taxonomy that locates this discipline: the five primitives all optimize the innermost of three nested loops; the developer-feedback and external-feedback loops are untouched, and a product moves at the speed of its slowest one
• Unknowns as the Agentic Bottleneck — comprehension debt made checkable: Thariq Shihipar's quiz gate ("I only merge after I pass the quiz perfectly") is the first concrete instrument for the deficit this page names
• Risk-Tiered Auto-Approval — loops aimed at review-adjacent toil, from PostHog (case-study): a PR-babysitting loop (CI monitoring, flaky-test reruns, branch freshness, comment triage) built on the same sweep/state design as Haack's babysit-prs, a qa-swarm reviewer panel with an outer loop that iterates until no new actionable threads appear, and an auto-stamper that decides which PRs need a human at all. Also the concrete cost figure this page's budget question lacks — Paul D'Ambra: "something like 60% of my token spend is burned automating the toil of handling CI and review and I don't regret a single dollar"
• Knowledge-Centric Self-Improvement — the memory primitive promoted to the object of improvement and measured: disposable agents, an external curated store as the only persistent state, and a bounded task-conditioned delivery rule for what the state file hands back
• Stopping Under a Noisy Verifier — the stop condition given a decision theory and a measured failure regime: when the checker's discrimination is low, a fixed round budget is catastrophically worse than committing the first draft (0.116 vs 0.700), the boundary belongs to the repairer rather than the checker, and the fallback below a certain checker quality is to stop calibrating and keep the incumbent
• Review as the Control Point — "fold human review into the agent's own loop / a second sub-agent checks the first" is one of the enumerated positions the CMU theory synthesizes; automated review is a review-dynamics construct there (raises throughput, cuts latency, but quality/security effect contested), and the theory formalizes the comprehension debt the maker/checker loop still accrues

Open Questions#

• Osmani's cost caveat is unquantified: at what token budget does a continuously-running loop stop paying for itself, and how do you instrument that? (Cf. Agent Loop Pattern's "who owns the budget when the model schedules its own loops.") Partially answered (magnitude only): a PostHog engineer running review/CI loops puts it at ~60% of personal token spend with no regret (Risk-Tiered Auto-Approval, case-study) — which establishes that the share can be a majority and still be judged worth it, but is a self-assessed share, not a break-even threshold or an instrument for finding one.
• If /goal's stop-check is itself a model, what verifies the verifier? The maker/checker split pushes the trust problem up a level, not away. Partially answered (2026-08-04) — the regress is bounded rather than closed: Wu et al. (empirical) don't verify the verifier; they make its unreliability a measurable scalar (Youden's J = 1 − ρ₀ − ρ₁) and show what each level of it permits. Above roughly J ≥ 0.18 a loop can act on a calibrated estimate of the checker's noise and stay within 2.8pp of the true-parameter reference; at J = 0.03 the label-free calibration collapses (and collapses further with more data), and the correct move is to stop trusting any estimate and fall back to an estimation-free keep-best rule. What is still open is the part the question actually asks: measuring J at all needs a small labeled probe, so the trust problem is relocated to a human-labeled sample rather than dissolved.
• Does loop-engineering converge on a single dominant shape (morning-triage → worktree → maker/checker → PR), or proliferate into many idiom-specific loops? The essay describes one shape "I keep using" but claims the primitives are general.

Sources#

• Loop Engineering — Addy Osmani, "Loop Engineering" (addyosmani.com, June 2026; surfaced via X)
• Boris Cherny: We Cut 80% of Claude Code's Prompt — Cherny, YC interview (2026-07-27, practitioner-opinion): the 20–30 daily self-maintenance routines (dead-code cleanup, experiment shipping, test writing/deletion, abstraction police) and the workflow/loop/routine taxonomy
• Driving the Agent Quality Flywheel from Your Coding Agent- Google Developers Blog — the eval-fix loop packaged as a coding-agent-driven skill (vendor-claim)
• Thread by @AndrewYNg — Andrew Ng, The Batch (2026-06-30), practitioner-opinion: which of three nested loops this discipline actually optimizes
• Verify, Repair, Repeat, or Stop? Robust Stopping for Noisy Verify-Repair Loops in LLM Agents — Wu, Shen, Yang, Peng & Hu (arXiv 2607.17641, 2026-07-20, empirical): §5.3 Table 1 and App. D Table 6 — the fixed-round-budget collapse and the per-instance marginal-gain rule that beats it; developed at Stopping Under a Noisy Verifier