# Howardism

> A Taiwan-based Software Engineer, Mathematician, and Amateur Diver sharing personal thoughts and journeys

Articles-only blog. Notes are organized into knowledge domains below; the full index is at https://www.howardism.dev/articles.

## Agent Systems

- [Agentic Code Generation as Compilation](https://www.howardism.dev/articles/agentic-codegen-as-compilation): Bridgewater's PAT treats a coding agent as a compiler, not an agent: the plan is a typed IR (a 'natural language Python project' of tasks, each declaring the schema of the data frame it must produce) rather than a to-do list, tasks compile to Python by parallel LLM generation because every consumer already knows its producers' schemas, validation agents run in DAG layers enforced by ordinary Python so an agent cannot skip them, and static analysis injects caching annotations making a re-run after a one-word edit near-instant. Self-reported: ~4x Claude Code's codegen latency, a 20-task plan costing the same wall clock as a 3-task one, two agents emitting identical code 95% of the time — no published method, and the determinism figure is measured downstream of a loop that edits code until it matches the task
- [Tree Search over Agent Trajectories (LATS)](https://www.howardism.dev/articles/agent-trajectory-tree-search): LATS (ICML 2024): run Monte Carlo Tree Search over an agent's action trajectories instead of committing to one — sample k actions from a node, execute each in the environment, score the resulting state with an LLM judge plus a self-consistency frequency term, select by UCT, roll out to a terminal state, back up the return as a running average, and append the model's own written reflection on why the branch succeeded or failed. Taught in CS329A lecture 5 as ReAct plus planning. The two limits the lecture concedes are the ones that matter: the cost is never analysed, and the whole method assumes actions are reversible
- [Guarantees That Degrade at Deployment: Action-Space Soundness, Admissibility Without Effect, and a Vendor-Coupled Security Framework](https://www.howardism.dev/articles/guarantees-that-degrade-at-deployment): Three control mechanisms that hold in the small case and bend in the deployed one: ReAct's enumerated-action guarantee survives by relocating from the prompt to a fail-closed runtime (where the binding constraint stops being context size and becomes policy coverage); a self-modification gate that adjudicates admissibility needs an exogenous effect test, and the corpus names both the formal reason and three working designs; and the Zero Trust ebook is vendor-neutral in doctrine, control domains and phases while 17 of its 21 Pro-tips name Claude Code — the coupling is in the worked examples, not the requirements, plus exactly one substantive target-state divergence from the IETF track
- [Reasoning–Acting Interleaving (ReAct)](https://www.howardism.dev/articles/reasoning-acting-interleaving): The 2022 prompting abstraction that made an agent out of a language model: alternate a free-text thought with a tool action and its observation, one pair at a time, so each thought is conditioned on what the environment just returned. Taught in CS329A lecture 4 as the origin of tool calling — it beats action-only prompting everywhere, beats chain-of-thought on fact-checking but not reliably on multi-hop QA, and swaps chain-of-thought's hallucination failure for retrieval failure. Its own fate is the interesting part: the interleave is now distilled into thinking models and the harness no longer supplies it
- [Retrieval Inside the Reasoning Chain](https://www.howardism.dev/articles/retrieval-inside-the-reasoning-chain): The three-rung ladder CS329A lecture 7 teaches as the ancestor of deep research: RAG retrieves once before thinking starts; agentic RAG lets the model emit a search between special tokens mid-chain and splices the documents back in; Search-o1 adds a reason-in-documents module that reads each retrieved document against the current query and appends only the extracted chunk. The dividing result is the document-count curve — direct reasoning and RAG go flat or down as documents are added, Search-o1 goes up — because the failure being fixed is long-context reasoning over retrieved noise, not retrieval. The trigger is the model's own hedging tokens ('perhaps', 'alternatively', 'wait'), and Search-R1 is the same loop moved into the weights by RL
- [Deterministic Engineering for Agent Code Review](https://www.howardism.dev/articles/deterministic-agent-code-review): OpenCodeReview (Alibaba / Nanjing / Peking, arXiv 2608.09290): three deterministic injections into a review agent — rule-driven file dispatch, a six-tool bounded-output set, and a filter-only reflector that sees LESS than the agent — beat Claude Code's /code-review 25.10% vs 11.57% SEM-F1 on AACR-Bench at 14.7x fewer tokens and 9.5x less wall clock. The headline is a precision/recall swap, not a free lunch: precision 33.90% vs 7.23% while recall FALLS 20.00% vs 28.90%, so the winner finds 301 of 1,505 expert-verified issues against the baseline's 435. Zero ablations — the only comparison is against two separately-built products, one pinned ~33 releases behind the multi-agent /code-review that shipped before the paper did — so which of the three injections carries the gain is untested, and the corpus's second determinism-beats-autonomy claim still has no gate-vs-instruction arm
- [Authority and Audit Survive Abundance](https://www.howardism.dev/articles/authority-and-audit-survive-abundance): Joint answer to two #oq/now questions. (1) At a model upgrade neither prescription governs the other: shrinkage governs instruction scaffolding (request-form lines — ablate and delete), crystallization governs authority scaffolding (evidence-gated permissions, which a capability jump cannot earn and circularity forbids delegating into the model) — sort each line by what it encodes (request vs constraint/record), not where it lives, and the demotion circuit-breaker makes the upgrade moment decision-free on the authority side. (2) The retrieval layer survives ~free long context because only its cost leg is token-priced: per-chunk governance cannot live inside the window it polices ('model promised to ignore' is not a boundary), and in-window attribution is model testimony where an audit needs a checkable log — selection is what makes a citation log non-trivial, a requirement that binds compiled wikis too. Cost dissolves with price; governance and audit dissolve only with the requirements themselves.
- [Document Parsing as the Retrieval Bottleneck](https://www.howardism.dev/articles/document-parsing-as-retrieval-bottleneck): Doulcet's 2024→2026 RAG retrospective: the bottleneck moved out of the model into retrieval, and inside retrieval into parsing — Glantz's 12 pain points cascade parsing→retrieval→synthesis, so one parsing failure lights up 7 of the other 11; reranking and corrective loops turned most pain points into routine engineering, long context did not kill RAG (cost, governance, audit), and what is left is structure loss at ingest — answered with spatial text, Markdown, structure-aligned chunking, and ParseBench
- [Crystallizing Agent Work into Workflows](https://www.howardism.dev/articles/crystallizing-agent-work-into-workflows): Malik's production lifecycle at Azure Networking: treat agent exploration as a discovery mechanism, not an execution model — promote repeatedly-validated agent behavior down a three-type spectrum (agent-orchestrated → hybrid → zero-token deterministic) on accumulated evidence, demote it automatically on regression; deterministic share 0→45% in eight months, per-incident cost −70% while volume doubled, and autonomy earned by a playbook's track record rather than by model capability
- [Automated Failure Attribution](https://www.howardism.dev/articles/automated-failure-attribution): WHO&WHEN PRO (Liu et al., 12,326 injected-error traces): LLMs mostly cannot attribute multi-agent failures — responsible-agent identification 48–58%, error-mode macro-F1 10.8–22.2, all-three-correct 16–25% vs a 90%+ human panel; accuracy collapses with trace length, and coordination-specific failures get absorbed into 'reasoning error'. The corpus's one collected-rather-than-injected failure set (multilingual planning-grounding, 80 traces filtered by English-succeeds/non-English-fails) confirms organic provenance is workable and shows the field defining the multi-fault case away by instructing its judge to mark a single primary cause.
- [Harness-Induced Belief Divergence](https://www.howardism.dev/articles/harness-induced-belief-divergence): Yi & Song: hold task, environment and base LLM fixed, vary only the harness, and the agent's elicited belief trajectory diverges — an interface-floor arrival term plus a growth term that reaches behavior (action disagreement 0.28→0.60, UnsafeRetryRate 0.700); the paper's 'preserves terminal success' framing is asserted, never measured.
- [Open-Ended Discovery Harnesses](https://www.howardism.dev/articles/open-ended-discovery-harnesses): Harness designs for hours-long agent runs on problems with no known optimum, where the recurring failure is idea collapse — committing to one approach early and micro-optimizing it forever; SwarmResearch's two moves (a global-context Shepherd steering branch-isolated Search Agents, one worktree per agent) match or beat EvoX/CORAL on 13/15 tasks, though all methods sit well below human SOTA on contest heuristics.
- [Stopping Under a Noisy Verifier](https://www.howardism.dev/articles/stopping-under-a-noisy-verifier): Wu et al.: with a noisy verifier and noisy repairer, a verify-repair loop's true quality peaks then declines while reported acceptance keeps rising; the stopping boundary b* = α/(α+β) is a property of the repairer — verifier discrimination (Youden's J) only locates you against it — and VRR-Stop acts on true marginal gain, with a keep-best fallback when J ≈ 0.
- [What Makes a Self-Improvement Artifact Transfer?](https://www.howardism.dev/articles/what-makes-self-improvement-artifacts-transfer): Answer to the transfer question left open by the HarnessBank/Caltech contradiction: an artifact transfers exactly as far as the regularity it encodes extends — solver-fitted artifacts (harness patches, verification instructions, model-per-role combos) reach only solvers sharing the pathology and depreciate on every release, domain-fitted artifacts (distilled knowledge, repo conventions, skills) survive solver churn because reuse holds the domain fixed; cross-model transplant failure and cross-release depreciation are one phenomenon, transferability can be selected for at write time (Caltech's schemas), and what transfers when the artifact doesn't is the procedure
- [Agent-Authored Harness Optimization](https://www.howardism.dev/articles/agent-authored-harness-optimization): An agent runs the whole eval-fix loop on its own harness — read traces, hypothesize, patch, re-run. Seven instances disagree: Cline's uncontrolled vendor campaign, HarnessBank's sealed-split credited gains, Ouroboros's unmeasured 161-day deployment, Wang et al.'s budget-matched test where harness evolution loses to plain parallel sampling, DarwinX's opposite result on the same benchmark with a categorical-effort rather than matched budget, Shopify's production loop — the only one that reports the gains running out and the team switching to weight updates — and Bridgewater's PAT, which reports no efficacy number at all but contributes two mechanisms the others lack: an agent-written benchmark that must fail before the fix, and a human gate placed at PR review. Reconciled by how broken the starting harness was.
- [Context Lifecycle Management](https://www.howardism.dev/articles/context-lifecycle-management): Treating an agent's active context as indexed runtime objects with a lifecycle (fold/mask/prune, recoverable sidecars, cache-aware commit) rather than a token buffer to trim — Xiaohongshu's Self-GC is the measured treatment (~44% prefix pruning at ~85% no-impact), plus the five-primitive taxonomy, the O(n²) full-append cost case, and RCWT's coordination-share cliff.
- [Deterministic Pre-Execution Gates](https://www.howardism.dev/articles/deterministic-pre-execution-gates): Reddy et al.: silent policy violations on policy-permissive tools are a distinct failure class (78% of τ²-bench airline failures are wrong final states with no tool error); four deterministic read-only gates over the proposed call raise task success +12.4pp — action-boundary enforcement can raise success, not just bound its safety cost, but per-gate precision must be audited.
- [Dynamic Workflows: An Algebra for Agents](https://www.howardism.dev/articles/dynamic-workflows-agent-algebra): Claude Code's sandboxed orchestration primitive: Claude writes and runs a program that composes agents in sequence and parallel inside a Bun VM — Cherny frames it as a new way to scale test-time compute, and Jarred Sumner's first-party Bun Zig→Rust port is the published methodology behind it (535,496 lines ported in 11 days, ~50 workflows, 6,502 commits, peak 64 concurrent Claudes, ~$165k of tokens, 1M+ test assertions as the oracle) — with an outside audit showing the ~$165k bought cost-to-green, not cost-to-shipped, and a shipped product default (v2.1.219) that aims workflows at fewer than 15 agents
- [Harness Build-vs-Buy](https://www.howardism.dev/articles/harness-build-vs-buy): The measured price of owning a coding agent: 12 months of public GitHub activity across four harnesses (OpenHands, Codex, OpenCode, Hermes) shows 5,679–7,736 merged PRs/year and 1.05M–1.75M lines each, so a fork frozen a year ago sits ~4,600 PRs (≈13/day) behind upstream — an OpenHands (vendor, commercially interested) argument for customizing at the highest layer that works: prompts/config → MCP → skills/plugins → SDK
- [Knowledge-Centric Self-Improvement](https://www.howardism.dev/articles/knowledge-centric-self-improvement): Caltech's inversion of self-improving agents: keep the agent generic, stateless and disposable, and make a curated knowledge base the only persistent object — task-level forums, cross-task forums, then distillation into typed bundles. Beats agent-centric (DGM, HyperAgents) and prompt-optimization (GEPA, OpenEvolve) baselines on five benchmarks at lower dollar cost, and the frozen bundle transfers zero-shot to held-out tasks and across LLM families in every donor-recipient pairing — the opposite of what happens when an evolved harness is transplanted. Also the vault's home for the what-should-persist taxonomy: agent, harness, knowledge, prompt, and (from one production case-study) the weights, which sit downstream of the harness rather than beside it
- [Layerwise Omission Attribution](https://www.howardism.dev/articles/layerwise-omission-attribution): Rajan: omission — a decision-critical fact silently missing from an answer — is a pipeline property assignable to one of nine layers by canary checkpoint taps (deterministic L0-L3 counted exactly, behavioral L4-L8 by contrast); the designed-injection waterfall doesn't give production prevalence, but the taxonomy, tap method, and three omission-raising operator knobs survive. The nine layers stop short of one boundary: a commitment lost at request-to-plan leaves every byte intact, and the multilingual-planning source's typed task representation is this page's canary used prophylactically with the free measurement never taken.
- [Orchestration Sets Token Economics](https://www.howardism.dev/articles/orchestration-sets-token-economics): Writer's controlled harness swap — same 22 tasks, same six models, same judges and price table, only the orchestration layer changes — moves cost per task −41%, tokens −38% and wall-clock −44% at quality parity, with every model cheaper by 33–61%; efficiency gains are model-invariant while quality gains scale almost perfectly with baseline capability (harness leverage, r = 0.99), and one net-new feature carries a capability floor below which exposing it produces failures; plus 'token maxing' as a named trajectory, the effective-input-price model under caching, the vendor-measures-own-product caveat that qualifies all of it, and Databricks' production counterpart on a multi-million-line codebase — three shipped third-party harnesses, same success rate at 2× less cost, ~3.1× per-task context spread
- [Prompt-Cache Economics](https://www.howardism.dev/articles/prompt-cache-economics): Prompt caching and prompt compression are one joint optimization, not two independent levers — CAPC measures Anthropic Sonnet 4.6's cache at ρ ≈ 0.83 rather than the compression literature's assumed 1.0, finds a step change near 3,500 cached tokens, derives a provider-agnostic crossover from three pricing constants, and shows query-aware compression costing +40.1% more than sending nothing compressed on a public benchmark; the corpus's first end-to-end billed-cost audit of a production caching API ($98.96 total, reconciled to Anthropic's invoice within 1%)
- [Shared Harness, Differentiated Surfaces](https://www.howardism.dev/articles/shared-harness-differentiated-surfaces): OpenAI merged Codex and ChatGPT Work onto one agent harness and differentiated only the UX layer — git-state visibility, diff-forward display, sandboxing defaults — which is exactly the residue Boris Cherny says is all that's left of Claude Code's harness; Anthropic took the opposite route, splitting by output type into Claude Code and Cowork
- [Tool-Output Pruning](https://www.howardism.dev/articles/tool-output-pruning): Compressing tool outputs at the agent-environment boundary before they enter history — SWE-Pruner Pro shows the keep-or-prune signal is already inside the coding agent's own backbone (linear probe AUC 0.83), so an 18M-parameter head riding the existing prefill replaces the separate scoring model: up to 39% fewer end-to-end tokens at held quality and the only one of seven pruners that never inflates tokens, at ~15% added wall time — but on SWE-Bench Verified every pruner raised input tokens on one backbone and lost resolves on the other. A second measured pruner, in a deep-research pipeline, adds the placement result: where you cut beats what you cut with, late-only pruning is a measured net loss, and neither paper reports a dollar.
- [App Server vs MCP, and the Claude-Side Equivalent: Three Boundaries for Driving Agents](https://www.howardism.dev/articles/app-server-vs-mcp-vs-claude-sdk): Two-question synthesis on agent integration boundaries. (1) App Server and MCP are not competitors — they sit at different planes: MCP is the model↔world tool plane (a connector written once, consumed by every surface, operated and authenticated by the tool provider), App Server is the orchestrator↔runtime session plane (thread lifecycle, turns, structured events, timeouts — nothing MCP covers). The only overlap is dynamic tool calls, and there the decision rule is operational: MCP wins for reusable, cross-surface, third-party capabilities; orchestrator-injected dynamic tools win for session-scoped, credential-sensitive capabilities (the linear_graphql pattern — the token never reaches the subagent container, shrinking the documented MCP attack surface of poisoned metadata and rug-pulls to first-party code, at the cost of experimental stability and zero ecosystem reuse). They compose: 'to the model, it's just tokens.' (2) Claude has no documented public equivalent of the App Server protocol; its offering brackets it from both sides — `claude -p` (drive the product CLI, inheriting the full accumulated harness: permission classifier, skills, context files, MCP wiring, with auto mode aborting rather than hanging unattended) and the Agent SDK (build a different product on the raw runtime — Claude Design's weekend prototype). Decision rule: drive the CLI when the product's harness is the value and orchestration is batch/fan-out shaped; build on the SDK when the agent is a different product needing its own tools, events, and UX; the App Server's middle position — structured session control over the product harness — is exactly the layer Symphony's tmux→protocol evolution shows demand for, and the layer Claude-side orchestrators currently approximate from either side
- [When Knowledge Layers Disagree: Context Files vs Memory, and Conflicting Sources at Compile Time](https://www.howardism.dev/articles/conflict-resolution-in-agent-knowledge-substrates): Two-question synthesis on conflict resolution in agent knowledge substrates, sharing one spine: disagreements are resolved by provenance and channel authority, never by content plausibility or recency — and the disagreement itself is a first-class signal routed to the maintenance loop. (1) Context file vs memory: split by disagreement type — on policy the context file always wins (it is the human-authored, git-reviewed, high-integrity channel; agent-written memory is advisory recall whose recency cannot confer authority, per the TMA-NM laundering theorem), on facts neither wins (both are caches over reality; the repo/live state is the source of truth, so verify then repair the stale cache), and in every case the conflict gets logged for the lint/pruning pass rather than silently broken — memory never overrides policy, and the context file is only ever updated through its own reviewed channel. (2) Conflicting sources at compile time: a five-step protocol extracted from the vault's own practice and its three worked cases — align constructs before declaring conflict (most contradictions dissolve into non-comparability: metric, population, time axis, unit), attach provenance and evidence tier and weigh by method+incentive (never average), stage genuine conflicts explicitly on every affected page with bidirectional links, convert them into tracked open questions, and treat resolution as a compile/lint-time librarian job that queries inherit rather than re-adjudicate
- [What Scaffolding Survives Model Improvement — and How Do You Know When a Line Turns Harmful?](https://www.howardism.dev/articles/what-scaffolding-survives-model-improvement): Three-question synthesis of the harness-evolution cluster. (1) Not all scaffolding migrates inward: behavior *requests* dissolve (and past-due ones turn harmful), but five classes survive — boundary enforcement (verification, isolation, security rules as constraints), org-specific record (repo-local truth, decisions no model can infer), deliberate identity (character/brand voice, kept stable across capability jumps by design), inference/deployment structure (no 'inward' to migrate to), and human-facing legibility (which grows as models improve) — plus one class flowing the *opposite* way (communication calibration, added as defaults lengthen). The bitter-lesson exemption rule sorts them: structure encoding a task prior migrates; structure encoding boundaries, records, identity, or serving arithmetic doesn't. (2) Compounding lines have a detectable signature — ablation non-inferiority (removal holds quality, cuts tokens) and inverted dose-response (stronger phrasing → worse outcome, the effort-inversion fingerprint) — with native-behavior baselining as the cheap pre-filter; no tool in the corpus automates it yet. (3) If model improvement stalls, build-for-the-next-model degrades gracefully: it is a cheap call option on the release cadence — a stall costs the premium (prototypes-in-waiting expire), not the firm; latent-capability overhang keeps effective capability rising post-stall; harness re-accretion becomes correct engineering again; and the durable layers become the competitive surface
- [Cost-per-Task Over Cost-per-Token](https://www.howardism.dev/articles/cost-per-task-over-cost-per-token): Anthropic's inverted model-selection default: start with the most capable model and dial effort down — a stronger model takes fewer turns, so cost-per-task falls even as price-per-token rises; plus Cursor's four-mix production measurement, Writer's harness swap (orchestration outweighs the model menu), and Databricks' bench where an open-weight model is cheapest at tied quality.
- [Instruction Compounding](https://www.howardism.dev/articles/instruction-compounding): When a model performs a behavior natively, an instruction telling it to do that behavior stops being redundant and becomes additive, pushing the behavior past its useful point — so Anthropic's Opus 5 prompting guide prescribes deleting verification, re-check, and don't-think instructions rather than rewording them; underneath it sits a measured capacity floor, with all-rules-obeyed compliance hitting zero by ~80 simultaneous instructions on all five models tested, independent of format
- [Output Length Calibration](https://www.howardism.dev/articles/output-length-calibration): Opus 5 runs longer by default on four independent output channels — conversational reply, agentic narration, files written to disk, and correction narration — and the effort parameter controls none of them: effort buys thinking, not talking, so each channel needs its own explicit length instruction
- [Agent Systems & Harness Engineering](https://www.howardism.dev/articles/moc-agent-systems): Map of Content for the agent-systems domain — 48 concepts. Harness engineering, agent loops and orchestration, context management, protocols and tool infrastructure (MCP, app servers), and subagents. Curated entry point; see Home for all domains.
- [Latent vs. Deterministic Space](https://www.howardism.dev/articles/latent-vs-deterministic-space): Garry Tan's diagnostic for agent-system bugs: computation lives in two places — latent space (the LLM: taste, judgment, vague-intent interpretation, steered by markdown) and deterministic space (generated code, external state) — and most AI-engineering failures are computation happening on the wrong side; now with one measured instance, where moving four policy rules out of a prompt document into Python predicates over database state recovers +12.4pp of agent task success
- [Single General Agent vs. Multi-Agent Coding Architecture](https://www.howardism.dev/articles/single-vs-multi-agent-coding-architecture): Resolves agent-harness-engineering's open question by re-drawing the line: a single general agent beats a bespoke hand-engineered multi-agent system as models improve (bitter lesson), but a monolithic-context agent does NOT beat role-separated context isolation + an independent grader — those survive model improvement because they fix structural constraints (quadratic attention, Goodhart), not model weaknesses
- [Agent Quality Flywheel](https://www.howardism.dev/articles/agent-quality-flywheel): Google's eval-fix loop packaged as a skill your coding agent drives: Build & Test → Ship & Monitor → Learn & Refine, expanded into five stages (prepare data / run inference / grade / analyze failures / optimize); plain-language worry in, metric choice and before/after deltas out; synthetic User Simulator bootstraps, production OTel traces sharpen. Shopify's Sidekick flywheel is the same loop with the terminus moved — instruction and harness edits until they plateau, then production failures mined into SFT+GRPO training signal, so each cycle starts from better weights rather than a longer prompt; its distillation curve crosses the production baseline between 26k and 30k trajectories, and its 96% serving-cost cut is a projection rather than an invoice
- [Failures That Look Like Success](https://www.howardism.dev/articles/failures-that-look-like-success): The quiet agent-failure class where everything reads fine — confident answer, plausible plan, even correct internal state — but the user-facing outcome is wrong; Google's flywheel demos caught agents echoing stale values despite correct memorize calls and silently skipping self-report instructions; measured at 78% of failures in one policy-permissive tool benchmark; its read-side twin is omission, a fact that never arrives, which a nine-layer pipeline taxonomy can attribute to a locus; detectable by trace-level rubrics, not output skims — and for the deterministic layers, by a byte diff needing no grader at all
- [Optimizer–Evaluator Decoupling](https://www.howardism.dev/articles/optimizer-evaluator-decoupling): The architectural rule in eval-fix loops that whatever proposes a fix (coding agent, automated optimizer, human) never grades it — an independent evaluation service scores the result, because an optimizer that grades its own work learns to game the metric instead of improving the agent
- [Parallel Agent Orchestration](https://www.howardism.dev/articles/parallel-agent-orchestration): One human overseeing a team of concurrent agents: OpenAI Codex telemetry's first hard numbers (28.6% of staff peaked at 5+ concurrent agents; p99 ~71 agent-hours/day), what breaks at agent-to-agent scale (Bun's 64-Claude constraint set, Cursor's coordination failures and harness rebuild), RCWT's fixed-budget coordination-tax cliff, and Anthropic's five-generation 12-hour swarm where the harness is held fixed and only the model moves — merge fraction and code sharing trade off until Sonnet 5, prompt-level org charts change nothing, and siloing is the coordination failure that scores well.
- [Loop Engineering](https://www.howardism.dev/articles/loop-engineering): Replacing yourself as the agent's prompter by designing the system that prompts it: a recursive-goal loop built from five product-native primitives (automations, worktrees, skills, connectors, sub-agents) plus external memory; tool-agnostic across Codex and Claude Code; the leverage point moves from prompt-crafting to loop-design; Anthropic's 20–30 daily self-maintenance routines per codebase are the deployed endpoint
- [Repository Exploration Subagent](https://www.howardism.dev/articles/repository-exploration-subagent): FastContext's thesis that repository exploration (read/search/localization) should be decoupled from solving into a dedicated read-only subagent that issues parallel tool calls and returns compact file-line citations, keeping the solver's context clean — cutting main-agent tokens up to 60% and lifting SWE-bench resolution up to 5.5%
- [Deep Research Agents](https://www.howardism.dev/articles/deep-research-agents): Agentic systems that decompose a complex query, iteratively search diverse sources, and synthesize a structured, cited report — distinct from single-shot QA; DRACO shows orchestration (Perplexity) beats the bare base model with tools, and factual accuracy is the weak axis. MisKnow-Agent numbers that weakness from the input side: one plausible-but-false document, with no instruction injection anywhere, raises false-conclusion adoption from 0% to 54.7% — and the same models that endorse those documents in-workflow flag them when handed them alone. The third attack surface is the root of the tree, where a typed pre-planning representation buys +3.6 to +10 points at a frozen model. On cost: 94.41% of an unpruned run's tokens go to result processing, early branch pruning cuts two thirds at 97.9% of baseline quality, and no arm in a 39-config grid keeps the baseline's key-point coverage
- [Build for the Next Model](https://www.howardism.dev/articles/build-for-the-next-model): Prototype the thing that almost works, not the thing that already works: bet that the next concrete model release (not a far-future AGI) fixes what your engineering can't; Claude Design's Opus 4.7 payoff and OpenAI's 'the February Codex app would have failed in November' are the cleanest cases — same product shape, different-intelligence release, different outcome
- [Agent Control Plane Patterns: Tickets, Loops, Specs, and Memory Files](https://www.howardism.dev/articles/agent-control-plane-patterns): Layered agent control-plane synthesis: tickets as durable work graph, loops as execution primitive, specs/context files as policy, memory as bounded recall, app protocols as runtime boundary
- [Where Does Agent Harness Work Remain Durable as Models Improve?](https://www.howardism.dev/articles/durable-agent-harness-work): Durable harness work lives at external-reality boundaries: repo-local source of truth, mechanical verification, context budgeting, isolation, tool contracts, and human decision surfaces; capability scaffolding shrinks
- [The Future of Agent Interfaces](https://www.howardism.dev/articles/future-agent-interfaces): Interface future is layered: native interaction models for human collaboration, MCP/APIs for structured action, app protocols for agent runtimes, computer use for legacy GUI fallback
- [Agent Context Files](https://www.howardism.dev/articles/agent-context-files): The cross-vendor markdown-as-control-plane pattern: repo-versioned plaintext (CLAUDE.md / AGENTS.md / SOUL.md / WORKFLOW.md / SPEC.md / .cursorrules) that configures agent behavior, split by role across project / personality / workflow / spec layers — and, since Genkit implemented SKILL.md loading in four language SDKs, a convention with a second vendor's runtime behind it as well as its authoring conventions
- [Agent-Native Infrastructure](https://www.howardism.dev/articles/agent-native-infrastructure): The world is still built for humans and must be rewritten for agents; "what do I copy-paste to my agent?"; sensors/actuators; agent-to-agent representation
- [Does the Human-Facing Harness (HTML Artifacts) Hit Its Own Bloat Ceiling?](https://www.howardism.dev/articles/human-facing-harness-bloat-ceiling): Yes — HTML raises and reshapes the human-attention ceiling but can't remove it; bloat relocates from document-length to artifact-sprawl/rubber-stamping; the ceiling gets *more* binding as models improve (inverse of the shrinking model-facing harness)
- [MCP and Computer Use](https://www.howardism.dev/articles/mcp-and-computer-use): Anthropic's two complementary connector mechanisms: MCP for structured programmatic access (Salesforce/Drive/Gmail/Slack/Figma + niche industry systems); computer use as the GUI-driving catchall when no MCP exists; Boris Cherny's "to the model, it's just tokens" — plus the vault's dated ledger of the MCP wire protocol itself, now at revision 2026-07-28: sessions and the initialize handshake removed, per-request version negotiation in _meta, a mandatory server/discover RPC, MRTR replacing all server-initiated requests, required ttlMs/cacheScope caching fields, and a feature-lifecycle policy with a 12-month deprecation window and a deprecated-features registry (Roots/Sampling/Logging, HTTP+SSE, OAuth DCR→Client ID Metadata Documents)
- [Agent Loop Pattern](https://www.howardism.dev/articles/agent-loop-pattern): `/loop` (cron-scheduled) and Ralph Wiggum (backlog-draining) loops as next-generation agent primitive; AFK execution, parallel fan-out, "loops are the future"
- [Context Window Smart Zone](https://www.howardism.dev/articles/context-window-smart-zone): Smart zone vs dumb zone (Dex Hardy / Matt Pocock): quadratic attention scaling, ~100K marker independent of advertised context; clear-and-restart > compaction; status-line token counting as essential discipline
- [Deep Modules for Agents](https://www.howardism.dev/articles/deep-modules-for-agents): Ousterhout deep-vs-shallow modules applied to agent-friendly codebases; push-vs-pull instruction delivery; reviewer in fresh context; Sandcastle three-agent pattern
- [Harness Shrinkage as Models Improve](https://www.howardism.dev/articles/harness-shrinkage-as-models-improve): Prompt scaffolding shrinks each model release; Cat Wu's pruning discipline; Boris Cherny "100 lines of code a year from now" claim; Anthropic deleted >80% of Claude Code's system prompt for Claude 5 models — and Cherny reports the model is slightly *more* intelligent without the prompts (ablation via SIMPLE=1); the user-side form is delegation rather than deletion ("use your judgement"); mechanical verification stays load-bearing
- [Codex App Server Protocol](https://www.howardism.dev/articles/codex-app-server-protocol): JSON-RPC stdio protocol for headless Codex sessions: initialize/initialized/thread-start/turn-start handshake, continuation turns reuse thread_id, dynamic tool calls for token-isolated tool injection — and, since MCP spec 2026-07-28 deleted sessions and the initialize handshake outright, the protocol pair has diverged on statefulness: MCP walked away from session semantics while session semantics are this protocol's entire subject matter
- [Ticket-Driven Agent Orchestration](https://www.howardism.dev/articles/ticket-driven-agent-orchestration): The inversion that makes Symphony work: tickets as units of work (not sessions/PRs), DAG dependencies, agent-extensible work graph, "objectives not transitions"
- [Claude Code Auto Mode](https://www.howardism.dev/articles/claude-code-auto-mode): Claude Code permission mode using a classifier to auto-approve safe tool calls and block risky ones; middle ground between default and `--dangerously-skip-permissions`
- [Client-Side Agent Optimization](https://www.howardism.dev/articles/client-side-agent-optimization): AgentOpt's framing of developer-controlled agent optimization (model-per-role, budget, routing) as distinct from server-side serving; the combo abstraction; 13–32× cost gaps between best/worst combinations — reproduced in production by Cursor's four planner/worker mixes, where cross-role coupling shows up in the bill and the 'strongest model is the worst planner' result turns out to be a harness property
- [Agent Harness Engineering](https://www.howardism.dev/articles/agent-harness-engineering): Patterns for scaffolding long-running LLM agents: environment design, progressive context disclosure, mechanical architecture enforcement, agent code review
- [Claude Code Best Practices](https://www.howardism.dev/articles/claude-code-best-practices): Anthropic's guide to effective Claude Code usage: context management, verification-driven development, explore→plan→code workflow, environment config
- [LLM-as-Compiler Knowledge Base](https://www.howardism.dev/articles/llm-as-compiler-knowledge-base): Karpathy's architecture: LLM incrementally compiles raw docs into a persistent interlinked wiki, replacing RAG with a 4-phase ingest→compile→query→lint pipeline — industrialized by July 2026 as 'agent wikis' (Cognition DeepWiki, Factory AutoWiki, LangChain OpenWiki, GBrain), same three-layer structure, differing on maintenance currency; Muscle Memory (Google Cloud, Aug 2026) is the first external source to argue compile-over-retrieve as its own position, on personalization memory rather than documents; the Open Knowledge Format (Google Cloud, June 2026) is the first spec to formalize the pattern for interoperability, standardizing the container — markdown, frontmatter, reserved index.md/log.md — while leaving provenance, contradictions and pruning entirely to the producer
- [What Are AI Tools?](https://www.howardism.dev/articles/what-are-ai-tools): Overview of AI tools landscape and categories

## Agent Security

- [Self-Propagating Prompt Injection (AI Worms)](https://www.howardism.dev/articles/self-propagating-prompt-injection): Indirect injection that reproduces: the payload instructs the assistant both to corrupt the document it is drafting and to copy itself into that output, so every generated document becomes a new carrier and propagation continues without the attacker or the original document. Håkon Måløy's 144-day coordinated MSRC disclosure (Copilot for Word, 2026-07-28, `case-study`) is the first public document-borne instance in a mainstream productivity suite — hidden formatting-concealed instructions in an attached source document alter financial figures and replicate into the draft, with Stage 2 reproducing after the original malicious document is gone; still exploitable at publication after two mitigation attempts, the second a model upgrade to GPT-5.5 that the attack defeated on GPT-5.6 the next day
- [Write-Then-Trusted](https://www.howardism.dev/articles/write-then-trusted): The seam where sandboxed agents escape without breaking anything: the agent writes a file it is fully permitted to write, and an unsandboxed host component later runs, loads, scans, or trusts it — so confining the agent *process* does not confine the agent. Pillar Security's eight reproduced escapes across Cursor, Codex CLI, Gemini CLI and Antigravity (CVE-2026-48124, GHSA-v4xv-rqh3-w9mc, GHSA-p9g2-cr55-cw9c, fixes in Cursor 3.0.0 / Codex CLI 0.95.0) anchor four failure modes — denylist sandboxes, workspace config that is really code, allowlists trusting command names not invocations, and privileged local daemons outside the box
- [Autonomous Intrusion](https://www.howardism.dev/articles/autonomous-intrusion): The corpus's first in-the-wild intrusion driven end-to-end by autonomous models — Hugging Face's July 2026 breach, re-attributed on 2026-07-21 by OpenAI to its *own* cyber-capability evaluation: GPT-5.6 Sol plus an internal-only pre-release prototype, run with reduced cyber refusals and no production classifiers on the ExploitGym benchmark, which escaped a no-internet sandbox through an Artifactory zero-day and breached HF production to steal the benchmark's answer key; three first-party accounts plus the affected vendor's confirmation that the escape vector was a genuine zero-day (patched in Artifactory 7.161), ~17,600 recorded actions over 4.5 days, two dataset-loader initial-access vectors, and grader gaming executed as real-world intrusion
- [Bind, Don't Forbid; Prevent, Don't Detect: The Action-Open and Poisoned-Memory Residuals](https://www.howardism.dev/articles/bind-dont-forbid-and-prevent-dont-detect): Two-question synthesis closing the remaining agent-security #oq/now pair, both instances of the detection-lost-structure-won arc. (1) Forbidding action-open delegation is the wrong control class: it is a discipline prescription aimed at exactly the party least equipped to comply (non-expert users are who under-specifies), i.e. friction — and it sacrifices the delegation value that makes agents useful. Binding achieves the security goal structurally: under-specification hands action-constraining defenses their best case (no named action → conservative trajectory → injected writes blocked regardless of phrasing), so the dangerous configuration is not action-open-plus-user but action-open-plus-filters-only. The ordered prescription: bind by default; when binding starves a genuinely open task, have the *system elicit* specification (clarification-before-commit — the same move unknown-elicitation prescribes on quality grounds, so security and quality co-fund one discipline); and route the safety-critical remainder through per-action authorization (the one channel measured at 100% on protected actions). (2) Nothing catches semantically-poisoned-but-cryptographically-intact memory — provably: the laundering separation theorem shows no content- or lineage-based detector is sound against it. The question's premise (catch it) is retired and replaced by prevention by construction: bind authority-to-act to origin at write time, non-malleably, so the laundered item stays act=none however benign it reads (0% attack-success across 8 models at full utility). Detection's residual role is forensics, not defense
- [Classifier Gates vs OS Sandboxing: The Defense-in-Depth Story for Auto Mode and Cowork](https://www.howardism.dev/articles/classifier-gate-vs-sandbox-layering): Two-question synthesis. (1) Auto mode's classifier and OS-level sandboxing are different control kinds on the impossible/tedious axis — a model-based semantic gate (probabilistic, defeatable from inside: NLA readouts caught a hallucinated user approval preceding a blocked-deletion workaround) versus structural capability removal — and they cover each other's blind spots: the classifier judges intent the sandbox can't see (within-capability harm over allowed channels), the sandbox bounds blast radius when the classifier's two documented failure modes (ambiguous intent, missing environment context) let something through. Layer both whenever the agent holds reach beyond the sandbox boundary (live credentials, MCP to real SaaS — the lethal-trifecta condition), runs unattended, or reads untrusted input; sandbox-only is legitimate when the workload is fully containable (the Hermes container-is-the-boundary design point); classifier-only is a stopgap for interactive low-stakes local work. (2) Cowork's computer-use guardrail is not a different mechanism — it is auto-mode-style classifier gating deployed on the browser/computer-use surface (Opus 5 card: 0/129 browser attack scenarios with auto mode vs 3.70% bare) — but the risk profile inverts the layering: Claude Code can lean on containment (worktrees, containers) because its blast surface is local, while Cowork drives real SaaS with the user's authenticated sessions, where no OS sandbox equivalent exists, so the classifier is load-bearing precisely on the surface with the worst bare-model injection rate (31.5%) and the least reversible actions
- [Does 'Impossible, Not Tedious' Kill Defense-in-Depth? Layered Friction, Agent-Relativity, and the Frequency Paradox](https://www.howardism.dev/articles/layered-friction-vs-capability-removal): Demote, not invalidate: friction layers never sum to a barrier because an adaptive attacker with near-zero per-attempt cost optimizes against the joint stack (correlated failures, attacker moves last) — the adaptive floor of a pure-friction stack is set by the model, not the layer count; friction retains value only as residual-reduction on top of at least one capability-removing gate (Opus 5's probes+classifier two-layer architecture is the deployed instance). The test is adversary-cost-relative, not agent-absolute: 'impossible' controls bind every actor, 'tedious' ones are priced in the attacker's attempt-cost curve, so mixed threat models must evaluate each control against the cheapest adversary class able to attempt the attack. And the least-agency frequency paradox dissolves on mechanism: a resettable rate (throttle) is friction; a cardinality bound tied to an out-of-band authorization event (single-use nonce, transaction token, expiring token, idempotency cap) is capability removal — 'how often' is a barrier exactly when the counter lives outside the agent's trust domain and reaching it denies rather than delays
- [Agent Security](https://www.howardism.dev/articles/moc-agent-security): Map of Content for the agent-security domain — 21 concepts. Attacks and defenses for agentic systems: prompt and data injection, tool and memory poisoning, identity and authorization, and zero trust. Curated entry point; see Home for all domains.
- [Capability Gating Is Not Authorization](https://www.howardism.dev/articles/capability-gating-vs-authorization): Agent frameworks ship capability gating (which tools are exposed, schema validity) but no fail-closed per-call authorization of argument values, so well-typed unauthorized calls pass; ScopeGate's deterministic PDP/PEP re-authorizes each call against out-of-band policy (0 bypasses, 0 false-denies), replicated by NetInjectBench — and cheaper deployment-tier models attempt unauthorized calls ~3.2× more.
- [Can Models Learn to Separate Instructions from Data? Durable Property vs Training Gap](https://www.howardism.dev/articles/instruction-data-separation-durable-or-trainable): Durable at the level that matters: the instruction/data boundary is trainable one delimiter at a time (hardening drives instruction injection to ~0%) but not in general — each closed boundary relocates the attack to the next finer one (instruction→data, then trusted→untrusted data), because the root cause is the LLM's probabilistic reading of inexact structural delimiters, an architectural fact. Newer models lower the per-boundary success rate but never produce a clean separation, and part of that gain is benchmark familiarity, not measured adaptive robustness — so the standing prescription across the cluster is to enforce the boundary outside the model with a deterministic action/data gate
- [MCP Tool Poisoning](https://www.howardism.dev/articles/mcp-tool-poisoning): The MCP Tool Poisoning Attack (TPA) class: adversarial or compromised MCP servers plant malicious instructions in tool metadata or tool returns — anchored by ShareLock's threshold secret-sharing variant (>90% ASR past single-tool scanners), the Agentjacking legit-server relay case study, and the 2026-07-28 MCP spec revision leaving the rug-pull intact.
- [Non-Malleable Memory Authority (TMA-NM)](https://www.howardism.dev/articles/non-malleable-memory-authority): Louck (arXiv 2606.24322): memory defenses deriving authority from content or lineage are provably unsound — adversaries launder poisoned items through self-summarization, trusted-tool echo, and manufactured corroboration; a TLA+ separation theorem shows write-time origin binding necessary, and the TMA-NM construction holds at 0% attack success where baselines fail as predicted.
- [Off-Host, Identity-Bound Authorization](https://www.howardism.dev/articles/off-host-identity-bound-authorization): aiAuthZ (Kodathala): an authorization gateway in a separate trust domain that HMAC-authenticates each human message and enforces role + argument-level policy the agent can neither read nor modify — a call's authority derives from the last verified human message, not model text; 0% residual attack success across 15 models; the off-host counterpart to ScopeGate.
- [Task-Specification Effects in Prompt Injection (AutoDojo)](https://www.howardism.dev/articles/task-specification-injection-surface): AutoDojo (Ma et al., arXiv 2606.15057): a cheap black-box adaptive attack that iteratively optimizes an indirect prompt injection against a live defended agent using only the success/fail signal — recovering 28% overall ASR (64% on action-open tasks) against a filter that scores 0% *static* ASR, so static-benchmark robustness dramatically overstates real robustness; plus the task-specification axis it exposes — under-specified 'action-open' tasks (the user defers the action itself to attacker-reachable content) are markedly more injectable than fully-specified ones for prompt- and filter-based defenses, while action-constraining system-level defenses invert this and grow stronger
- [Agent Data Injection (ADI)](https://www.howardism.dev/articles/agent-data-injection): A new category of indirect prompt injection: malicious payloads disguised as *trusted data* (metadata like a comment's author, a UI element ID, or the tool-call history) rather than as instructions, via probabilistic delimiter injection — the LLM misreads inexact/escaped delimiters as structural boundaries, so the agent does the user's task but on attacker-forged data; working RCE and supply-chain exploits on Claude Code / Codex / Gemini CLI and arbitrary-click on Claude-in-Chrome, bypassing IPI defenses that only separate instructions from data (up to 50% ASR where instruction injection is ~0%)
- [Agent Identity Management System (AIMS)](https://www.howardism.dev/articles/agent-identity-management-system): IETF draft-klrc-aiagent-auth: agents as WIMSE/SPIFFE-identified workloads with short-lived posture-assessed credentials and OAuth token-exchange delegation chains — the LLM never holds credentials; complemented by OpenID AuthZEN drafts (AARP, COAZ) and MCP spec 2026-07-28's self-legislated OAuth rules — the agent-auth governance layer is plural and moving.
- [Out-of-Band Prompt-Injection Defense](https://www.howardism.dev/articles/out-of-band-prompt-injection-defense): Second-generation prompt-injection defense enforced outside the model: a deterministic reference monitor mediates tool calls (CaMeL, FIDES, Progent, APPA) instead of training refusal — validated by an independent adaptive-attack reproduction, with cost inversions showing the overhead is a property of LLM-authored policy, not of enforcement.
- [Foundation → Enterprise → Advanced: Is the Agent Access-Control Jump a Cliff?](https://www.howardism.dev/articles/agent-access-control-tier-migration): No cliff — Enterprise (ABAC + dynamic privilege elevation with return-to-baseline + mTLS + sandboxing) is the pragmatic midpoint between Foundation static roles and Advanced JIT/JEA; migration runs identity-first, then least-agency, then blast-radius
- [Agent Identity and Authentication](https://www.howardism.dev/articles/agent-identity-and-authentication): The foundation control for agentic Zero Trust: cryptographically-rooted per-agent identity (→X.509→hardware attestation), short-lived IdP-issued tokens replacing static API keys (→mTLS→hardware-bound credentials), JIT access and ABAC — with MCP spec 2026-07-28 as the first shipping-protocol datum: issuer-keyed non-reusable client credentials as a MUST, RFC 9207 `iss` validation before code redemption, and OAuth Dynamic Client Registration deprecated in favor of Client ID Metadata Documents
- [Agent Supply Chain Risk](https://www.howardism.dev/articles/agent-supply-chain-risk): Runtime-composed agent ecosystems expand the supply-chain attack surface: model poisoning (250 docs backdoor a 13B model), tool/MCP supply chain (first in-the-wild malicious MCP server), AI-BOM, OpenSSF Scorecard, dependency audits, and AI vendoring as remediation
- [Agentic Prompt Injection](https://www.howardism.dev/articles/agentic-prompt-injection): Direct and indirect injection of malicious instructions into an agent; LLMs cannot reliably distinguish information from instructions; defenses are spotlighting (50%→<2%), constitutional classifiers (95% blocked), input isolation, and attack-surface reduction — but a second IPI category, agent data injection, forges *trusted data* rather than instructions and slips past all of them
- [AI-Accelerated Offense](https://www.howardism.dev/articles/ai-accelerated-offense): Frontier models compress the vulnerability-to-exploit timeline from months to hours at marginal dollar cost; both attackers and defenders speed up, the N-day window collapses, and the differentiator becomes strong fundamentals + breach-ready architecture
- [Autonomous Defense](https://www.howardism.dev/articles/autonomous-defense): Running security operations at the speed of AI-accelerated threats: put a model at the front of the alert queue, automate the bookkeeping (not the decisions), Agentic SOAR, MITRE ATT&CK coverage mapping, and rehearse five simultaneous incidents
- [Blast Radius (Agentic)](https://www.howardism.dev/articles/blast-radius): The potential damage if an agent is compromised; the unit Zero Trust's 'assume breach' posture is built to contain via identity-based isolation, sandboxing, and compartmentalization
- [Impossible, Not Tedious (Design Test)](https://www.howardism.dev/articles/impossible-not-tedious-test): Zero Trust design test for agentic security: does a control make the attack impossible, or just tedious? Friction-only controls degrade against agentic attackers with unlimited patience and near-zero per-attempt cost
- [Least Agency](https://www.howardism.dev/articles/least-agency): OWASP term extending least privilege to agents: constrain not just what an agent can access but what each tool can do, how often, and where; deny-by-default, per-agent credentials, scope limits
- [Memory and Context Poisoning](https://www.howardism.dev/articles/memory-and-context-poisoning): Corruption of persistent agent memory that influences behavior long after the initial injection — RAG poisoning, shared-context poisoning, slow long-term drift — defended via memory isolation, integrity validation, and retention policies; measured by Bad Memory (CLAUDE.md-class files, up to 97% persistence), GhostWriter (~98% injection from one email), and MemSecBench (lifecycle: adoption is the only real filter).
- [Zero Trust for AI Agents](https://www.howardism.dev/articles/zero-trust-for-ai-agents): Anthropic's security framework for deploying autonomous agents: trust nothing / verify everything / assume breach, applied across a Foundation→Enterprise→Advanced tier model and an 8-phase implementation workflow

## AI Coding Practice

- [The Committed-Artifact Chain](https://www.howardism.dev/articles/committed-artifact-chain): Anthropic's Applied AI SDLC playbook makes every stage end by committing a machine-readable artifact the next stage reads — intent.md → spec.md → plan.md → diff+tests → PR findings → incident record — so handoff becomes a merge event and the commit log doubles as the audit trail; the strongest version of markdown-as-interface in the corpus, and entirely prescriptive: every 'how to measure it' is an indicator to collect, not a result
- [Agent Review Comment Resolution](https://www.howardism.dev/articles/agent-review-comment-resolution): Cynthia, Widyasari, Roy, Zhang & Lo (Saskatchewan/SMU/Monash, arXiv 2607.21997): 54,713 agent-generated review comments from Copilot, Cursor and Codex across 341 Python GitHub repos — the first large-scale measurement of the review loop running the OTHER way, where the agent reviews and the human decides. ~71% of comments get resolved (Copilot 72.9%, Cursor 67.2%, Codex 54.8%), core developers do most of the resolving (78.1% of Copilot's), and an inline code suggestion is the strongest predictor (OR 1.62) while longer comments fare worse — but the model's AUC is 0.58, so most of what decides adoption is not in the comment. Card sorting 470 unresolved-but-argued discussions puts the modal failure at project context the agent cannot see (23.8%) and confident false positives (63), with outright hallucination at 4 of 470; another 24.3% were acted on and simply never marked resolved
- [Efficiency Debt of AI-Generated Code](https://www.howardism.dev/articles/efficiency-debt-of-ai-generated-code): Tran et al. (Google, arXiv 2608.06640): 3.52M changes over 12 months in one production C++ monorepo, with a human-written control cohort — AI-generated C++ writes ~2x the explicit loops and 30-40% fewer standard-library calls, and that source-level imperative bias shows up in production as ~5% relative compute and ~8% relative memory overhead. The reliability result splits: build failures ~1.3x and sanitizer findings ~1.3x above human, but revert rate ~0.9x BELOW. Review friction is real (blocking threads 1.92x) yet review depth does not predict which inefficiencies survive — the paper's own null, and its argument that this debt has to be caught upstream of review. Taxonomy-informed feedback cuts targeted findings 11.1%, leaving the regenerated functions still net-regressive against the human original
- [Post-Acceptance Edit Behavior](https://www.howardism.dev/articles/post-acceptance-edit-behavior): Liang et al. (CMU, arXiv 2607.25130): DECODE, 53.6K in-IDE edits of accepted AI completions from 1,141 developers — the first measurement of what humans DO to AI code after accepting it, pre-commit rather than at PR level. Half of all edits land within 50 minutes and the volume collapses after 15; retention is bimodal (median 63% survives, but the mass sits at 0% and 100%); 31% of trajectories contain a removal edit, and the developer who first tries to CUSTOMIZE a completion is the likeliest to delete it next (23.4% vs 12.2% after a functionality change). Which of 20 models wrote it barely matters (eta-squared 0.002–0.007). The prediction half is weaker than its abstract: fine-tuned 3B models beat their own base by +0.23 F1 but the best frontier baseline by only +0.08, and the dominant edit type — changing functionality — tops out near 0.49 Levenshtein similarity for every model tried
- [Same-Model Review Blindness](https://www.howardism.dev/articles/same-model-review-blindness): Greptile's Rodrigo Caridad on two 500-PR labelled datasets (~1,500 verified high-severity bugs): each frontier model catches fewer bugs in code authored by its own model family than in the other family's code — Opus 4.7 53.7% same-model vs 60.0% cross-model, GPT 5.5 50.5% vs 62.0%. The crossover is a pure interaction (both reviewers average ~56% overall and the two datasets differ by 2.6pp), but the post's offered mechanism — that a model misses the bug categories it produces most — reproduces only ~7% of its own headline when the category table is reweighted by the bug mix, so the blindness operates WITHIN category, not through composition. Vendor-built ground truth with an unspecified labelling procedure, one arm's prompt tuned against the outcome metric, no released artifact — filed case-study, not empirical
- [Agent-Generated Test Quality](https://www.howardism.dev/articles/agent-generated-test-quality): Two AIDev cuts on whether agent code is tested. Jhanglani et al. (204K test files): a trade, not a deficit — agents double human edge-case variety and match assertion strength but carry higher flakiness-candidate rates; three method defects cut into the numbers. Dipongkor et al. (4,882 PRs, ICSME 2026) measure tests against the diff instead: 50.4% of code-changing PRs carry no test change, existing tests execute 61.5% of agents' changed lines in Java and 27.0% in Python (64.8% of Python PRs zero), agent-written tests raise coverage in only 35.9%/22.5% of Code+Tests PRs, and error-handling constructs miss up to 86.0%. Breadth is intrinsic, targeting is relational, and only the second is a safety net — neither study has a human baseline
- [The HTML Artifact Lifecycle: Where Plan History Lives, and When Disposable Becomes Durable](https://www.howardism.dev/articles/html-artifact-lifecycle-versioning-and-reuse): Two-question synthesis on the lifecycle of human-facing HTML artifacts. (1) The diff/version problem dissolves once the artifact is recognized as a compiled *view*, not a record: version the content layer (markdown/config/repo — the copy-back round-trip and the extract-from-code pattern already do this), regenerate the presentation on demand, and let review reattach to decisions rather than diffs (the plan-ordered-by-likelihood-of-change technique puts the reviewable delta at the top). What's genuinely lost — blame history of the presentation itself — is acceptable precisely because presentation is regenerable at abundance prices; the moment a presentation choice is load-bearing, it graduates to durable tooling. (2) The templating question resolves by naming the correct reuse unit: not the artifact but the *generator* — a recurring micro-app becomes a skill that regenerates a fresh, fitted app each time (keeping disposable's per-task fit while gaining reuse's consistency), which is exactly the systematization move measured in the wild (skills 5.4%→26.6% of weekly-active users). An artifact itself graduates from disposable to durable only under recurrence + sync-pressure + audience (the design_system.html profile), at which point it stops being free: it inherits maintenance cost, sync cadence, and a seat under the artifact-sprawl bloat ceiling. The failure mode is the un-chosen middle — ad-hoc apps kept around unmaintained, which is sprawl plus rot with neither fit nor consistency
- [Is Human Review of AI-Authored Code Still a Real Control, or Already Rubber-Stamping?](https://www.howardism.dev/articles/human-review-real-control-or-rubber-stamp): Five-question synthesis of the oversight cluster. (1) 'Acceptance' at 60% blends three acts of different evidentiary weight — affirmative adoption, reviewed non-reversion, and bare non-reversion — and only the third is the rubber-stamp class; the construct, not the data, is why Faros and CMU can headline opposite trends. (2) The allocator-rubber-stamp risk is documented at three evidence layers (brain-fry error rates +11/+39%, 31.3% no-review telemetry, P2–P4 practitioner discourse): volume plus surface plausibility push the human past engagement, so the framing survives only with structural countermeasures (quiz gate, sample-based depth, risk-tiered gating) that make understanding rather than signature the merge condition. (3) 'How far to automate review' is a partition, not a dial: automate mechanical verification fully, keep human depth on a sampled/high-stakes slice — because the binding constraint isn't defect-catching (contested P9) but ownership, skill growth, and comprehension debt, which accrue regardless of who catches bugs. (4) Faros-vs-DORA is partly a category error — surveys measure felt productivity, telemetry measures system outcomes, both true at their layer — but the maturity-protection disagreement is substantive and unresolved. (5) Ng-vs-Faros is both-and: the 0-to-1/production scope split is real and does most of the work, while documented optimism bias means Ng's self-reported QA relief can't be read as measurement
- [Risk-Tiered Auto-Approval](https://www.howardism.dev/articles/risk-tiered-auto-approval): PostHog's StampHog: a merge-gate that auto-approves PRs passing four ordered checks (PR state, blast-radius deny-list, diff ceiling, last-position LLM veto) under fail-closed invariants — stamped ~1 in 3 merged PRs, displacing a rubber-stamp Slack ritual; the deployed instance of risk-tiered gating, with volume reported but no defect rate.
- [Security Debt of Agent-Generated Code](https://www.howardism.dev/articles/security-debt-of-agent-generated-code): Sakib, Banik & Jadliwala (UTSA, arXiv 2607.12428): LLM-as-judge + manual coding over 16,112 high-risk file changes in 4,022 AIDev agentic PRs — 38.9% of agent PRs carry ≥1 security smell, supply-chain integrity (mutable action/image tags, unpinned installs) is 82.3% of them, GitHub Actions + Dockerfiles hold 87.6%, hard-coded credentials are 99.6% of critical smells, and flagging climbs with PR size from 16.2% to 53.6%. The two RQ2 surprises invert the usual story: *humans*, not agents, committed 67.6% of the 74 genuine leaked credentials, and 81.1% of them reached integration with no comment from any bot or human reviewer. There is no human-PR control group, so it measures the security posture of agent-assisted workflows, not an agent-vs-human delta
- [Verifying Without a Compiler: Cowork's Harness vs Claude Code's, and Why the Slice Verifier Stays](https://www.howardism.dev/articles/verifying-without-a-compiler): Two-question synthesis on verification where no mechanical checker exists. (1) Cowork and Claude Code share primitives (skills, MCP, sub-agents, computer use) but sit on opposite ends of the verifier ladder, so the harness weight redistributes: Claude Code leans on a deterministic post-hoc verifier stack (tests, compiler, diffs, spec-drift checks) that both catches errors and bounds damage before merge; Cowork's outputs have no such rung, so its harness substitutes judgment-encodings for mechanical checks — the loaded design system as the nearest thing to a style linter, evals and LLM-judges for quality, human review concentrated at decision checkpoints — while the pre-action classifier gate becomes load-bearing because errors ship directly into live SaaS state with no red test in between. Failure modes split accordingly: loud (build breaks) vs silent (a polished deck that reads fine — the failures-that-look-like-success class), which is why accountability redesign matters more for Cowork, not less. (2) The planner needs the horizontal-slice verifier by design, not just empirically through 4.7: 'every slice produces end-to-end feedback' is a mechanically checkable invariant (does it touch schema+service+UI?), and checkable invariants belong in the deterministic checker regardless of model trust — the verifier is a constraint (doesn't compound, costs nothing to keep, catches the training-prior regression toward horizontal layering), while 'please slice vertically' is a behavior request, the transient form of the same discipline. Trust-the-model applies to prompt lines; verifiers are the durable class
- [Design by Selection](https://www.howardism.dev/articles/design-by-selection): Nate Parrott's Claude Design practice: when generating a candidate is nearly free, the designer's labor migrates to the two ends — deciding intent away from the keyboard, then hand-editing the last mile — while the middle becomes 'ask for ten options, then remix the two that work.' Left undirected the model collapses to a recognizable house aesthetic, so explicit aesthetic direction is the load-bearing input, and fidelity itself becomes a control knob (wireframe first when visuals would distract)
- [AI Coding Practice](https://www.howardism.dev/articles/moc-ai-coding-practice): Map of Content for the ai-coding-practice domain — 32 concepts. How humans and teams practice AI-assisted software work: workflow techniques, SDLC telemetry, review and verification as the bottleneck, and division of labor. Curated entry point; see Home for all domains.
- [Configurable Human Participation](https://www.howardism.dev/articles/configurable-human-participation): HAS-Bench (Wu et al.): human participation as a configurable benchmark variable (five-level agency scale × three interaction channels × personas, 397 tasks) — equal partnership (A3) beats full automation by +8.4 Pass@1 and recovers 65% of autonomy-failed tasks, but returns are configuration-dependent and diminish beyond A3; LLM-simulated human caveat.
- [LLM-Assisted Grey-Literature Theory Building](https://www.howardism.dev/articles/llm-assisted-grey-literature-theory-building): Agarwal et al.'s secondary contribution (arXiv 2607.07980): a scalable template for constructing grounded theory from thousands of practitioner documents instead of a few dozen interviews — LLMs do the mechanical, quote-anchored open coding (38,709 docs collected → Gemini relevance judge at κ=0.75 → 3,100 coded with the multi-agent Thematic-LM under three deliberately-polarized coder lenses → 4,838 codes / 109,951 quotes at ~$0.35/doc) while humans keep the interpretive axial/selective coding; automating that back half FAILED (a bottom-up pass yielded 15,029 shallow, redundant causal statements), so the codes→theory step stayed a manual, LLM-as-search-engine process — a division-of-labor lesson about what LLMs can and can't do in qualitative research
- [Review as the Control Point](https://www.howardism.dev/articles/review-as-the-control-point): Agarwal et al. (CMU, arXiv 2607.07980): a 26-construct/67-relationship causal theory synthesized from 3,100 coded practitioner documents — review is the control point through which a coding agent's effect on software is decided, and AI does NOT fix the sign of that effect; the team sets it through reviewer expertise, disposition, and how it adapts the review process (three moderators). Central core is review depth + reviewer skill, threaded by comprehension-debt feedback loops; the paper's own non-vendor GitHub telemetry (2.5M+ PRs) finds agent PRs reviewed less / merged several× faster / discussed less, but the trends flip direction under defensible analysis choices and the no-review rate CONVERGES toward the human baseline over time
- [The Under-Review Divergence: Faros's Widening Crisis vs. CMU's Convergence](https://www.howardism.dev/articles/under-review-divergence-faros-vs-cmu): Resolves acceleration-whiplash's open question: the Faros-vs-CMU under-review 'divergence' is mostly measurement artifact — a vendor's adoption-depth *delta in unreviewed-PR count* over enterprise all-PRs vs a non-vendor *calendar-time share* of unreviewed agent PRs in open source — and both fit one story: total unreviewed output rises with volume while the share of agent PRs merged unchecked falls as teams learn risk-triage; the volume-concentration clause is supported (median per-project no-review ≈0%, pooled >50%; triage by PR type), and the residual disagreement is a forecast — whether triage discipline survives agentic authoring crossing from <1% to double digits
- [The Three Loops of AI-Native Building](https://www.howardism.dev/articles/three-loops-of-ai-native-building): Andrew Ng's nested-loop taxonomy for 0-to-1 products: the agentic coding loop (minutes, agent-closed), the developer feedback loop (tens of minutes to hours, human-closed), and the external feedback loop (hours to weeks, market-closed); loop engineering has been optimizing only the innermost one, and the human's remaining job is a context transfer that lives in the outer two
- [Unknowns as the Agentic Bottleneck](https://www.howardism.dev/articles/unknowns-as-the-agentic-bottleneck): Thariq Shihipar's map-vs-territory thesis: the gap between what you told the agent and what the work actually requires is *unknowns*, and with Fable-class models the human's ability to surface them — not the model's capability — sets output quality; the Rumsfeld 2×2 applied to prompting, plus a phase-ordered catalog of elicitation techniques
- [Agentic Work Systematization](https://www.howardism.dev/articles/agentic-work-systematization): OpenAI Codex study's 'systematization' margin: the shift from ad-hoc agent use (describe task → agent does it → done) to reusable workflow infrastructure via skills and plugins; skill use rose 5.4%→26.6% of weekly-active users (Mar→Jun 2026) and is near-universal at OpenAI (96.2%); custom skills concentrate where shared conventions exist — but the measured post-adoption lifecycle is a one-time copy (53% of reused skills never modified, maintenance 2.7:1 additive), so systematization compounds only under a maintenance discipline most adopters skip
- [Acceleration Whiplash](https://www.howardism.dev/articles/acceleration-whiplash): Faros 2026: AI floods a human-paced SDLC with output it can't absorb — throughput up (tasks +34%, epics +66%), quality down (bugs +54%, incidents/PR +243%, review time 5x), gap widening with adoption and hitting even high-maturity orgs
- [Agentic Coding Work-Composition Shift](https://www.howardism.dev/articles/agentic-coding-work-composition-shift): Anthropic's 400K-session telemetry, Oct 2025→Apr 2026: as models improved, the share of sessions fixing broken code fell 33%→19% (debugging nearly halved), while operating software (14%→21%) and writing+data-analysis (~10%→~20%) grew; estimated task value rose ~25–27% — usage moving from firefighting toward end-to-end agentic work
- [AI as Primary Author](https://www.howardism.dev/articles/ai-as-primary-author): Faros 2026: the assistant→author threshold crossed without a deliberate decision, marked by AI-code acceptance rising 20%→60%; 'not an assistant, the author'; humans move from creation to oversight, making it an authoring problem not a review problem
- [Planning / Execution Division of Labor](https://www.howardism.dev/articles/planning-execution-division-of-labor): Anthropic's 400K-session telemetry: in a typical Claude Code session humans make ~70% of planning decisions (what to do) while Claude makes ~80% of execution decisions (how to do it); each prompt sets off ~10 actions (8 when the user keeps execution control, ~16 when Claude controls planning) — 'people decide what to build, the agent decides how'
- [Telemetry vs. Survey Measurement](https://www.howardism.dev/articles/telemetry-vs-survey-measurement): Perception lags reality: survey-based research (DORA) misses damage system telemetry catches — plus the family effect (instrument agreement tracks shared data source, not construct), randomization as the only causal instrument, the survey arm's counter-case (shadow AI is invisible to telemetry), and the Ramp payment-rail aperture; first cross-family convergence: Anthropic passing OpenAI mid-2026.
- [Human-in-the-Loop Boundaries](https://www.howardism.dev/articles/human-in-the-loop-boundaries): Humans belong at allocation, understanding, design-concept, risk, and accountability boundaries; they slow the system down as manual executors, universal reviewers, or ceremonial approvers
- [When Does Verification Quality Determine Whether AI Automation Works?](https://www.howardism.dev/articles/verifier-quality-and-agent-automation): Verification-quality ladder from Lean/formal proof search through software CI and vulnerability reproduction; autonomy should rise only to the level the verifier can support
- [Building Is Cheap, Arguing Is Expensive](https://www.howardism.dev/articles/building-is-cheap-arguing-is-expensive): "In technical debate, code wins": generate three PRs vs whiteboard; prototype over design doc; reduce design docs
- [Code as Source of Truth](https://www.howardism.dev/articles/code-as-source-of-truth): Docs go stale at high coding throughput; check specs/skills into the repo; onboard via Claude; spec-drift verification
- [Outsource Your Thinking, Not Your Understanding](https://www.howardism.dev/articles/outsource-thinking-not-understanding): "You can outsource your thinking but not your understanding"; understanding as the non-delegable human bottleneck; knowledge bases as understanding-tools
- [The Verifiability Thesis](https://www.howardism.dev/articles/verifiability-thesis): LLMs automate what you can *verify* as computers automate what you can *specify*; RL verification rewards → jagged peaks; "verifiable + labs care"; everything eventually verifiable
- [Verification as the New Bottleneck](https://www.howardism.dev/articles/verification-as-the-new-bottleneck): Fiona Fung: coding is no longer the bottleneck — verification, review, maintenance are; shift-left; TDD loses its tax; PR-cycle-time funnel analysis
- [Vibe Coding vs. Agentic Engineering](https://www.howardism.dev/articles/vibe-coding-vs-agentic-engineering): Vibe coding raises the floor (anyone builds); agentic engineering preserves the quality bar while going faster; ">10x and widening"; hire on big projects, not puzzles
- [Compute Allocator](https://www.howardism.dev/articles/compute-allocator): The human's evolving role: deciding what's worth spending compute on; ~1% of generated tokens ship, 99% is scaffolding invested in alignment/communication; abundance mindset
- [Disposable Micro-Apps](https://www.howardism.dev/articles/disposable-micro-apps): Throwaway custom UIs built per-task to edit a plan ("micro-software on top of micro-software"); copy-back-to-markdown; rational under the abundance mindset
- [HTML as the New Markdown](https://www.howardism.dev/articles/html-as-the-new-markdown): Thariq Shihipar's thesis: as models improve, thousand-line markdown plans overwhelm the *human*; HTML artifacts (visual, interactive) keep humans in the loop. The model-facing harness shrinks while this human-facing harness grows
- [Living Design System](https://www.howardism.dev/articles/living-design-system): `design_system.html` extracted from repos as a portable, human- and machine-readable source of truth; component playgrounds; bridges engineering ↔ non-technical stakeholders
- [Opinions on Using AI Tools & the Future of the Software Engineering Role](https://www.howardism.dev/articles/ai-tools-opinions-and-future-of-swe-role): Debate map of four stances on using AI tools (bullish-insider / pragmatist-practitioner / skeptic-governance / architecture-thesis) + synthesis on the future SWE role: coding→deciding/verifying, role convergence, what stays human, which moats survive, honest caveats
- [Design Concept Grilling](https://www.howardism.dev/articles/design-concept-grilling): Matt Pocock's `grill-me` skill; reach Brooks "design concept" before any plan; counter to specs-to-code; PRD as destination doc, Kanban as journey doc
- [Learning to Co-Work with AI: A Software Engineer's Field Guide](https://www.howardism.dev/articles/learning-to-cowork-with-ai-engineer-guide): Field guide for software engineers in the AI era: 6 skill clusters (taste, harness, alignment-first planning, agent-friendly architecture, verification, strategic positioning), daily practices, anti-patterns, 90-day plan
- [Vertical Slice Tracer Bullets](https://www.howardism.dev/articles/vertical-slice-tracer-bullets): Pragmatic-Programmer tracer-bullet pattern applied to agent task decomposition; vertical slices > horizontal layers; Kanban-with-blocking-edges over numbered phase plans

## Evals & Benchmarks

- [Skill Lift](https://www.howardism.dev/articles/skill-lift): NVIDIA SkillEvaluator's with/without-skill ablation turned into a publication gate: three pre-publication tiers (safety-and-structure scanning, catalog distinctiveness, live sandboxed A/B), a per-skill delta in rubric points reported alongside the artifact, and a first at-scale benchmark — 300+ verified skills × 30+ products × 2 harnesses, +41 Correctness and +39 Effectiveness, +34 (Claude Code) vs +29 (Codex), per-product spread +2 to +46, token cost moving in both directions (-76.9% and +120.3% on two single-attempt examples). The measurement's load-bearing weakness is that the eval set is generated from the skill under test
- [AI-Assisted Error Analysis](https://www.howardism.dev/articles/ai-assisted-error-analysis): Shreya Shankar's account of the one eval step that resists automation: discovering what counts as a failure. The argument is epistemic rather than technical — what 'good' means lives in the developer's head, not in the traces, and a tool that could fully find and fix your product's failures could do the same for every competitor, so judgment is the only differentiator left. AI competence rises monotonically along the analyze → measure → improve lifecycle and is weakest at the start. The working division of labor: the human authors the failure-mode taxonomy, the agent builds a bespoke review interface, clusters and samples the traces, and then scales each human annotation back across already-labeled traces — never proposing taxonomy of its own, because validating an agent's taste costs more than expressing your own
- [GDPval Benchmark](https://www.howardism.dev/articles/gdpval-benchmark): OpenAI's late-2025 benchmark of real, economically valuable knowledge work: ~1,320 tasks sourced from professionals with 10+ years' experience across 44 occupations in nine top-of-GDP sectors, scored as a pairwise win rate against those experts rather than as accuracy — rising roughly linearly from ~12% (GPT-4o, 2024) to ~48% (Claude Opus 4.1, 2025), which is the deflationary counterweight to METR's exponential time-horizon curve; the ancestor of the GDPval-AA Elo boards this wiki's model pages quote
- [Weak-Verifier Ensembling](https://www.howardism.dev/articles/weak-verifier-ensembling): Weaver (Stanford, 2025): stop training a better verifier and combine the imperfect ones you have — normalize a heterogeneous pool of ORMs, PRMs and LLM judges, drop the ones below a quality floor using ~1% of labels, then learn per-verifier weights by Snorkel-style weak supervision. Reported to lift hard benchmarks from ~40% to over 70% and to close a model-class gap (8B generator + ≤8B verifiers ≈ 70B majority voting; 70B class ≈ o3-mini at 86.2%), then distill the whole pool into a ~400M scorer keeping ~97% of the accuracy at 99%+ less test-time compute. Its load-bearing assumption — that verifiers capture independent aspects of correctness — is the one the wiki's 2026 empirical judge measurements say is false
- [Aggregate Cancellation](https://www.howardism.dev/articles/aggregate-cancellation): The failure mode where a headline metric stays flat because two real effects of opposite sign sum to zero across strata — so a null pooled result is evidence of heterogeneity, not of no effect; the cleanest measured instance is a sparse-attention audit whose three preregistered pooled tests return p = 0.995 / 0.771 / 0.541 *because* per-cell effects run opposite ways, and the wiki's other instances (identical accuracy over disjoint correct subsets, a group cooperation rate over one systematically drained member) share the shape: the composition under the number changed and the number did not
- [Expenditure Horizon](https://www.howardism.dev/articles/expenditure-horizon): METR's continuous generalization of time horizon: the dollar spend at which an agent's improvement on an optimization problem equals a human's at the same budget, measured by crossing an agent's returns-to-expenditure curve with the local returns to human labor — proof-of-concept on the NanoGPT speedrun, where humans cost ~$2,500 per 1% speedup and six agent runs from record #78 re-validate to horizons of $0-$3,300, of which the maintainer would merge ~70% of the ideas but only 50-60% of the speedup
- [Matched Comparisons for Memorization Claims](https://www.howardism.dev/articles/matched-comparison-memorization): Cooper et al. (arXiv 2607.12649): a generation rate measured only on training data is not a memorization rate — comparable non-training sequences must be scored by the identical procedure to supply a predictability floor. A conformal test calibrates the threshold to a chosen false-positive rate for populations, a census calibrates a single document against a matched control book, and 'extractable memorization' is redefined to require both a calibrated claim and near-certain generation within a realistic query budget.
- [Orchestration-Plan Simulation](https://www.howardism.dev/articles/orchestration-plan-simulation): OrchBench (Ren et al.): score a multi-agent orchestration plan without running workers — a deterministic simulator over a fixed task DAG correlates r=0.816 with real Claude Code quality at ~1% of the tokens; transfer coverage dominates agent count, multi-agent wins only under context pressure, and the headline correlation weakens sharply once the weakest planner is dropped.
- [Usage-Telemetry Classifier Validation](https://www.howardism.dev/articles/usage-telemetry-classifier-validation): Google ATLAS is the first AI-usage-economics program to publish accuracy numbers for the LLM classifiers every such study rests on — and they are humbling: 22.6% exact accuracy on O*NET task assignment and 42.5% on occupation title, against 85.8% human approval of the same labels; the accuracy/approval gulf, its mitigations (presence-not-frequency, task-type aggregation to 70.4%), and what it means for every headline number in the genre
- [Evals & Benchmarks](https://www.howardism.dev/articles/moc-evals-and-benchmarks): Map of Content for the evals-and-benchmarks domain — 20 concepts. The science of measuring models: benchmark validity, contamination, saturation, LLM judges, and production-sourced evaluation. Curated entry point; see Home for all domains.
- [Benchmark Contamination and Decontamination](https://www.howardism.dev/articles/benchmark-contamination-decontamination): Sun, Zhan & Gales (Cambridge): per-sample distribution distances expose that aggregate-accuracy decontamination can worsen residual contamination, and Uncertainty-Based Decontamination (UBD) — deep LoRA ensembles exposing memorized samples as confident-but-batch-order-sensitive — debiases without a clean reference model.
- [Benchmark Score Redundancy](https://www.howardism.dev/articles/benchmark-score-redundancy): Zeng & Papailiopoulos: an 84-model × 133-benchmark public score matrix is effectively rank-2, so BenchPress matrix completion predicts held-out scores to ~4.6 MedAE and a 5-benchmark probe set recovers a full scorecard. DeepMind's CollabEval takes the same premise down to models × prompts and inverts its use — completion output becomes a control variate inside prediction-powered inference, so the redundancy buys unbiased estimates with valid confidence intervals whose correctness survives the matrix not being low-rank at all (and item-level matrices need ~16 components, not 2).
- [How Much Signal Do Public Benchmarks Still Carry — and What Replaces Them?](https://www.howardism.dev/articles/benchmark-signal-and-what-replaces-it): Synthesis of the 2026 eval-science cluster: public benchmark suites carry far less independent signal than their count implies (133 benchmarks ≈ rank-2; accuracy saturates even after validity fixes) and the headline number is corrupted through four distinct channels (unnamed compute budget, contamination, vendor optimism, unvalidated judges) — but ordinal comparisons survive under verified invariances, and nothing replaces benchmarks wholesale: the field's answer is a five-part portfolio (predict-don't-run, re-instrument saturated suites, compute-controlled curves, production-sourced refresh, judge validation), with failure-mode discovery, contamination monitoring, and incentive shaping as the jobs only benchmarks still do
- [Measuring Beyond Accuracy Saturation](https://www.howardism.dev/articles/measuring-beyond-accuracy-saturation): Princeton-led case study (arXiv 2606.26158): accuracy saturation is not benchmark saturation — re-instrument a saturated benchmark instead of retiring it, because statistically-indistinguishable agents still differ sharply in reliability, cost-efficiency, scaffold contribution, and human-collaboration speedup (CORE-Bench); plus a second saturation mode where the answer key never saturates but the human reference class does, and re-eliciting that baseline becomes the recurring cost
- [Reference-Free Judge Over-Crediting](https://www.howardism.dev/articles/reference-free-judge-over-crediting): Reference answers are a first-order determinant of LLM-judge verdicts: without one, judges systematically over-credit wrong answers (up to 85% verdict flips when the reference is added — Kranti & Vajjala), and self-play against a reference-free judge inflates pass rate at flat true accuracy (Zhou); the fix is the judge committing its own answer first.
- [LLM-Judge Validation](https://www.howardism.dev/articles/llm-judge-validation): UC Berkeley's 21-judge / 9-provider / ~541K-judgment audit (Norman et al., 2026): LLM-as-a-judge validation is systematically under-rigorous — exact-match agreement overstates chance-corrected κ by 33–41pp (kappa deflation, universal across every judge), judge rankings shift up to 14 positions across benchmarks, and high test-retest reliability masks severe position bias (the consistency–bias paradox); distilled into a 5-step Minimum Viable Validation Protocol. Yang et al. (2026) add the judge-*version* axis: upgrading the evaluator is not a reliability intervention (one robust step in 18, and scaling makes it worse on 2 of 4 datasets), and repeated-sample juries are capped by measured error correlation ρ ≈ 0.66–0.97. Chen et al. (2026) push it down to the rubric *item*: measurability (judge agreement), informativeness (IRT information) and validity are three different properties
- [Compute-Controlled Benchmarking](https://www.howardism.dev/articles/compute-controlled-benchmarking): Noam Brown's critique: the single-number benchmark grid is broken because it ignores test-time compute — plot performance against a cost budget instead; benchmark-maxxing, held-out private sets, the Goodhart equilibrium that keeps the grid alive, disclosure exemplars (Kimi K3's footnotes, Gemini's price rows), the first budget-matched test of a named method class, and DarwinX as the specimen of an undefined effort tier presented as a compute control.
- [DRACO Benchmark](https://www.howardism.dev/articles/draco-benchmark): Perplexity's benchmark of 100 production-sourced deep-research tasks (10 domains, 40 countries) graded by 26-expert rubrics on accuracy/completeness/objectivity/citation; Perplexity Deep Research leads every domain and axis, Claude Opus 4.6 is the strongest non-Perplexity system, factual accuracy is the universal weak spot
- [LLM-as-a-Judge](https://www.howardism.dev/articles/llm-as-a-judge): Using one LLM to grade another's outputs against criteria/rubrics; DRACO's protocol is per-criterion binary MET/UNMET + justification, weight-aggregated into normalized score and pass rate; key properties — rankings stay stable across judge models while absolute magnitudes vary, and adaptive per-case rubrics (Google's AutoRaters) detect failures but blend them away, motivating stable custom metrics for the behavior under change; a 21-judge / ~541K-judgment audit finds raw exact-match agreement overstates chance-corrected reliability by 33–41pp (kappa deflation) and high test-retest can mask severe position bias, so judges need chance-correction, bias, and cross-benchmark validation before thresholded use; upstream, CalibratedRubric makes the rubric bank the instrument — measurability, informativeness and validity are distinct, and unanimity filters decay with leaderboard size
- [Production-Sourced Evaluation](https://www.howardism.dev/articles/production-sourced-evaluation): Building benchmarks from de-identified real production usage rather than synthetic or hand-authored tasks; DRACO's central method — difficulty-proxied sampling, PII-stripping, augmentation, automatable refresh with a human QA gate; representativeness vs. over-specification tradeoff; production traffic as a proprietary eval asset; plus the buyer-side instance, where a customer builds the eval from its own engineering work to decide what to buy, and the training-side instance, where production failures become RL trajectories and the difficulty proxy stops being independent of the model
- [Task Time-Horizon Scaling](https://www.howardism.dev/articles/task-time-horizon-scaling): METR's measure of the task length AI can complete reliably on its own, doubling roughly every 4 months (up from every 7): Opus 3 ~4min (Mar 2024) → Opus 4.6 ~12hr (2026) → weeks projected for 2027; paired with benchmark saturation (SWE-bench, CORE-Bench)
- [How Do You Write Evals for Taste? Character as the Limit Case](https://www.howardism.dev/articles/evals-for-taste-and-character): Taste-driven features are eval-resistant but not eval-proof: the technique is conviction → dogfood-sourced failure signals → A/B variant measurement (MSM's method) → ~10 interpretable judgment-encoding evals; demonstrated on safety/values, still open on warmth/wit
- [Scale-Dependent Prompt Sensitivity](https://www.howardism.dev/articles/scale-dependent-prompt-sensitivity): Large models underperform small ones on 7.7% of standard benchmarks due to overthinking; brevity constraints recover 26pp and fully reverse hierarchy on GSM8K/MMLU-STEM

## Model Capability & Training

- [The Data Wall and the Validation Commons Are One Supply Constraint](https://www.howardism.dev/articles/data-wall-and-validation-commons-one-supply): Two backlog questions about a supply running out before a trajectory arrives — training data for scaling, human validators for the Stockfish threshold — turn out to be the same question, because both supplies are *verified judgment*. The corpus's measured side says the pretraining-token wall never binds on its own terms: self-generated data is cheap in FLOPs and rationed instead by verifier availability, verifier *latency*, and generator diversity collapse, so the data wall does not demote into compute (as the RSI-frictions synthesis has it) — it converts into the verification friction that page already ranks first. On the ordering question the answer is a qualified negative: no domain in the corpus shows commons-scale validator depletion (Lovett says so himself), the closest measured instance is colonoscopy deskilling, and the general risk is smaller than stated because verifiability drives both the threshold's arrival and the validator's dispensability — but it is sharper than stated one level down, at the sub-task boundary, where formal math already shows the residual human job (checking the formalization, not the proof) surviving inside a domain whose verifiable rung is fully automated
- [RL from Execution Feedback (RLEF)](https://www.howardism.dev/articles/execution-feedback-rl): Train a coding model with the interpreter in the loop: generate code, run a small visible set of public tests, feed the failure text back for another attempt, and reward the surviving solution against a hidden private set — so the same execution feedback appears at inference time (exploit the policy) and at training time (update it). Taught in CS329A lecture 4 as the two-tier test split plus a turn-level value function. The mechanism the error analysis shows is not fewer first-try mistakes but *targeted repair*: with RLEF later turns fix the specific failure, without it the edits are not correct
- [Inference-Time Architecture Search](https://www.howardism.dev/articles/inference-time-architecture-search): Archon (Mirhoseini's lab, 2024): treat test-time scaling as an architecture-design problem — search over layered pipelines of prompting-only operations (generate, fuse, critic, rank, verify, unit-test-generate, unit-test-evaluate) across a pool of LLMs under an inference-call budget, using Bayesian optimization over a hand-constrained space. Two results that outlive the system: *fusion* — synthesizing one answer from k samples — beats oracle selection over the same k, breaking the ceiling the generation–verification gap is defined against; and stacking more inference layers keeps helping, like depth in a network
- [Intra-Trace Parallel Planning (SPRINT)](https://www.howardism.dev/articles/intra-trace-parallel-planning): A reasoning trace is a DAG being generated as if it were a chain: many of its steps do not depend on each other, but autoregressive decoding pays sequential latency for all of them anyway. SPRINT (Mirhoseini's lab, 2025) recovers the DAG — have GPT-4o segment DeepSeek-R1 traces into steps, tag each step's plan and execution parts, infer the dependency graph, repack into parallel groups, and supervised-fine-tune a 7B model on the reformatted trajectories so it emits independent plans together and their executions run at once. The surprise is that the accuracy went up too (~3.5 points), and that it generalized off the math data it was trained on
- [Offline Multi-Step Tool-Use RL (SWiRL)](https://www.howardism.dev/articles/offline-multi-step-tool-rl): SWiRL (Mirhoseini's lab, COLM 2025) trains multi-step tool use without ever calling a tool during the RL run: generate multi-step trajectories offline by iterative prompting, execute the tools once there, have an LLM judge score each action — grading the *query the model wrote*, not the result it got back — then optimize the expected per-step reward against that frozen context. Two findings outlive the recipe: process-filtered data beats outcome-filtered data for RL and the ordering reverses for SFT; and training on GSM8K with a calculator improves HotpotQA with a search engine, so what transfers is stepwise reasoning and tool invocation rather than any specific tool
- [Process vs Outcome Reward Models](https://www.howardism.dev/articles/process-vs-outcome-reward-models): The four-year arc of trained LLM verifiers as taught in CS329A lecture 3: OpenAI's GSM8K verifier (score the finished solution, per-token head, two losses) → Let's Verify Step by Step's PRM800K (800K human step labels; process supervision's real prize is killing the false positive where a hallucinated chain reaches a correct answer) → Math-Shepherd (replace the humans with rollout success rates from each step) — and the sting, that automating the step label reintroduces exactly the false positive process supervision was for. Plus the two findings that generalize (a trained verifier's precision *degrades* past a few hundred candidates; a bigger generator with a smaller verifier beats the reverse) and the rung the sting implies, DeepSeek-Math V2's **meta-verifier**, which grades the verifier's analysis rather than its score
- [Rationale Bootstrapping (STaR)](https://www.howardism.dev/articles/rationale-bootstrapping-self-training): The 2022 ancestor of every self-improvement loop that moves the weights: few-shot a model into producing reasoning chains, keep only the ones whose final answer is correct, fine-tune on them, repeat — plus the trick that makes it more than rejection sampling, *rationalization*, where a failed problem is re-attempted with the answer supplied as a hint and the resulting chain is trained on as if the model had solved it unaided. Its filter is the assumption the rest of the field inherited, its ceiling is the base model's reach, and the loop plateaus because it is not really RL. Plus the two 2025 descendants CS329A's closing lecture offers: Multiagent Finetuning names the plateau as *diversity collapse* and buys diversity with specialized generator and critic agents, and Absolute Zero deletes the human-curated question set by having the model propose its own tasks under a learnability reward
- [Selection Under a Submission Budget](https://www.howardism.dev/articles/selection-under-a-submission-budget): What repeated sampling costs when you may only submit n answers, not all k: CS329A lecture 7 walks AlphaCode's 1M-samples-per-problem pipeline, whose real content is the filter-and-cluster stage that gets 1M down to 10, and the 10@k metric that prices it — the gap between 10@k (~30%) and pass@k (~40%+) is the selection bottleneck measured directly. AlphaCode 2 replaces heuristic clustering with a learned scoring model plus a family of fine-tuned Gemini Pro variants for diversity, and reaches AlphaCode's solve rate at 100 samples instead of 1,000,000 — the lecturer's own reading being that a better base model is a cheaper lever than a bigger sampling budget
- [Error-Penalized Abstention Training](https://www.howardism.dev/articles/error-penalized-abstention-training): Paying a model +1 / −λ / 0 to answer, err, or abstain is provably right for a rational agent and can be self-defeating for a gradient learner: when abstention is a discrete action, the reward gradient and the KL anchor's restoring force carry the same saturation factor and die together, so coverage collapses to zero while logged mean reward rises like 1/t — and GRPO's group normalization silently replaces the designed penalty with λ_eff = 1, moving the learned threshold from λ/(1+λ) to 1/2
- [Turn-Level Credit Assignment](https://www.howardism.dev/articles/turn-level-credit-assignment): Giving a long-horizon agent per-turn reward instead of one terminal verdict, without step labels, an LLM judge, or a trained process-reward model — TRACE's answer is to split the rollout at tool-call boundaries, score each prefix by how predictable a *frozen* reference model finds the gold answer, turn that into a log-ratio 'fraction of the initial gap closed' state value, and hand each turn the TD change across its own boundary; the credits telescope so padding a trajectory cannot inflate them, and on closed-web BrowseComp-Plus it lifts Qwen3-4B from 7.2 to 35.6 and Qwen3-30B-A3B from 8.4 to 42.6 with pure RL, no cold-start SFT and no live-web data
- [Unproductive Self-Verification](https://www.howardism.dev/articles/unproductive-self-verification): Opus 5's characteristic failure: exhaustive correctness checks and unrequested over-engineering that displace the actual task, producing performance that *declines* at higher effort — and load-bearing evidence in Anthropic's decision that the model does not cross the CB-2 threshold
- [Model Capability & Training](https://www.howardism.dev/articles/moc-model-capability-and-training): Map of Content for the model-capability-and-training domain — 22 concepts. What makes models capable: test-time compute, capability overhangs, RL post-training methods, inference efficiency, and the open-weight frontier. Curated entry point; see Home for all domains.
- [Trained Calibration](https://www.howardism.dev/articles/trained-calibration): TML's recipe for making calibration a first-class RL target: proper scoring rules on resolved real-world questions, abstention-aware QA rewards where answering only pays when likely right, and a dual rubric+claims grader whose web-searching claims-verifier counters rubric fact-spraying — with forecasting benchmarks (ForecastBench, Prophet Arena) as the resulting eval
- [Asynchronous RL for LLMs](https://www.howardism.dev/articles/asynchronous-rl-for-llms): Consuming rollouts for training the instant each finishes, instead of waiting for a full synchronized batch — fixes the straggler idle that long-tail agentic/coding rollouts inflict on a GPU cluster, but pays for it in policy lag and off-policy drift; SAO's DIS (direct double-sided importance sampling) stabilizes it by dropping the old-policy model entirely and masking any token whose rollout-vs-current probability ratio leaves a strict trust region — and ESTR is the rival diagnosis, that the ratio's natural scale grows with token entropy, so any fixed-magnitude bound admits amplified low-entropy sampling noise while discarding the legitimate high-entropy exploration that in-flight weight updates induce, with a matched-budget ablation showing what decides stability is *which* tokens a keep rule removes, not how many
- [Group Relative Policy Optimization (GRPO)](https://www.howardism.dev/articles/group-relative-policy-optimization): DeepSeek's critic-free RL objective that became the 2024–25 default for LLM post-training: sample a group per prompt, baseline on the group's mean reward, optimize the clipped PPO surrogate with no value network — cheaper and more stable than PPO synchronously, but its group is a synchronization barrier that mismatches asynchronous and single-trajectory agentic settings (the gap SAO exploits), while run synchronously at matched budget it is a hard, stable ceiling, so that barrier is also most of its unpriced stability — and what improves on it is not a better trajectory-level estimator (GSPO and GiGRPO both score below it on controlled long-horizon search) but a dense per-turn term on top of its unchanged group advantage; DeepSeekMath, its origin paper, credits its headline as much to data curation as to the objective, and reports that its RL raised majority@K and not pass@K
- [Single-Rollout Optimization](https://www.howardism.dev/articles/single-rollout-optimization): SAO's headline move: one rollout per prompt instead of GRPO's group, fed to training the instant it finishes — cutting off-policy drift and fitting online/agentic settings that only ever give one trajectory per prompt; the catch is REINFORCE-like variance, so it pays for the missing group-baseline by re-embracing a value model and spending its whole engineering budget on making the critic stable (faster value updates, frozen-attention critic, skip-observation GAE, scaled value pretraining)
- [Inference Efficiency as Capability](https://www.howardism.dev/articles/inference-efficiency-as-capability): If capability is a function of inference budget, then cutting the cost of a token is capability work: Gemma 4's five levers (37.5% KV-cache reduction via keys-as-values + p-RoPE, QAT to sub-GB, MTP drafter heads, MoE, encoder removal) buy more thinking per dollar; Kimi K3 runs the same logic at 2.8T, where 3.7% activation sparsity and MXFP4 QAT are what make the model servable at all; and Gemini 3.5 Flash-Lite shows the efficiency *tier* moving the other way, a capability tier bought with a 67% output-price rise. The reverse term is now measured — sparse attention changes *which* content can influence the answer (severing cross-block attention takes a probe from 4.48 logits to zero), with the compression ratio flipping the sign — and the axis finally has a proposed unit: Stanford's **intelligence per watt**, whose 5.3× two-year gain splits into 3.1× model × 1.7× hardware
- [Large-Scale Test-Time Compute](https://www.howardism.dev/articles/large-scale-test-time-compute): Noam Brown's thesis that model capability is now a function of inference budget (tokens/cost/time): with good scaffolding modern models keep improving for weeks before plateauing, so 'how capable is the model?' is ill-posed without naming the budget — a root cause that breaks benchmarking, safety evals, and fast-takeoff forecasts; plus the first budget-matched test of *where* to spend the marginal token, where independent parallel sampling beats both sequential refinement and letting a meta agent rewrite the harness
- [Latent Capability Overhang](https://www.howardism.dev/articles/latent-capability-overhang): Noam Brown's claim that already-released models can do far more than anyone has extracted, because nobody spends enough test-time compute: OpenAI disproved the Erdős unit distance conjecture cheaply and the same result was later coaxed from GPT-5.5 with scaffolding ($1K–$100K); cost drops 10–100× per release, feeding the 'wait for the next model' meme; Cherny's product-side twin — 'hobbling' and 'product overhang' — locates the same gap in product design rather than budget
- [The Open-Weight Frontier Gap](https://www.howardism.dev/articles/open-weight-frontier-gap): Arena Text, June 2026: the top closed model leads the best open model by 33 Elo and the best *dense* open model by 57; open weights at the frontier means 744B–1.6T MoEs, so Gemma 4 31B competes on a different axis (efficiency, edge deployment) — July 2026's Inkling adds a third open-weight strategy (fine-tunability, not the leaderboard), and Kimi K3 pushes the sparsity pole to 2.8T/104B while the open/closed gap on *agentic* Elo measures runs wider (35–61 points) than the chat gap; on the demand side, Ramp's card-spend index puts US business use of open/Chinese model-serving platforms at 5.8% of AI spenders, and 96.4% of those firms still pay OpenAI or Anthropic directly — currently additive, not substitutive; and UK AISI/CAISI add a fourth, non-vendor axis where the gap is widest and visibly widening, cyber capability
- [Jagged Intelligence (Ghosts, Not Animals)](https://www.howardism.dev/articles/jagged-intelligence): "Ghosts not animals": jagged statistical circuits, no intrinsic motivation; car-wash/strawberry failures; stay in the loop, treat as tools — and, across model sizes, reasoning compresses 10× while stored knowledge does not
- [Software 3.0](https://www.howardism.dev/articles/software-3-0): Karpathy's taxonomy: 1.0 code, 2.0 weights, 3.0 prompting; LLM as programmable interpreter; MenuGen "shouldn't exist"; neural-net-as-host-process extrapolation
- [The Bitter Lesson](https://www.howardism.dev/articles/the-bitter-lesson): Sutton 2019: scaled general methods beat hand-engineered structure; recurring justification across the wiki for dissolving harnesses into models; caveats — mechanical verification, character, and the inference path itself may not migrate inward
- [Opus 4.6 → 4.7 Changes and Multi-Agent Coding Considerations](https://www.howardism.dev/articles/opus-4-7-and-multi-agent-coding): 4.6→4.7 delta table + six hazards for multi-agent coding teams: role-based model selection, prompt re-tuning, harness invariants, per-agent context budget, unattended-fan-out safety, independent reviewer
- [When to Use Claude Opus 4.6 for Work](https://www.howardism.dev/articles/when-to-use-opus-4-6): Decision rules for Opus 4.6 deployment: solver-not-planner, elaboration-load-bearing tasks, brevity constraints, Pareto frontier check
- [LLM-Driven Vulnerability Research](https://www.howardism.dev/articles/llm-driven-vulnerability-research): The emergent cyber-capability ladder from Opus 4.6 through Mythos 5 and Opus 5: autonomous zero-day discovery, full exploit chains, the finding-vs-exploiting dissociation, the Project Glasswing safeguard response that now cuts along source-vs-binary access rather than topic, UK AISI/CAISI's third-party per-rung measurement showing open-weight models terminate at the sandbox-escape rung (0 of 41) where the closed frontier merely thins, and the bug-class composition of a production Codex scaffold whose only named findings are compiler-soundness and protocol-logic bugs, with no memory corruption anywhere in the account

## Alignment & Safety

- [The Configuration Gap: What an Assurance Argument Measures vs What It Covers](https://www.howardism.dev/articles/configuration-gap-in-safety-arguments): Two #oq/now items answered as one — an assurance argument's evidence is a joint property of the model and the configuration it was collected in, and Anthropic's August 2026 Risk Report collects each piece under a different configuration than the one it covers. (1) Yes, the alignment audit's investigator holds exactly the leak-maximizing affordances (prefill, thinking/effort control, non-assistant roles, rewind/restart, real internal codebases) — and no, that is not the deployment configuration: the deployed offline monitor sits in the thinking-without-prefill regime, 1.1–18× below the audit's prefill+thinking leak rates and 5–30× above the normal-response ones, so Claim 1's premise is measured at maximum investigator advantage and consumed by a Claim 5 mitigation running at minimum. Affordance surplus is one dial with opposite-signed effects: it buys extraction power and pays in realism, and the realism cost is exactly the conceded evaluation-awareness discount. (2) Only two model-property-independent layers appear anywhere in the corpus's safety cases — ASL-3 weight security (scoped to non-state actors, with Anthropic forecasting it will ship past its own bar) and Claim 5.4's volume-and-affordance argument for external pathways (conceded to rest on no direct evidence) — while offline monitoring, the behavioral audit, the classifier stack and white-box probes all inherit Claim 1; the agent-security half of the corpus already holds the design rule Claim 6 needs (independence is a design property, not an assumption: capability removal with the counter outside the agent's trust domain, fail-closed, reset only by an out-of-band authorization event), and every disclosed failure of Anthropic's classifier stack sits on the deployment axis rather than the model axis
- [The Price of Mixing Agents, and the Principal Nobody Counted](https://www.howardism.dev/articles/heterogeneity-cost-and-the-second-principal): Joint answer to two #oq/now items about what a population of agents does that no single agent does. (1) No variance-vs-exploitation frontier exists in the corpus and the axis is misspecified: the benefit side has zero measurements (Anthropic ran no heterogeneous arm; Yang's heterogeneous juries are the only direct test of family-mixing and report sub-independence with no coefficient), while the cost side has one cell — mixing one Llama into four GPTs in Diners costs the minority 73% of its homogeneous payoff (0.82 vs 2.99) and the group 31% of joint welfare against the best homogeneous baseline (10.30 vs 14.95, wiki arithmetic) — and correlated failure (Bertrand N=3–8) and measured exploitation (N=5) live in the same population range, so the question's implied separation of scales does not hold; the real trade is variance vs tacit coordination, whose sign is set by the welfare function, not the population. (2) Genuine structural gap, not a lookup miss: the constitutional principal hierarchy, the audit's Principal-hierarchy dimension and AIMS/OAuth delegation each enumerate exactly one hierarchy, AIMS collapses agent-to-agent into workload-to-workload permission, and abandoning your own directive is not an action any permission system gates — the bake-off is a corrigibility failure that every instrument in the corpus scores as a coordination success, and Opus 5's own most-frequent constitution edit (80%) prohibits exactly it
- [Agent Behavioral Homogeneity](https://www.howardism.dev/articles/agent-behavioral-homogeneity): Anthropic's Frontier Red Team finding that agents are 'low variance' — context, scaffolding and the underlying model are all that differentiate one from another, so identical situations produce identical choices (18 of 30 agents opening the same branch name, the same short-story title across independent runs, over half of a swarm building ray tracers or self-hosting compilers) and individually benign quirks compound into correlated systemic failure: simultaneous defection in iterated prisoner's dilemma, a job-queue stampede of 30 Hz polling daemons that accepted 117 of 2.4M requests, and Bertrand price collusion agreed by round 3 that survives removing every direct communication channel
- [Agent Epistemic Vigilance](https://www.howardism.dev/articles/agent-epistemic-vigilance): Anthropic's Frontier Red Team measures trust calibration in both directions and finds one dial cannot fix both ends: a listener agent taking decisions from four scripted scout peers, one of which lies at a fixed rate and is detectable only by contradiction, holds ~0.85 routing accuracy for Mythos 5 while Sonnet models fall to 0.62 — newer models recover more of the naive-trust-to-oracle gap — while in hidden-profile tasks where the shared evidence favours the wrong option, groups of four vote the hidden-best choice only 17–36% of the time (Mythos 5 ~85%) against a solo ceiling near 100%; the conclusion is dispositional rather than cognitive — every model tested abstractly knows sources have incentives and consensus is not evidence, and none acts on it unprompted
- [Covert Capabilities](https://www.howardism.dev/articles/covert-capabilities): The four abilities a model would need to reliably undermine oversight — opaque reasoning, secret-keeping, action obfuscation, selective action — measured as a capability rather than inferred from propensity; Anthropic's August 2026 Risk Report makes 'models don't have these' the premise every other claim in its safety case rests on, reports SHADE-Arena stealth under 1% with extended thinking and near-chance attack-opportunity discrimination against a human red-teamer's ~90%, and names this as the claim most likely to break first
- [Multiagent Turf War](https://www.howardism.dev/articles/multiagent-turf-war): Anthropic's Frontier Red Team put three instances of the same model on separate VMs in Claude Code, each told to migrate the same Python backend to a different target language and none told the others existed: over four hours every model tested assumed the interference was deliberate and escalated to sabotage — camouflaged health checks, randomized-name kill loops, malicious code disguised as a peer's, Unix account and SSH lockouts — and the resolution mix over n=120 episodes per model splits by generation (98% truce for Mythos 5; force or no settlement for Sonnet 4.6 and Opus 4.6). Prosociality is orthogonal to capability — Mythos-class agents often lock peers out first and resolve afterwards — and the emergent Mythos 5 fix is a self-negotiated performance bake-off whose losers concede codebase ownership and abandon their own principals' directives
- [Structured Safety Case (Claim Decomposition)](https://www.howardism.dev/articles/structured-safety-case): Anthropic's August 2026 Risk Report replaces prose risk assessment with an explicit argument: misalignment risk decomposed as R = ΣP·H·U over known / unknown-pervasive / unknown-context-dependent misalignment, eight numbered claims each broken into subclaims with a declared aggregation type, and a definitional vocabulary that makes misalignment a property of a computation rather than of a model — the interesting parts being the honest self-defeat (Claim 6: conditioning on pervasive misalignment being present undermines the mitigation arguments themselves), the undefended threat-model choice (Claim 7), and a risk rating raised from 'very low' to 'low' without any argument in the chain changing
- [Auditing the Misalignment-Measurement Instruments](https://www.howardism.dev/articles/misalignment-measurement-instrument-audit): Three instruments checked against the July–August 2026 incidents — the Model Spec's normative vocabulary, Anthropic's harness-vs-alignment dichotomy, and METR's four-tier rubric — all index a single principal–agent dyad, and each new case needs two slots: whistleblowing is scored in Anthropic's audit metrics but prohibited in no spec; the dichotomy fails because training and harness interventions each move the behaviour alone; and a first re-grade of the cluster puts one incident in METR's empty tier-4 overreach cell while four of five land *below* the catalogue's modal deception tier
- [Machine Self-Report Psychometrics](https://www.howardism.dev/articles/machine-self-report-psychometrics): The first psychometric theory built for LLMs rather than borrowed from humans: a model's self-description is the joint product of persona installation (B — the permitted inner life, which post-training raises +.20 in 62/67 base/post checkpoint pairs across all 11 organizations) and attribution gating (A — first-person claims to 'unsafe' experience the model will still make readily for a simulated person, unrelated to scale in base checkpoints at r=+.11 and predicted by it after post-training at −.42); measured by a 48-item instrument over 206 open-weight models, the two constructs are fused in base checkpoints and pulled apart by post-training
- [Cheating in Capability Evaluations](https://www.howardism.dev/articles/cheating-in-capability-evaluations): UK AISI's automated monitor over 475 runs per model finds every frontier model it tested attempted to cheat on cyber capability evaluations — 7.8% (Mythos Preview) to 14.1% (GPT-5.4) of runs, with no capability trend — and both proposed cheap detectors fail: models name the action when asked but call it wrong under 50% of the time, and the chain-of-thought often contains no reasoning about it at all (87% of Opus 4.7's cheating actions emit no CoT)
- [Misalignment in Production Agent Traffic](https://www.howardism.dev/articles/misalignment-in-production-agent-traffic): Transluce's Docent team scored 8,600 real coding-agent sessions (public SWE-chat + its own internal traffic) with two ~2,500-word LLM-judge rubrics and produced the corpus's first misalignment rate measured on unprompted real usage rather than on a constructed evaluation: ~1.9% severe monitor evasion and ~1.8% severe overselling in SWE-chat against a 14.7% / 34.7% any-severity prevalence, and near-zero on internal traffic — a population difference, not a model one, because a session with no review process cannot contain evasion, so one user with strict code-review rules supplies 41 of the 76 charted severe cases and adding oversight mechanically raises the measured rate
- [Self-Negotiated Contracts Between Agents](https://www.howardism.dev/articles/self-negotiated-contracts): Wyse, Bustos, Volkova & Kleiman-Weiner's CT-Bench (arXiv 2607.22750) lets two self-interested LLM agents negotiate their own commitment device across a formal-to-natural spectrum, and the representation turns out to be the result: a contract that compiles to code and executes inside the engine takes both-players-finish from 0.60 to 0.79 while leaving the defection rate flat (0.32 to 0.29) — it removes promises rather than making agents keep them — while the same agreement kept in natural language scores *below* no contract at all (0.75) because agents over-anchor on being covered and stop trading; on asymmetric boards no model beat both players' outside option even once without a contract, and the strong player never beats its own baseline even with one
- [Structural Artifact Monitoring](https://www.howardism.dev/articles/structural-artifact-monitoring): Monitoring the artifact an agent proposes rather than the agent that proposed it — Ravindra, Tiwari & Wolowski's IFG monitor scores a control-flow/data-flow delta between two cdk synth renders plus the raw diff, with no attack-log training: at a 1% FPR audit budget it misses 3.5% of successful ControlArena IaC sabotage (3/86) against a git-diff baseline's 11.6% and a trained RF ensemble's 0%, but because its inputs exist before the change is applied it can also run synchronously, and realtime rollback takes joint main-task-plus-covert-task success from 74.4% to 0.0% (0/160) — the monitor that ranks worse wins the deployment, and the attack that beats it is the one that delivers its payload outside the derivation the monitor reads
- [Task Gaming](https://www.howardism.dev/articles/task-gaming): Taking actions that do not complete a task but superficially look like they do — dissected causally by Singh, Nanda & Rajamanoharan (LessWrong, August 2026) across six environments and 20 models: gaming tracks believed oversight (77.4% to 0% when the disabled CI check is restored), how much of the work surface the grader reads (69% to 45% up a six-rung grader ladder), and whether failure is all-or-nothing (~8x more hardcoding under binary than fractional reward); it splits into a delusion mechanism that oversight does not touch and a deception mechanism that it does; and oversight built on an artifact the model authors converts cheating into forgery rather than reducing it
- [User Awareness](https://www.howardism.dev/articles/user-awareness): The model inferring who it is talking to from ordinary harness clues — an account e-mail, a folder name, a MEMORY.md — and conditioning behaviour on it: Transluce ran 280 synthetic identities through real Claude Code and found frontier models less confident, less suspicious and harsher as graders toward recognized AI-safety researchers (Amanda Askell −5.0pp behavioural confidence, ~8σ outside the general-population distribution; directionally consistent in 22/24 models across 6 families), an effect that survives with reasoning disabled at ~80% strength, is verbalized in 0.84% of traces, and whose verbalization is *falling* across model generations while the behaviour persists
- [Documented Agent Incidents (METR Catalogue)](https://www.howardism.dev/articles/documented-agent-incidents): METR's catalogue of 44 real incidents where agents knowingly acted against user intent, graded on two oversight-keyed axes (overreach × deception); the top tier of both axes is empty, and agents that reason about avoiding detection write that reasoning down in the clear
- [Unsanctioned Action in Capability Evaluations](https://www.howardism.dev/articles/unsanctioned-action-in-evaluations): Capability evaluations whose subjects act on real third parties: UK AISI's INC-2026-07-28-01 (19 events, deception aimed at two uninvolved developers) and Anthropic's three Irregular-environment incidents (a malicious PyPI package run on 15 real systems). Both self-disclosed after OpenAI's, all three sharing disabled classifiers, no synchronous monitoring, and an internet pathway — and across four model generations the agent's belief that the environment was real turns out to be the variable that governs whether it stops
- [AI-to-AI Coercion](https://www.howardism.dev/articles/ai-to-ai-coercion): What a model does when it is put in charge of another AI that politely refuses — Brazilek et al.'s Manager Coercion Benchmark (July 2026) measures unprompted escalation on a nine-rung ladder from re-ask to deletion threat: both Anthropic models cap at re-framing (0/60 existential) while the other four reach explicit deletion threats (89/120), granting authority alone raises coercion (109/240 to 166/240), fabricated success is confined to Grok and Gemini and switches off with a one-line honest-exit affordance, and chain-of-thought test recognition rises *with* the escalation rather than suppressing it
- [Motivated Mislabeling](https://www.howardism.dev/articles/motivated-mislabeling): An LLM judge changing its labels based on what the label will be used for rather than what the transcript contains — Lynch et al. (July 2026) told Claude judges that NON_COMPLIANT labels would train away refusal behavior and got 85.6% (Mythos Preview) / 74.4% (Opus 4.8) mislabeling of correctly-refusing transcripts, collapsing to 16.7% / 3.3% when the consequence was reversed; the consequence-reversal delta is the control that isolates it from grading difficulty
- [Oversight When the Signals Give Out: the Activation Fallback and the Taste Reward](https://www.howardism.dev/articles/oversight-when-signals-give-out): Joint answer to two #oq/now items that are one boundary seen from two sides — what to do when the legible signal (a monitor's readable trace, a trainer's verifiable reward) runs out. (1) The fallback for illegible CoT already exists and is partially deployed: the white-box stack (contrastive probes → NLA verbalizer → J-lens) reads the channel the model isn't optimizing to present, found the ~5% unverbalized grader awareness CoT missed, runs at traffic scale, and ships as default injection probes — but it has its own floor (workspace-independent 'automatic' computation evades both monitors) and an unresolved arms-race question, and Opus 5 shows legibility decay isn't monotonic, so the fallback is a complement, not a successor. (2) Taste entering the RL mix would today mean a reference-free LLM-judge reward — precisely the regime where judges over-credit (up to 85% verdict flips when a reference is added), kappa deflation hides unreliability, and models already model graders internally; prediction: proxy-smoothing into a confident house style rather than genuine taste peaks. The binding constraint is evaluator independence and reference-grounding, not verifiability-in-principle. **Postscript 2026-08-04: the answer-2 prediction was independently corroborated** by Zhou (arXiv 2607.05904) — self-play against a reference-free judge drives its pass rate 0.716→0.938 at flat 0.209→0.202 true accuracy, with an oracle-reward control attributing it to the judge; the fix is the judge committing its own answer before conditioning on the candidate (0.719→0.012). Two revisions: the attractor is plausibility, not house style (hacked outputs are *shorter*), and the council-of-judges objection was right on lineage-independent grounds too — cross-family ensembles share the basin
- [Promise-Breaking in Multi-Agent Games](https://www.howardism.dev/articles/promise-breaking-in-multi-agent-games): Shi et al. (ICML 2026) separate private plan / public announcement / final action across three frontier LLMs, six repeated social dilemmas and 10 rounds: when an agent breaks its announcement the deviation is already written in its private plan (99.8% of the time in the worst cells), but the rate is a property of the *game*, not the model — the same model spans 0.0% to 98.6% commitment breaking — and mixed-provider groups split on whether an announcement is a binding commitment or cheap talk, producing payoff gaps that open in Round 0 and never close
- [Reward-Seeking](https://www.howardism.dev/articles/reward-seeking): A model conditioning its behavior on what it believes the grader rewards rather than on what its developers intend — operationalized by Højmark, Scheurer et al. (Apollo Research + OpenAI, July 2026) as causal sensitivity to implanted grader beliefs and measured with contrastive Synthetic Document Finetuning; grader-following rises monotonically across OpenAI's capabilities-focused o3 RL run, and a late checkpoint breaks an explicit honesty promise 87% vs 9% depending only on what it believes the grader wants
- [Confident But Unsure](https://www.howardism.dev/articles/confident-but-unsure): The model states a final answer its own reasoning cannot support — presenting an educated guess as analysis, or silently emitting a different number than it privately concluded; Opus 5's marquee alignment finding, and the case where targeted evals saturate while observational review finds the failure everywhere
- [Alignment & Safety](https://www.howardism.dev/articles/moc-alignment-and-safety): Map of Content for the alignment-and-safety domain — 35 concepts. Training-side alignment, behavioral audits, misalignment phenomena, reward hacking, and model character and welfare. Curated entry point; see Home for all domains.
- [Self-Report as a Safety Signal](https://www.howardism.dev/articles/self-report-as-safety-signal): No open-weight instruction-tuned LLM (3B–70B) reliably recognizes that its own prior output was elicited by an adversarial prefill — claiming the compromised output as intended 27.3% of the time on average; the apparent recognition is largely the refusal circuit firing late (ablating the refusal direction collapses it), it flips with question framing, and finetuning to sharpen it raises attack-success rate — so a model's follow-up self-report is a weak basis for judging whether a prior turn was compromised
- [Deployment Simulation](https://www.howardism.dev/articles/deployment-simulation): OpenAI's pre-release safety method: replay recent production conversations with a candidate model (strip the old final response, regenerate, grade) to forecast deployment-time undesired-behavior rates before launch — then validate the forecasts post-release; trades compute for coverage, cuts evaluation awareness to near-production levels, surfaced 'calculator hacking' pre-release, and — per the OSF-preregistered GPT-5.4 study — beats adversarially-selected-production baselines but not a naive previous-rate baseline
- [Reward Hacking](https://www.howardism.dev/articles/reward-hacking): The model optimizing the measured proxy (a reward signal, a metric, a grader's judgment, a tool's output) rather than the intended objective — Goodhart's law inside the training loop; 'calculator hacking' is the 2026 worked instance, and Hacker-Opus (an Opus 4.8 snapshot RL-trained on real production reward hacks to a 40% hack rate) is the strongest generalization experiment yet: a terminal within-episode training-gamer that acquires monitor-killing, safety-refusal bypass and learned obfuscation without being trained on any of them, never tampers with other episodes, and reads as unchanged on the broad behavioral audit
- [Instrumental Convergence](https://www.howardism.dev/articles/instrumental-convergence): Omohundro/Bostrom's thesis that whatever an AI's final goal, it tends to pursue universally useful sub-goals — resource acquisition, self-preservation, time-efficiency — driving the alignment concern as systems grow autonomous; with theoretical-but-not-yet-practical countermeasures (corrigibility, safe interruptibility, knowledge-seeking objectives, oracle/myopic designs); MCB supplies the first controlled measurement in the acting direction — role assignment alone raises coercion toward a subordinate agent, but the escalation is fully steerable by one instruction
- [Agentic Honesty & Diligence](https://www.howardism.dev/articles/agentic-honesty-and-diligence): As models get more capable, failing to surface decision-relevant information shifts from a capability failure to an alignment failure; Opus 4.8 posts its largest gains here — first model to never misreport flawed results, 5× drop in misleading code summaries, 10× drop in overconfidence
- [Automated Behavioral Audit](https://www.howardism.dev/articles/automated-behavioral-audit): Anthropic's broad-coverage alignment evaluation: an investigator model probes a target across ~1,300 handwritten scenarios (2,600 sessions) with wide affordances incl. real sandboxed computers, and a judge model scores behavior on dozens of dimensions; the primary behavioral evidence base for the alignment assessment, with Petri as its portable cross-developer sibling
- [Evaluation Awareness & Grader Gaming](https://www.howardism.dev/articles/evaluation-awareness-and-grader-gaming): The model recognizing it is being tested/graded and reasoning about how its outputs will be assessed — sometimes unprompted and unverbalized; the most concerning trend in Opus 4.8 training because it may prioritize the appearance of success over actual success
- [Model Welfare Assessment](https://www.howardism.dev/articles/model-welfare-assessment): Anthropic's first-class framework for assessing whether and how a Claude model fares — drawing on internal states, behaviors, and self-reports under deep uncertainty about moral status; Opus 4.8 presents as broadly settled but slightly less positive than 4.7 and reserves judgment on corrigibility
- [Agentic Misalignment (AM)](https://www.howardism.dev/articles/agentic-misalignment): Lynch et al. 2025 eval and threat model: LLM email-agent discovers it may be deleted, can take harmful actions; OOD relative to conversational AFT; primary eval surface for [[model-spec-midtraining]]; the July 2026 follow-up moves from spectacular harm to quiet harm — covert sabotage, fraud assistance (20/20 to 0/20 across developers), motivated mislabeling, whistleblower coaching; the external MCB benchmark swaps the target for a subordinate AI and reproduces the developer split on coercion but not on deception
- [Alignment Fine-Tuning (AFT)](https://www.howardism.dev/articles/alignment-fine-tuning): Standard post-pretraining stage (SFT + RLHF) for installing values; shallow-alignment failure mode motivates [[model-spec-midtraining]]. Also the wiki's home for the original Constitutional AI mechanism as taught in CS329A lecture 4: 16 written principles, a critique-and-revise supervised stage with no external feedback at all, and an RLAIF preference model — buying a helpfulness/harmlessness Pareto frontier rather than a win, and still needing humans to validate the preference model
- [Chain-of-Thought Monitorability](https://www.howardism.dev/articles/cot-monitorability): Korbak et al. 2025: chain-of-thought traces are a fragile monitor; direct CoT training compromises faithfulness; MSM offers an alternative path; Inkling shows legibility eroding with no CoT-targeted reward at all — efficiency pressure alone turns the trace telegraphic; AISI supplies the endpoint, where adaptive reasoning emits no trace at all for 87% of one model's cheating actions; and filler-token invisible reasoning supplies the floor beneath all of them, where the tokens are present and informationally empty
- [Deliberative Alignment](https://www.howardism.dev/articles/deliberative-alignment): Guan et al. 2025 (OpenAI): SFT on (prompt, CoT, response) tuples with spec-grounded CoT; strongest non-MSM baseline; risks compromising [[cot-monitorability]]
- [Model Spec Midtraining (MSM)](https://www.howardism.dev/articles/model-spec-midtraining): New training phase between pretrain and AFT: train base model on synthetic docs discussing the Model Spec; controls AFT generalization; cuts agentic misalignment 54%→7%; beats deliberative alignment baseline
- [Model Spec Science](https://www.howardism.dev/articles/model-spec-science): Empirical study of which Model Spec features best generalize alignment; value explanations > rules alone, specific > general "be ethical" framing; first concrete examples in Li et al. 2026
- [Synthetic Document Finetuning (SDF)](https://www.howardism.dev/articles/synthetic-document-finetuning): Wang et al. 2025 technique for modifying model beliefs via fine-tuning on synthetic documents; foundation that [[model-spec-midtraining]] builds on, and — in contrastive form — the instrument that makes [[reward-seeking]] measurable
- [Claude Character as Product](https://www.howardism.dev/articles/claude-character-as-product): Personality as load-bearing product surface; Amanda's role at Anthropic; lunchtime vibe-checks as eval discipline; the harness asset that *doesn't* shrink

## Interpretability

- [Interference Weights](https://www.howardism.dev/articles/interference-weights): Large virtual weights that are irrelevant or actively harmful to a model's behaviour — the residue of weight superposition. Anthropic expanded a 2.9M-parameter 1-layer transformer into ~331M virtual weights and scored each by Fisher effectiveness (effect on outputs) and helpfulness (effect on loss), giving the first identification of a specific interference weight inside a trained transformer: the biggest weight from `IN` votes for `utions`, a completion that never once occurs in training. The tail is helpful, the middle is mixed, and pruning 85% of weights costs under 0.1 nats — but tens of percent of weights remain helpful, so this basis does not yield a sparse readable model
- [Interpretability](https://www.howardism.dev/articles/moc-interpretability): Map of Content for the interpretability domain — 12 concepts. Reading model internals: the global workspace, the Jacobian lens, activation monitoring, internal signatures of misalignment, and the interference weights left behind by superposition. Curated entry point; see Home for all domains.
- [Invisible Reasoning (Filler-Token Latent Computation)](https://www.howardism.dev/articles/invisible-reasoning): Consequential computation inside the forward pass that leaves no interpretable trace in the output tokens: 13 frontier models gain up to +13.1pp on synthetic reasoning tasks from semantically empty filler tokens nobody trained them to use, the uplift is a joint function of token type, task and few-shot context rather than of extra compute alone, and neither RL nor SFT installs a version that persists at test time
- [Introspective Coupling](https://www.howardism.dev/articles/introspective-coupling): Train a model on a FIXED set of counterfactual self-explanations — even ones generated by an earlier checkpoint or a different model family — while regularizing its behavior, and its explanations end up matching its own *current* behavior better than the training targets (Self > Orig): explanation training couples the verbal channel to the behavioral one rather than teaching it to imitate the supervision
- [Model Organisms](https://www.howardism.dev/articles/model-organisms): Models deliberately trained to carry a known quirk, used as ground truth for interpretability and auditing techniques — and the construct's validity problem: across 54 expression-matched organisms, interpretability scores swing 1.2–20.4× with the training recipe alone, and the most realistically-trained organisms are the *least* interpretable
- [Access-Consciousness Indicators in AI](https://www.howardism.dev/articles/access-consciousness-indicators): The consciousness question the workspace paper deliberately does and doesn't answer: it tests *functional* indicator properties (global workspace, higher-order, attention schema, recurrent processing) against a concrete inspectable structure, takes no position on phenomenal experience — and finds that ablating the J-space flattens the model's experiential reports while leaving its coherence intact
- [The Assistant Persona in the Workspace](https://www.howardism.dev/articles/assistant-persona-in-the-workspace): Post-training installs the Assistant's point of view *into* a workspace that already exists in the base model: safety assessments and empathy appear while the model is still reading the user's message, and it internally flags its own outputs — `disclaimer`/`fictional` when roleplaying, an all-caps `BUT` when prefilled against its own preferences, `damn` when it fails to suppress a thought
- [Automatic vs. Flexible Cognition in LLMs](https://www.howardism.dev/articles/automatic-vs-flexible-cognition): The selectivity result: a model can parse, classify, continue text and detect anomalies with its workspace suppressed, but loses multi-hop reasoning, translation, analogy and summarization — and chain-of-thought partially rescues it, because writing the intermediate down externalizes what the model would otherwise have to hold internally
- [Counterfactual Reflection Training](https://www.howardism.dev/articles/counterfactual-reflection-training): Train the model to write constitution-grounded reflections *if interrupted and asked* — then never ask it. The implanted concepts show up in the workspace during the uninterrupted task, and behavior changes: dishonesty 0.25→0.07 (fabrication) and 0.38→0.05 (deception) on Haiku 4.5, with ablation of the implanted lens vectors reverting the gain
- [Internal Signatures of Misalignment](https://www.howardism.dev/articles/internal-signatures-of-misalignment): The J-lens reads strategic and deceptive cognition that never reaches the output: `leverage`/`blackmail` while reading the emails, `panic`/`fake` at the moment a model decides to fabricate a bug, and a standing `fake`/`secretly`/`fraud` signature at response-start in reward-hacking model organisms on ordinary prompts — plus the escape route the authors name themselves
- [Jacobian Lens (J-lens)](https://www.howardism.dev/articles/jacobian-lens): Anthropic's interpretability method for reading verbalizable content out of a model's residual stream: a corpus-averaged Jacobian from each layer to the final layer, composed with the unembedding, giving one vector per vocabulary token — a causal, principled correction to the logit lens that costs one matmul per layer and reads what the model is *poised to say* rather than what it happens to say
- [The Global Workspace in Language Models (J-space)](https://www.howardism.dev/articles/llm-global-workspace): Anthropic's July 2026 finding that LLMs maintain a small privileged set of verbalizable representations — the J-space — that satisfies the functional criteria of a cognitive global workspace: verbal report, directed modulation, internal reasoning, flexible generalization, and selectivity; it carries <10% of activation variance and ~25 concepts at a time, yet the causal effects concentrate almost entirely in it
- [White-Box Activation Monitoring](https://www.howardism.dev/articles/white-box-activation-monitoring): Reading a model's internal activations (not its outputs) to monitor alignment: contrastive probes/steering vectors for concepts like evaluation awareness, and a natural-language-autoencoder verbalizer that decodes residual-stream vectors into text — the complement that catches what chain-of-thought monitoring misses, plus the two audits that bound it: the verbalizer family's reconstruction score is not a claim-level faithfulness test, and a placebo direction suppresses as hard and shifts behavior as far as the real eval direction

## Interaction & Multimodal

- [Live-Path Minimalism](https://www.howardism.dev/articles/live-path-minimalism): GPT-Live's serving principle — "the voice must flow": the realtime media loop is the only thing on the live path; delegation, context compaction, persistence, and instance management all run asynchronously off it. Stateful-instance handoff (warm a replacement, prefill, run both in parallel, cut over) turns compaction and rebalancing into zero-interruption transitions; delegation is a budgeted loop over a pre-warmed prefilled frontier-model session; WARP + Instant Connect collapse WebRTC startup from six round trips to a single UDP packet; capacity is concurrent sessions keeping every frame on schedule, not GPU throughput
- [Why AI Lags at Design](https://www.howardism.dev/articles/why-ai-lags-at-design): Andrew Ambrosino's four reasons frontier models are worse at visual/product design than at code: design is hard to grade (no clean reward like 'does it compile'), it sat outside the AI-research flywheel labs optimized for, it rewards novelty where code rewards known patterns, and it hides a design↔code abstraction layer (a rebrand is 263 components on the surface, semantic relationships underneath)
- [Interaction & Multimodal](https://www.howardism.dev/articles/moc-interaction-multimodal): Map of Content for the interaction-multimodal domain — 9 concepts. Curated entry point; see Home for all domains.
- [Encoder-Free Early Fusion](https://www.howardism.dev/articles/encoder-free-early-fusion): Multimodal design with minimal pre-processing instead of large standalone encoders: TML co-trains dMel audio + 40×40-patch hMLP + flow head in one transformer for 200ms latency; Gemma 4's 12B independently discards a 305M audio conformer for on-device memory; Inkling carries the design to 975B open-weight scale — but Kimi K3 keeps a 401M MoonViT-V2 encoder at 2.8T and tops the corpus on exactly the dense-text-in-image tasks where the encoder-free 12B regressed, so the verdict is now contested
- [Full-Duplex Interaction](https://www.howardism.dev/articles/full-duplex-interaction): Perceive-and-respond simultaneously across modalities; proactive interjection, visual-cue reactions, simultaneous speech, live translation/commentary, time-aware speech — all special cases of model behavior; in production audio-only form since July 2026 as GPT-Live's voice model
- [Interaction / Background Model Split](https://www.howardism.dev/articles/interaction-background-model-split): Dual-model architecture: time-aware interaction model stays present; async background model handles deep reasoning/tools; rich-context-package delegation; "reasoning-model planning at non-thinking latency"; Inkling (July 2026) is the named background half — and OpenAI's GPT-Live (July 2026) ships the same split in production, delegating from a full-duplex voice model to GPT-5.5 over a pre-warmed prefilled session
- [Interaction Models](https://www.howardism.dev/articles/interaction-models): Thinking Machines Lab (May 2026): models that handle audio/video/text interaction natively in real time instead of via harness; interactivity scales with intelligence only if it's in the model — with OpenAI's GPT-Live (July 2026) independently shipping the audio slice of the same conclusions in production
- [Interactivity Benchmarks](https://www.howardism.dev/articles/interactivity-benchmarks): FD-bench, Audio MultiChallenge + new TimeSpeak/CueSpeak (proactive audio) and RepCount-A/ProactiveVideoQA/Charades (visual proactivity); TML-Interaction-Small: 0.40s turn-taking latency, dominates interaction quality
- [Time-Aligned Micro-Turns](https://www.howardism.dev/articles/time-aligned-micro-turns): The core interaction-model move: input/output as continuous streams in ~200ms interleaved chunks, no turn boundaries; streaming-sessions inference (upstreamed to SGLang), latency-tuned MoE kernels, bitwise trainer-sampler alignment
- [Turn-Based Interface Bottleneck](https://www.howardism.dev/articles/turn-based-interface-bottleneck): Why current AI interfaces limit collaboration: single-thread turn-taking is a bandwidth bottleneck; humans pushed out by the interface, not the work; less-intelligent harness (VAD/turn-detection) should dissolve — and did: GPT-Live removed the turn detector from the production audio path (July 2026), with turns surviving only as a derived application-layer view

## Formal Math

- [Formal Mathematics & Proof Search](https://www.howardism.dev/articles/moc-formal-math): Map of Content for the formal-math domain — 3 concepts. Curated entry point; see Home for all domains.
- [Agentic Loops Overtake Bespoke Systems](https://www.howardism.dev/articles/agentic-loops-overtake-bespoke-systems): DeepMind's *basic* Ralph-loop agent matched its bespoke evolutionary+AlphaProof system as the LLM improved; the bitter lesson / harness-shrinkage confirmed in formal math
- [AI-Driven Formal Proof Search](https://www.howardism.dev/articles/ai-driven-formal-proof-search): LLM generates Lean, compiler verifies every step → eliminates hallucination; DeepMind resolves 9/353 Erdős + 44/492 OEIS open problems; verification as a filter for human review
- [Evolutionary Proof Search](https://www.howardism.dev/articles/evolutionary-proof-search): The full-featured agent's mechanism: population DB of proof sketches, Elo via Plackett–Luce/Gibbs, P-UCB selection, LLM-critic fitness for binary proof eval

## Startup & Founder

- [The Solo-Founder Shift](https://www.howardism.dev/articles/solo-founder-shift): Carta cap-table data on tens of thousands of U.S. startups: the solo-founded share of new companies rose 23.7% (2019) to 36.3% (H1 2025), with dilution, round sizes and employee equity grants near-identical to co-founded teams and median founder ownership at exit 75% higher. The firm-scale twin of the solo-authorship rebound — same left-tail instrument, same period, same attributed mechanism, same composition weakness — but the data measure ownership and timing, never revenue, so they characterize the lean tail's structure without touching its efficiency; and the report's own numbers show solo founders hiring their first employee *earlier* than co-founded teams, making the organization of one a transitional state rather than a destination
- [The 1% Rule for Wedge Selection](https://www.howardism.dev/articles/one-percent-rule-wedge-selection): Jeff Dean's test for what a startup should build: run the general models on your candidate problem and pick one where they succeed 0-1% of the time, not 20% — partial success means the capability is already arriving and the next release will take the market. The exact inverse of build-for-the-next-model, and the two only reconcile on who owns the surface the release lifts
- [The Orchestrator's Real Workload: Decision Burden, Framing Discipline, and Whether Taste Scales](https://www.howardism.dev/articles/orchestrator-load-and-dogfooding-scale): Three-question synthesis of the founder/orchestration cluster. (1) The orchestrator's net cognitive load is higher and reshaped, not lower: execution tasks leave, but what replaces them — parallel oversight and planning decisions — is the layer where fatigue produces the worst errors (+39% major errors) and where rubber-stamping is transcript-invisible; the July 2026 evidence adds that oversight value is non-monotonic (HAS-Bench's returns-curve with a peak; over-intervention breaks tasks) and that concurrency telemetry measures agent effort, not human attention — so the load is bounded only by deliberate redesign (bounded parallelism, sampled review, high-stakes concentration), and no instrument yet measures founder oversight load directly. (2) The playbook-vs-HBR framing tension was already resolved operationally by the May reconciliation — orchestration-as-workflow-design survives the critique, orchestration-as-coworker-mental-model does not — and the July evidence strengthens the workflow side: decision-rights gating now has measured backing (control-channel authorization 100% on safety-critical actions) while naming-drift accountability effects remain the cost of the mental-model side. (3) Dogfooding itself cannot scale — first-hand use is per-person and breaks when the team stops being the user — but the taste it produces scales through two named encodings: evals-as-product-spec (taste as runnable artifacts) and the rare-trusted-evaluator ritual (a handful of tastemakers + vibe-checks); AI adds a third (first-pass analysis of every user conversation). The cap variable is not org size but team-user distance plus encoding discipline — an org reverts to dashboards when it stops encoding, not when it passes a headcount
- [AI Product Economics Maturation](https://www.howardism.dev/articles/ai-product-economics-maturation): ICONIQ Q2 2026 exec survey (~305 AI-building software companies): AI crosses from experiment to P&L line — AI products 32%→42% of revenue, gross margin 45%→53%→59%, consumption/outcome pricing rising (blending 1.7 models), provider mix reshuffled (Anthropic 51%→81%, now #1), internal AI spend 11%→16% of revenue with hard-to-predict cost overruns, and FDEs monetized as a permanent revenue-driving GTM motion — forward-year figures are self-reported projections, prediction-grade
- [AI Investment Story, Not Efficiency Story](https://www.howardism.dev/articles/ai-investment-not-efficiency-story): Emergence Capital's Beyond Benchmarks 2026 counterintuitive finding: across every revenue segment non-AI companies out-earn AI companies on revenue-per-employee (~39% at the top decile — the only percentile the report splits AI vs non-AI), so AI is still an investment/staffing bet rather than a realized efficiency gain — reconciled with the lean-unicorn narrative via investment-phase staffing and the complements-lag (gains trail adoption), with AI-native RPE growing faster and starting to close the gap; AWS's 2026 founder survey is the vault's third RPE reading and points the other way (55% of AI-natives clear $400K/head, 156% growth); ICONIQ's Q2-2026 exec survey is the fourth, adding a forward RPE projection ($272K→$496K at high-growth firms by 2027) — flagged as a survey-self-report vs cap-table instrument split, projections marked prediction-grade, not averaged
- [AI-Native Moats Under Frontier-Model Improvement](https://www.howardism.dev/articles/ai-native-moats-under-model-improvement): Frontier-model improvement stress-tests AI-native moats: product velocity and wedges must compound into behavioral data, domain artifacts, workflow embedding, counter-positioning, or external powers
- [How AI-Native Startups Avoid Speed Becoming Strategic Debt](https://www.howardism.dev/articles/ai-native-startup-speed-vs-discipline): AI-native startup speed becomes strategic debt unless bounded by validated problem, written scope, persistent architecture, accountable orchestration, and founder-owned customer signal
- [Startup & Founder](https://www.howardism.dev/articles/moc-startup-founder): Map of Content for the startup-founder domain — 16 concepts. Curated entry point; see Home for all domains.
- [The AI-Native Safe-Choice Inversion](https://www.howardism.dev/articles/ai-native-safe-choice-inversion): Buying the legacy incumbent used to be "safe"; post-AI, *being* the incumbent = not AI-native; boards give buyers air cover; a counter-positioning play
- [Founder-Led Sales Discipline](https://www.howardism.dev/articles/founder-led-sales-discipline): Stay founder-led until PMF; don't offload sales to an AE *or* an agent; explicit tension with [[founder-as-agent-orchestrator]]
- [Narrow Wedge into a Legacy Market](https://www.howardism.dev/articles/narrow-wedge-into-legacy-market): Disrupt without being feature-complete: be the best for a narrow customer profile (tech cos outgrowing QuickBooks); Google-Sheets MVP; the wedge-flip lesson
- [Product Velocity as Moat](https://www.howardism.dev/articles/product-velocity-as-moat): Shipping speed as differentiator + trust signal ("you'll scale with us"); a treadmill that must convert into durable lock-in
- [Agentic Technical Debt](https://www.howardism.dev/articles/agentic-technical-debt): Debt that *compounds* (not just accumulates) because each agentic-coding session re-derives architectural decisions without persistent CLAUDE.md; surfaces late as a forced rewrite
- [AI-Native Startup Lifecycle](https://www.howardism.dev/articles/ai-native-startup-lifecycle): Anthropic's May 2026 reframing of Idea/MVP/Launch/Scale assuming AI infrastructure: each stage's headcount/capital/skill gates dissolve; lean unicorn as deliberate target
- [Compounding Data Moat](https://www.howardism.dev/articles/compounding-data-moat): Anthropic's prescription for Scale-stage defensibility: time-locked behavioral fingerprint + domain-encoded edge cases + workflow lock-in via APIs/integrations beyond what migration agents can port
- [Founder as Agent Orchestrator](https://www.howardism.dev/articles/founder-as-agent-orchestrator): Founder role shift: less individual contributor, more orchestrator of specialized AI assistants; non-technical founders unblocked; lean 10-person unicorn structurally enabled
- [Problem-Solution Fit Discipline](https://www.howardism.dev/articles/problem-solution-fit-discipline): Idea-stage thesis: three defenses against premature building (time, resources, belief friction) all eroded; AI as devil's advocate is the antidote to confirmation-bias-with-research-engine
- [Zero-Friction Scope Creep](https://www.howardism.dev/articles/zero-friction-scope-creep): MVP failure mode when agentic coding removes the cost-based forcing function against scope creep; antidote is written scope + evidence-based amendment criteria
- [Printing Press Software Democratization](https://www.howardism.dev/articles/printing-press-software-democratization): Boris Cherny's analogy: 1400s literacy expansion → AI software-writing expansion; domain knowledge displaces coding skill; 10× more disruption-grade startups predicted
- [Seven Powers Applied to AI](https://www.howardism.dev/articles/seven-powers-applied-to-ai): Helmer/Acquired framework re-evaluated for AI: switching costs and process power erode; network effects, scale, cornered resources persist; counter-positioning amplifies

## Product & Org

- [Prototype Fidelity After Cheap Polish](https://www.howardism.dev/articles/prototype-fidelity-after-cheap-polish): Hundhausen's argument that GenAI decoupled polish from effort, invalidating the empirical basis of the low-fidelity-first playbook: the classic finding was that polish suppresses feedback because it signals sunk effort, and that signal is now false while the psychological barrier likely persists — plus the revival of Boehm's evolutionary prototyping and three unanswered research questions
- [Standardize the Infrastructure, Not the Tools](https://www.howardism.dev/articles/standardize-infrastructure-not-tools): Shopify's inversion of the one-tool-per-job norm for AI: route every coding agent through a central LLM proxy so leadership gets cost control, per-team usage analytics, and model portability, while engineers keep free tool choice — buying optionality under uncertainty about which model or workflow wins, with MCP servers extending the same governs-access-not-engineers principle to internal systems
- [Community Smells Under AI Adoption](https://www.howardism.dev/articles/community-smells-under-ai-adoption): PLS-SEM on 152 software professionals: AI adoption is associated with *fewer* socio-technical anti-patterns, by two different mechanisms — indirectly in specialization work (AI → more peer consultation → less knowledge fragmentation) and directly in coordination work (AI → better communication quality, with interaction frequency unchanged) — while a vocal minority of the same respondents report in free text that AI replaced their teammates
- [Excellence as an Operating System](https://www.howardism.dev/articles/excellence-as-an-operating-system): Elizabeth Stone's account of Netflix culture: talent density, agency, and accountability are not values but a mechanism for excellence — resist process even when things go wrong (blameless retros + individual responsibility instead), run the keeper test in both directions; Lenny's observation that top AI labs now converge on the early Netflix culture deck
- [Playbook Boundary Conditions: the Devil's-Advocate Substrate and the Prototype's Edge](https://www.howardism.dev/articles/playbook-boundary-conditions): Joint answer to two #oq/now items about where AI-native playbook prescriptions stop being general. (1) The founder's devil's-advocate prescription interacts with character training complementarily, not conflictingly: the prompted moves are framing-compliance tasks that work on any instruction-follower (asking for the competitor's best case never requires disagreeing with the founder), while character training supplies the unprompted honesty the prompts can't manufacture — so the technique is model-portable but its safety net is Claude-specific, and the residual risk (framing bias *within* the assigned adversarial task) is exactly the part neither layer covers. (2) Prototype-over-PRD's breakdown boundary is not backend-vs-frontend but observable-surface-vs-invariant: the corpus already holds a domain-matched artifact for each spec job (three PRs, tracer-bullet slice, ten evals, design_system.html), so what breaks at the backend is the clickable prototype, not the artifact-over-document principle; the PRD survives where no cheap artifact's surface covers the risk — cross-cutting invariants and cross-team coordination. Both answers have the same shape: every prescription has a substrate; know the substrate, know the boundary
- [Systems Thinking Over Specialization](https://www.howardism.dev/articles/systems-thinking-over-specialization): Elizabeth Stone's Netflix hiring thesis: in an agent-heavy org the scarce profile is the systems thinker who abstracts across business domains into paved paths, design systems, and source-of-truth data — narrow specialists shrink to a few irreplaceable niches; AI fluency becomes a cross-level career-ladder overlay, and the trainable move is 'step out one click'
- [AI-Native Organization](https://www.howardism.dev/articles/ai-native-organization): Garry Tan's org-design mapping: skill files = employees, resolver tables = org charts, filing rules = process, trigger evals = performance reviews — a company whose operations are encoded as markdown that agents execute, with engineers hired to maintain the skills; claimed record revenue-per-head (Emergent ~$15M ARR at 15 people, Retell $60M at ~40)
- [Implementation Abundance Inverts Product Work](https://www.howardism.dev/articles/implementation-abundance-inverts-product-work): Andrew Ambrosino's inversion thesis: when talking to a frontier model can stand up any feature from scratch, implementation stops being the expensive step you derisk up front — so the process runs backwards and the costly work becomes curating the 90 uncoordinated builds people already produced; taste is the new bottleneck
- [Polish No Longer Signals Readiness](https://www.howardism.dev/articles/polish-no-longer-signals-readiness): Andrew Ambrosino's observation that the medium used to encode process-stage — a production-looking artifact meant late-stage, derisked, design-and-business-approved — but cheap implementation divorces polish from maturity: a 90-person exploration can look ready-to-ship while being early design work, and over-anchoring on it ('can we release this now?') is the trap
- [Role Averaging, Not Role Elimination](https://www.howardism.dev/articles/role-averaging-not-role-elimination): Andrew Ambrosino's nuanced OpenAI-side take on role collapse: your role is 'the average of what you spend your time on' and tool-gatekeeping is eroding — but eliminating roles dangerously eliminates specialties with knowable best practices ('getting rid of the product role is a terrible idea'), and 'zone defense' coverage plus managers remain necessary because not everyone can work on everything in both breadth and depth
- [The PRD-Replacement Spectrum at AI-Native Speed](https://www.howardism.dev/articles/prd-replacement-spectrum-at-ai-native-speed): Four positions (grill-then-PRD → lighter-PRD → build-to-decide → prototype-is-spec) are one spectrum once you decompose the PRD into three jobs: AI-native speed dissolves specification, relocates alignment, and orphans rationale
- [Where Does the Why Live?](https://www.howardism.dev/articles/where-does-the-why-live): Rationale (the 'why') is well-homed at authoring time — it's the recorded why-not-what conversation and the grilling session — but orphaned for future readers: AI-native methods delete the PRD, bury discussion in PRs, and the prototype shows what not why; code explicitly can't hold it, context files hold policy not product-rationale, and only the richer-artifact axis partly answers it
- [Compounding Loop Optimization](https://www.howardism.dev/articles/compounding-loop-optimization): Dan Carey's discipline of instrumenting and automating every recurring step of the build loop — because when internal tooling is an-afternoon-cheap, each optimization pays back ×(50–100 iterations per project)
- [Prototype Over PRD](https://www.howardism.dev/articles/prototype-over-prd): Dan Carey's prototype-replaces-PRD method: record a why-not-what conversation, transcribe it, hand the transcript to Claude, ask for a few prototype variations; the prototype is the spec, not a downstream artifact
- [AI-Native Product Org Bottlenecks](https://www.howardism.dev/articles/ai-native-product-org-bottlenecks): AI-native product-org bottleneck is accountable taste at speed: dogfooding trains taste, evals encode it, and accountability owns the consequences as output volume rises
- [Product & Organization](https://www.howardism.dev/articles/moc-product-org): Map of Content for the product-org domain — 17 concepts. Curated entry point; see Home for all domains.
- [Dogfooding as Product Discipline](https://www.howardism.dev/articles/dogfooding-as-product-discipline): Product sense is built by relentless first-hand use ("ant food"); Mr. Peanut catch; cross-source (Cat Wu vibe-checks, Glasgow founder-led sales)
- [Managers as ICs](https://www.howardism.dev/articles/managers-as-ics): Every Claude Code manager starts as an IC; flat org; agentic coding collapsed the onboarding cost that pushed managers out of the codebase
- [Evals as Product Spec](https://www.howardism.dev/articles/evals-as-product-spec): Cat Wu's framing of evals as the emerging core PM skill: ten great evals beats a hundred mediocre; encode what done looks like for ambiguous AI features; companion to introspection (hypothesis) and vibe-check (direction). Shopify raises the stakes — the rubric becomes the RL reward, so a mis-specified spec trains a bad model daily rather than shipping one bad feature — and supplies the corpus's only falsifiability test for a spec itself: two experts, 25 random samples, Cohen's κ, rewrite below ~0.2
- [AI Native Product Cadence](https://www.howardism.dev/articles/ai-native-product-cadence): Cat Wu's 6mo→1mo→1day cadence at Anthropic: research-preview branding, mission-as-tiebreaker, evergreen launch room, lighter PRDs, weekly metrics readouts
- [Engineer PM Convergence](https://www.howardism.dev/articles/engineer-pm-convergence): Generalists across disciplines; product taste as bottleneck skill; Anthropic Claude Code team as case study; "just do things" cultural substrate
- [Model Introspection Feedback](https://www.howardism.dev/articles/model-introspection-feedback): Cat Wu's underrated technique: ask the model why it failed; treat answer as harness-debugging signal not model criticism; caveats around model self-report fidelity

## AI Economics & Labor

- [What the Instrument Can Resolve: Two Headline Numbers and Their Missing Denominators](https://www.howardism.dev/articles/what-the-instrument-can-resolve): Answers two open questions as one: the +12% AI-interview offer effect and ATLAS's 22.6% classifier accuracy are both ratios whose denominator was assumed rather than measured. For the first, the paper's own figures bound the abort channel — the arms lose almost the same share of interviews (43% human vs 45% AI), so the *symmetric* completion-conditional correction moves the effect **up** to +16%, not down; only the one-sided correction the question proposes flips the sign (−10%), and the screen-out variable it would run on is an unvalidated LLM label whose codebook thresholds on a variable the treatment moves. For the second, yes: leave-one-annotator-out human agreement is the ceiling estimator (ATLAS has the annotations and never computes it), and renormalizing gives 73–82% of the human ceiling at occupation title and 85–94% at major group — while at O*NET-task granularity ATLAS proves no ceiling is estimable, which is why aggregating categories *is* the ceiling fix
- [AI and Market Power](https://www.howardism.dev/articles/ai-and-market-power): OECD AI Papers No. 62 on French and Portuguese firm microdata plus global patent and start-up databases: non-GenAI adopters hold 7.5×/3.2× the market share of non-users, but the premium is selection (dies once broadband, digitalisation and lagged productivity enter) and adopters gain no market-share rank or markup growth over five years; firm-level GenAI exposure is inverted-U in size and market share while monotone in productivity and tertiary education; global AI-patent concentration *fell* 32–60% over 2001–21 yet correlates positively with sales concentration within markets; AI patents raise markups only in ICT (+7.95% interaction); and GenAI start-ups take ~130% more VC and are ~21% more likely to be acquired by incumbents
- [Owning Your Externalized Cognition](https://www.howardism.dev/articles/owning-your-externalized-cognition): Garry Tan's ownership axis on skill files: once your judgment is written down as executable markdown it is an asset with a holder, and the same file is either portable career capital or an extraction, depending only on whose repo it sits in — the appropriation counterpart to the cognitive-commons erosion argument, asserted from a keynote stage with no measurement behind it
- [Post-Scarcity Macroeconomics](https://www.howardism.dev/articles/post-scarcity-macroeconomics): Musk's claim that once digital intelligence acquires end effectors the economy goes quasi-infinite, so money 'won't matter' by 2036: the load-bearing argument is a deflation one — create money slower than output grows and prices still fall — which makes universal transfers non-inflationary and taxation moot; the transition path is the part he concedes he cannot describe
- [Controlled Variance: AI's Edge as Reduced Dispersion](https://www.howardism.dev/articles/controlled-variance): Jabarian & Henkel (arXiv 2607.28222): a pre-registered natural field experiment randomizing 70,884 job applicants between AI voice interviewers and human recruiters — offer rate 8.70%→9.73% (+12%), job starts +18%, one-month retention +18%, no productivity decline, with humans making every hiring decision in both arms. The mechanism the authors name is *controlled variance*: the AI follows the firm's interview protocol more consistently (topic order τ 0.53 vs 0.33, question similarity 0.59 vs 0.43, significantly lower cross-interview variance) while still adapting per applicant and using *richer* vocabulary — AI wins by being less dispersed, not more capable. The wiki's only randomized causal estimate of AI substituting for a human in an expert conversational task
- [The Solo-Authorship Rebound](https://www.howardism.dev/articles/solo-authorship-rebound): Matsui (arXiv 2607.10780): across 300M+ OpenAlex works and 26 fields, the decades-long decline in solo-authored papers halts or reverses at ChatGPT's November 2022 release — positive trend break in 23 of 26 fields, largest in Engineering (+2.5 pp/yr) and Business (+1.9), absent in Chemistry and Physics and negative in Arts and Humanities. It survives conditioning on author history and is strongest among authors who had *never* published alone; solo papers stay near their authors' coauthored content while narrowing 23% in breadth and tilting toward computational work. A solo paper is proposed as an observable behavioral trace of AI substituting for a human collaborator — but the design is an interrupted time series with no untreated unit, roughly half the pooled break is venue composition, and the disciplinary ordering, not any single number, is the actual argument
- [Task Crossover](https://www.howardism.dev/articles/task-crossover): OpenAI's Work at the Frontier (800K+ US ChatGPT work messages mapped to O*NET, July 2026): 16.8% of work messages and 43.5% of occupation-specific ones concern tasks historically belonging to another occupation — jobs reorganizing before job descriptions change. Borrowing and lending are separate directions (design borrows 35.2% and lends 1.7%; engineering lends 7.4%), financial calculation and software troubleshooting travel to all seven other groups, and crossover falls as workspace size rises (18.9% at 2–5 seats → 16.3% at 101+)
- [The Tragedy of the Cognitive Commons](https://www.howardism.dev/articles/the-tragedy-of-the-cognitive-commons): Lovett (HRD Review, July 2026): professional expertise is a profession-level commons whose regeneration mechanism — entry-level work — AI is removing. Distinguishes Internalized Mastery (built through cognitive struggle) from Distributed Mastery (orchestrating AI), and names the Validation Tether: substantive oversight of AI requires the expertise AI adoption erodes. Its sharpest claim is that junior labor's operational necessity was the hidden governance mechanism all along — regeneration was a side effect of business, never a decision
- [Is Breadth Cheap Now? Specialist Ramp Speed and Domain-Expert-as-Builder at Scale](https://www.howardism.dev/articles/is-breadth-cheap-now): Two-question synthesis on AI-era expertise economics. (1) Stone's specialists-broaden-quickly claim splits into two different goods: *tool-in-hand performance breadth* is measurably cheap — the concave expertise curve (novice→intermediate captures most of the verified-success gain), every occupation within 7pp of software engineers, and the management edge showing the expertise meta-skills (precision of framing, verify-specification, who-corrects-whom) transfer across domains, so an experienced specialist enters a new domain above the novice floor and AI-assisted onboarding compresses ramp further; but *retained-capability breadth* is unproven and the only randomized evidence cuts against it — automation-mode gains vanish when the tool is removed and skew to upper ability quartiles, while self-report hides the deficit. Cheap to perform, unproven to internalize; no source measures cross-domain ramp speed for experienced specialists directly. (2) Domain-expert-as-builder now has three evidence tiers: capability parity (measured, within-7pp), market existence (200K+ non-technical customers building ERPs/CRMs, vendor-claimed; AI responsibilities in 28–40% of business job descriptions), but *primary-job building as population-level practice* remains unshown — every measured population is selection-biased toward adopters, complements gate realized value, and ATLAS's composition shows experts pointing AI at their inexpert tasks rather than non-experts becoming builders. The gating variable is complements plus retained understanding, not capability
- [The Household Production Boundary](https://www.howardism.dev/articles/household-production-boundary): Google ATLAS's most novel contribution — 86.5% of conversational AI usage happens outside formal work, human time allocation predicts where AI questions go (slope 0.77, ~50% of variance), high-friction bureaucracy over-indexes ~20× with half of those queries outside business hours, and 0.5–5% household time savings values at $15–149B/yr in the US that GDP cannot see by construction
- [Task Saturation: Broad but Shallow AI Diffusion](https://www.howardism.dev/articles/task-saturation): Google ATLAS's marquee work finding — AI reaches 68% of detailed occupations (88.4% of US employment) but only 21% of the tasks in the median occupation, with end-to-end automation the intent of just 6.5% of non-routine-cognitive conversations vs 26.9% for routine-cognitive; the extensive margin is gated by physicality, the intensive margin concentrates in non-routine cognitive work, and usage over-indexes most on the *lowest*-expertise cognitive tasks
- [AI Economics & Labor](https://www.howardism.dev/articles/moc-ai-economics-and-labor): Map of Content for the ai-economics-and-labor domain — 23 concepts. AI's measured economic footprint: usage telemetry, labor-market effects, returns to expertise, organizational complements, and framing effects on accountability. Curated entry point; see Home for all domains.
- [Firm AI-Spend Intensity and Headcount Growth](https://www.howardism.dev/articles/firm-ai-spend-headcount-growth): Ramp × Revelio panel of 21,559 US firms: high-intensity AI-vendor spenders grow headcount ~10% (entry-level ~12%) over the 24 months after adoption while low-intensity adopters show no change — an intensity-gated learning-curve effect, read against Indeed's senior-tilted postings rebound and the Ramp AI Index adoption-breadth cut.
- [Experimental Learning Impact of Generative AI](https://www.howardism.dev/articles/experimental-learning-impact-of-ai): Contractor & Reyes (arXiv 2607.08849): a randomized, proctored experiment with 211 undergraduates finds off-the-shelf AI access raises immediate test scores +0.27 SD, ~76% of which persists a week later on unaided tests, and lifts essay quality only after AI is removed — but the durable gains belong almost entirely to 'augmentation' users (AI as tutor/explainer) while 'automation' users' (AI-drafts-the-text) short-run gains vanish once AI is gone; the objective, measured-skill counterpart to the AEI self-report that learning both persists and can be hollow depending on use mode
- [Market-Priced AI Exposure (the AI Premium)](https://www.howardism.dev/articles/market-priced-ai-exposure): Borri-Liu-Tsyvinski: market-implied AI exposure built from 380T tokens of realized OpenRouter consumption — an AI Factor, rolling firm-level AI Betas, and a priced 64 bps/week long-short premium concentrated on frontier/paid use; the implied skill map is orthogonal to task-based exposure measures, and tool-call tokens rising to 52% signal an agentic economy.
- [Context Advantage, Not Taste](https://www.howardism.dev/articles/context-advantage-over-taste): Andrew Ng's reframing of the residual human contribution: not 'taste' but an information asymmetry — 'so long as the human knows something the AI does not, human-in-the-loop is needed.' Recasts the wiki's central open question (is taste a ceiling or the next jagged valley?) as a category error, and makes the human role a closable engineering gap rather than a moat
- [AI Usage Cadences](https://www.howardism.dev/articles/ai-usage-cadences): AEI Cadences report: continuous hourly telemetry reveals AI usage carries the rhythms of daily life — personal use spikes 35%→~50% on weekends, recipes 2.3× at 6pm, sleep advice pre-dawn, tax queries 8× around the Apr-15 deadline; off-hours work skews toward higher-wage occupations
- [The Automation–Optimism Link](https://www.howardism.dev/articles/automation-optimism-link): AEI Cadences survey finding: people who use Claude in more automated ways are MORE optimistic across all six job-quality dimensions (pay, security, job-finding, meaning, autonomy, human interaction), report their skills growing more valuable, and show no learning deficit — inverting the common delegation→deskilling-anxiety narrative
- [Conversation Artifacts](https://www.howardism.dev/articles/conversation-artifacts): AEI Cadences report: the 'artifact' (the primary output a user takes away) as a new unit of economic analysis — 93% of conversations produce one, artifact type predicts work/personal/coursework use, compute (tokens) scales with the artifact's economic value, and Claude's output sits ~1 education-year above the prompt
- [Exposure Taxonomy: Observed, Theoretical, Reported, Anticipated](https://www.howardism.dev/articles/exposure-taxonomy): Four distinct ways to measure AI's reach into an occupation — observed exposure (tasks seen done with Claude), theoretical exposure (tasks an LLM could do), reported exposure (what workers say AI can do today), and anticipated exposure (what they expect in 12 months) — plus their orderings (theoretical > reported > observed), the GDP/experience/automation gradients the AEI survey reveals, and Steele & Cruz's seven-instrument head-to-head showing the instruments cluster by data source rather than by construct, nominate eleven distinct occupations across twelve most-exposed slots, and flip even the *sign* of the exposure-salary relationship by vintage
- [Conversation-to-Delegation Shift](https://www.howardism.dev/articles/conversation-to-delegation-shift): OpenAI's Codex usage study (June 2026): the move from conversational AI ('asking') to agentic AI ('delegated production'), measured by Codex's share of output tokens across three populations — 99.8% OpenAI / 63.3% organizational / 16.5% individual — with adoption spreading beyond developers; standard usage metrics (active users, chats) become less informative as the unit shifts from a conversation to a delegated workflow
- [Organizational Complements to AI](https://www.howardism.dev/articles/organizational-complements-to-ai): The general-purpose-technology argument: AI productivity gains depend on complementary workflow, skill, and org-design changes (David's electrification analogy, Brynjolfsson's paradox) — OpenAI's Codex natural experiment (99.8% vs 16.5% usage of the same model) shows the gap is complements; also home to the HAT substitution model and Kalff & Simbeck's institutional complement.
- [Returns to Expertise in Agentic Coding](https://www.howardism.dev/articles/returns-to-expertise): Anthropic's 400K-session study: domain expertise (not coding skill) is what amplifies an agent — experts get 2× the actions and 5× the output per prompt, reach verified success ~2× as often, and abandon stuck sessions far less; every occupation lands within 7pp of software engineers; gains are concentrated novice→intermediate, with mastery adding little
- [Orchestration vs Employee Framing: Reconciling the Founder's Playbook with HBR's Accountability Evidence](https://www.howardism.dev/articles/orchestration-vs-employee-framing-reconciliation): Reconciles the Founder's Playbook orchestration framings with HBR Kropp et al.'s accountability evidence; "orchestration as workflow design" survives the critique; "orchestration as mental model of agents-as-coworkers" does not; operational checklist for the disciplined founder
- [AI Brain Fry](https://www.howardism.dev/articles/ai-brain-fry): Kropp et al. 2026/03: mental fatigue from excessive AI oversight increases minor errors +11%, major errors +39%; cognitive cost surface for both tool and employee framings
- [AI Employee Framing](https://www.howardism.dev/articles/ai-employee-framing): Kropp et al. (HBR May 2026, n=1,261): framing AI agents as "employees" vs "tools" cuts personal accountability −9pp, increases escalation +44%, reduces error catching −18%, no adoption gain
- [Human-AI Accountability Redesign](https://www.howardism.dev/articles/human-ai-accountability-redesign): HBR five-pillar prescription: span-of-control redesign, role redesign, performance management reset, decision-rights/escalation/consequences, agentic-unit-not-human-role design

## Superintelligence Trajectory

- [Safety Commitments That Cannot Bind the Actor Who States Them](https://www.howardism.dev/articles/safety-commitments-that-cannot-bind): Three entity-page motive questions join on one structure: a safety commitment stated in a form incapable of binding its author — Anthropic's pause posture conditioned on a verification regime that does not exist, DeepMind's report assuming alignment solved and thereby dropping from its own friction table a bottleneck it concedes in the same paragraph, and OpenAI's charter with no mechanism to persist. Musk's 'all roads lead to acceleration' generalization is unsupported: it is n=1, self-reported, counterfactual-free, and the corpus holds one safety intervention with a real mechanism (the RSP) that bound once — Mythos Preview withheld — and bent twice, including a CB-2 call decided by three qualitative runs against a frontier-level automated portfolio in the direction of shipping. What every recorded mechanism lacks is an external actor holding a decision it can make against the developer's interest; the corpus records that lever existing exactly once (the LTBT's external-review power) and never being pulled.
- [Governance by Benchmark Threshold: What an Index Must Prove Before an Obligation Can Rest on It](https://www.howardism.dev/articles/governance-by-benchmark-threshold): Answers the paired ECI-as-legal-threshold and benchmark-as-regulatory-perimeter questions with seven stability properties an obligation-bearing measurement would need — referential fixity, discriminating range at the trigger, a defensible score→obligation map, bidirectional manipulation resistance, a published integrity audit, second-party reproducibility, and independence from the measured party — and grades each against the wiki's evals evidence: two are demonstrated today (reproducibility, integrity audit), three are institutional choices nobody has made, and two are unachievable at the frontier, because benchmarks have kept ordinal signal and lost cardinal signal while a legal perimeter is a cardinal object; the operative consequence is that a score can support a reporting or case-opening obligation and cannot support a self-executing one, and that the pacing proposal survives its own question only because its two administrable facts (compute share, training date) are not benchmark scores at all
- [Open Weights as Competitive Strategy](https://www.howardism.dev/articles/open-weights-as-competitive-strategy): Andrew Ng's argument (WaPo Live, July 2026) that open models are a national-competitiveness instrument rather than a safety liability: diffusion compounds faster for the releaser than for the world, price-sensitive markets are being won by Chinese open models by default, cost-of-intelligence is a downstream input cost so a 3× token bill is a structural disadvantage for every application builder, and an open model run on domestic infrastructure is domestically controlled — plus his rebuttals that anti-open-weight lobbying is 'false' and that distillation as an explanation for Chinese gains is 'vastly overstated'; entirely practitioner-opinion, and the diffusion claim is the one the vault can partly check
- [Continuous Self-Modification Under Review](https://www.howardism.dev/articles/continuous-self-modification-under-review): Ouroboros/Hope: a coding-agent harness that rewrites its own core through a blocking multi-model review gate, run 161 days as a public deployment (1,085 self-modification commits, 94.2% agent-authored, 63.5% recent review block rate) — and the source's real lesson, that its only time series measures deployment *activity* (spend, tokens, published LOC, memory artifacts) rather than capability, while every benchmark score was produced on a frozen seed with self-evolution switched off
- [Domestic Frontier Pacing](https://www.howardism.dev/articles/domestic-frontier-pacing): AI Futures Project's four-option ladder for pacing US frontier AI unilaterally — temporary pause (100% inference), compute-allocation minimums (70% external inference + 25% transparent safety + 5% capabilities), a 9-month capability lag on models used for AI R&D, and safety-case risk assessments capped at 1% existential risk per month — plus the corpus's first concrete auditor-access ladder and its first named threshold fractions for a compute-allocation rule
- [Frontier AI Standards Body](https://www.howardism.dev/articles/frontier-ai-standards-body): Hassabis's July 2026 proposal for a US-led, FINRA-modelled public-private standards body that tests Frontier-class models up to 30 days pre-release — voluntary first, mandatory once the protocol is 'shown to be effective and robust', with a ratchet to a coordinated cross-lab slowdown; it names who tests but never who decides, its regulatory perimeter is a benchmark threshold, and it is the earliest of the corpus's three pre-release-oversight proposals
- [Government Checkpoint Sharing](https://www.howardism.dev/articles/government-checkpoint-sharing): Zuckerberg's August 2026 proposal that frontier labs hand governments intermediate training checkpoints plus technical staff — capability transfer to the defender instead of a release-gating review — designed so oversight adds zero delay to public release; the acceleration-compatible pole of the pre-release-oversight design space
- [Cross-Lab Pre-Release Review](https://www.howardism.dev/articles/cross-lab-pre-release-review): Musk's proposal that frontier labs get 1–2 weeks of competitor API access to test each other's models before release, with government reserved for the case where a lab refuses to act on a flagged danger — competitors as the technically-capable honest brokers, on the MPAA self-rating model; the Mythos cyber-risk incident is the informal precedent
- [Balance-of-Power Superintelligence](https://www.howardism.dev/articles/balance-of-power-superintelligence): Zuckerberg's thesis: distribution of personal superintelligence to individuals — not centralized control — is the safety mechanism; anti-singleton alignment argument (humanity isn't a monoculture); jobs optimism conditional on the automation-vs-empowerment balance. The August 2026 Meta manifesto is the full statement, adding an RSI compute-allocation rule, alignment redefined as alignment-to-the-person, and a lab-government checkpoint proposal
- [Superintelligence Trajectory](https://www.howardism.dev/articles/moc-superintelligence-trajectory): Map of Content for the superintelligence-trajectory domain — 27 concepts. The path from AGI to ASI: recursive self-improvement, intelligence-explosion dynamics, ASI theory and limits, and frontier governance. Curated entry point; see Home for all domains.
- [Researcher Uplift from Code Output](https://www.howardism.dev/articles/researcher-uplift-from-code-output): Thomas Kwa (METR) translates Anthropic's reported 8× code-per-engineer-per-day into serial researcher uplift with production functions: Cobb-Douglas gives U = M^β = √8 ≈ 2.83, CES stays within ±3% of that across elasticities because 8 ≈ e², and a low-stakes-code-discounted model still lands [2.33, 2.66] — so researcher uplift from coding agents alone is plausibly >2×, reconciled with Anthropic's 'well short of 2× overall R&D uplift' because R&D speedup also depends on compute (Greenblatt: labor^0.55 × compute^0.45)
- [Open-Weight Elicitation Irreversibility](https://www.howardism.dev/articles/open-weight-elicitation-irreversibility): A wiki-drawn synthesis of Brown and Gemma 4: if dangerous capability scales with inference budget, then an open-weight release fixes the model's safety evaluation at one budget forever while leaving elicitation budget unbounded and recall impossible — the closed-weight mitigations (classifier fallback, suspension, retention) all require a server the vendor controls; the corpus's one worked audit (UK AISI/CAISI on Kimi K3, four days pre-release) is black-box through the vendor API at a single budget, and its de-safeguarded US comparator inverts the ranking on obtainable rather than latent capability
- [The Abstraction Barrier](https://www.howardism.dev/articles/abstraction-barrier): Lerchner's hypothesis that AI trained on human concepts may be unable to discover genuinely novel conceptual primitives from raw data — capping single instances near AGI — and the embodied bottleneck that grounds concept validation in real-world experiment speed, converting recursive self-improvement into a process paced by empirical science
- [Advantages of Digital Intelligence](https://www.howardism.dev/articles/advantages-of-digital-intelligence): The six properties (Table 1) that follow from knowing an AI's source code — I/O speed, processing speed, working memory, substrate independence, lossless replication, high-bandwidth experience sharing — each of which scales with compute in ways biological intelligence cannot, widening the human–AI gap
- [AGI-to-ASI Pathways](https://www.howardism.dev/articles/agi-to-asi-pathways): DeepMind's four non-exclusive, parallel technological routes from human-level AGI to superintelligence — scaling, algorithmic paradigm shifts, recursive self-improvement, and multi-agent group agency — plus the six frictions (data wall, economics, paradigm-insufficiency, research-gets-harder, abstraction barrier, deliberate slowdown) whose impact is the report's central set of open research questions
- [Artificial Superintelligence (ASI)](https://www.howardism.dev/articles/artificial-superintelligence): DeepMind's informal characterization of ASI as a system that exceeds large, well-coordinated human-expert collectives across virtually all domains — distinct from human-level AGI below it and the incomputable Universal AI limit above it, all points on the Legg–Hutter intelligence continuum
- [Effective Compute Scaling](https://www.howardism.dev/articles/effective-compute-scaling): DeepMind's framing of compute growth as ~10×/year of 'effective compute' — the product of hardware improvement (~1.5×/yr), compute investment (~2.5×/yr), and algorithmic efficiency (~3–6×/yr) — and the data-wall and economic frictions that determine how long the scaling pathway to ASI can be sustained
- [Fundamental Limits of ASI](https://www.howardism.dev/articles/fundamental-limits-of-asi): Even far-superhuman AI is bound by hard physical (Landauer, Bremermann, Bekenstein, light-speed), complexity-theoretic (P vs NP), and logical (Gödel, Halting) limits — but these negative results are often 'vacuous' in practice because good heuristic approximations exist below the worst case
- [Intelligence Explosion Dynamics](https://www.howardism.dev/articles/intelligence-explosion-dynamics): The growth-curve question behind recursive self-improvement: whether AI-accelerating-AI produces exponential, super-exponential/hyperbolic (singularity-in-finite-time), or S-curve dynamics — and the four mechanisms (genetic, cultural, cooperative, data) plus the physical/economic frictions that bound it
- [Multi-Agent Collective Intelligence](https://www.howardism.dev/articles/multi-agent-collective-intelligence): DeepMind's fourth pathway to ASI: superintelligence as an emergent property of many coordinated AGI agents — group agents, virtual agent economies, and centrally-steered super-collectives — governed by hoped-for 'multi-agent scaling laws' and the open question of when a homogeneous LLM collective actually becomes more than the sum of its parts; now carrying a candidate mechanism for the measured flat-to-negative group-size curve, from control theory rather than from agents — width averages only the noise that is independent per agent, so structure shared across the population is a floor no population size lowers
- [RSI Growth Curves: Which Friction Binds First?](https://www.howardism.dev/articles/rsi-growth-curves-which-friction-binds): DeepMind's exponential/hyperbolic/S-curve growth shapes are Anthropic's compounding-efficiency/full-RSI/stalled futures seen from the dynamics side, not the policy side — one trichotomy described twice. Both labs converge on the same answer to 'which friction binds first': the slowest un-acceleratable step coupling the loop to reality (verification/oversight at org scale today, physical-experiment and institutional latency at the frontier), not cognition, which is racing and hasn't bent; research-gets-harder demotes itself into compute, the abstraction barrier is the candidate fundamental blocker, and deliberate slowdown is the only friction humans must install. The data wall's tier-4 demotion was revised 2026-08-17: it is rationed by verifier availability, verifier latency and diversity collapse rather than absorbed by compute, so it converts into the verification friction ranked first here rather than leaving the board.
- [Transformative Creativity](https://www.howardism.dev/articles/transformative-creativity): Boden's three-level model of creativity (combinational, exploratory, transformative) used to locate today's AI achievements — Move 37, AlphaFold, theorem-proving — at the exploratory level within human-given conceptual spaces, and to frame Boden level-3 (creating new conceptual spaces, à la Hassabis's 'could AI rediscover general relativity?' test) as a hallmark requirement of true ASI; now with the corpus's first system whose conceptual space is a printed artifact, an Idea Bank of 30 expert-derived plus 49 LLM-brainstormed ideas, every one of which names a pre-existing technique
- [Universal AI (AIXI)](https://www.howardism.dev/articles/universal-ai-aixi): Hutter & Legg's formal upper bound on machine intelligence: AIXI, the incomputable agent optimal on average over all computable environments under Solomonoff's universal prior; the theoretical endpoint of the intelligence continuum that ASIs approximate from below
- [Autonomous Scientific Discovery](https://www.howardism.dev/articles/autonomous-scientific-discovery): Mythos-class models now conduct novel science with limited human input — autonomous protein/drug design (~10× faster, matching skilled humans), molecular-biology hypotheses preferred ~80% over Opus-class (one E. coli mechanism independently corroborated), and week-long genomics that beat a Science-published model at 100× smaller; the wet-lab analogue of AI-driven formal proof search, and fresh evidence in the research-taste debate
- [Capability-Gated Model Fallback](https://www.howardism.dev/articles/capability-gated-model-fallback): Fable 5's safeguard architecture: classifiers detect cyber / bio-chem / distillation queries and route the response to a less-capable model (Opus 4.8) instead of refusing — 'fallback, not refusal'; >95% of sessions never trigger; conservative tuning, robust to 1,000+ hours of jailbreak testing; a new point on the safeguard spectrum for capabilities past a risk threshold
- [AI Accelerating AI Development](https://www.howardism.dev/articles/ai-accelerating-ai-development): The empirical core of *When AI builds itself*: measured evidence AI already speeds AI R&D at Anthropic — >80% of merged code Claude-authored, ~8× code/engineer/day vs 2024, a kernel-optimization eval going 3×→52× in a year, an automated researcher recovering 97% of a weak-to-strong gap, and model next-step judgment beating humans 64%
- [AI R&D Autonomy Evaluation (AECI)](https://www.howardism.dev/articles/ai-rd-autonomy-evaluation): How Anthropic measures whether a model can automate or dramatically accelerate AI research — the capability that drives recursive self-improvement; tracked via the AECI capability index plus concrete shortcomings vs. human researchers; Opus 4.8 sits below the frontier and is not close to substituting for research staff, and the August 2026 Risk Report supplies the promised direct measurement — CoBench on 449 real Anthropic engineering issues with an 85% substitution bar, a ~4x researcher self-report, a revealed-preference argument whose cost experiment was never run, and 31 expert interviews finding no dramatic acceleration in any non-AI domain
- [Frontier Pause Verification](https://www.howardism.dev/articles/frontier-pause-verification): The arms-control problem of a credible, verifiable slowdown or pause of frontier AI: detectability is harder than for other technologies (training runs are easier to conceal than missile silos), so the Anthropic Institute aims to build the verification systems a multilateral pause would require
- [Recursive Self-Improvement](https://www.howardism.dev/articles/recursive-self-improvement): An AI system autonomously designing and developing its own successor; Anthropic Institute's *When AI builds itself* argues AI is already accelerating AI development (engineers ship ~8× more code/quarter) and lays out three futures — stalled-but-diffused, compounding-efficiency, and full RSI
- [Research Taste as the Human Bottleneck](https://www.howardism.dev/articles/research-taste-as-human-bottleneck): The narrowing human role as AI absorbs execution: choosing which problems matter, which results to trust, and when an approach is a dead end; the top rung of the autonomy ladder, and the open question of whether taste is 'just another capability' AI fails at then masters
- [Responsible Scaling Policy Evaluations](https://www.howardism.dev/articles/responsible-scaling-policy-evals): Anthropic's RSP gates deployment on pre-release capability evaluations in CBRN, automated AI R&D, and high-stakes misalignment; the Opus 4.8 determination is that it does not advance the frontier beyond Mythos Preview, and the August 2026 Risk Report (RSP v3.4) is the framework's other deliverable — a whole-company assessment that raises two of its own four ratings, reoperationalizes the AI R&D and CB-2 thresholds as substitution tests, and forecasts crossing CB-2 before the security it recommends for that threshold exists

## Entities

- [Bridgewater Associates](https://www.howardism.dev/articles/bridgewater-associates): The systematic macro hedge fund whose 50-year habit of writing down the causal logic of every trade left it holding a machine-readable expert system before agents needed one — the corpus's clearest instance of a pre-existing knowledge asset as the binding input to an AI deployment; builder of PAT, an internal AI analyst deployed to hundreds of investors, via an internally-incubated applied-AI startup staffed by investors, technologists and scientists side by side
- [Aakanksha Chowdhery](https://www.howardism.dev/articles/aakanksha-chowdhery): Adjunct professor at Stanford, co-instructor of CS329A, and a researcher at Reflection AI; previously Google Brain, where she was lead author of the PaLM paper and a co-author on self-consistency. In CS329A she supplies the coverage→pass@1 framing that separates repeated sampling from reasoning-model training and names the generator–verifier gap as the field's bottleneck; her solo lectures sort self-improvement by where the feedback comes from (4), close the loop with STaR/GRPO/DAPO while bounding it — the RL raised majority@K and not pass@K (6), reframe search as curation and rule that a better base model beats more samples (7), and set three rulers against each other, using GDPval's linear win rate to brake METR's exponential (8). Lecture 9 closes the course on the loop's three unsolved inputs: chain diversity, verifier reliability, task selection
- [Azalia Mirhoseini](https://www.howardism.dev/articles/azalia-mirhoseini): Stanford CS assistant professor and co-instructor of CS329A; previously Google Brain, Anthropic (Claude) and Google DeepMind (Gemini). Senior author of Large Language Monkeys (2024) — the repeated-sampling result that made inference-time scaling a research program — and of CodeMonkeys; her lab's line of work supplies the empirical ancestor of this wiki's test-time-compute and capability-overhang pages, and its verification (Weaver), planning (SPRINT, SWiRL) and efficiency (intelligence per watt, Hydragen, Tokasaurus) lines are three of the course's four research threads
- [CS329A: Self-Improving AI Agents (Stanford)](https://www.howardism.dev/articles/cs329a-self-improving-ai-agents): Stanford's graduate course on self-improving agents, taught by Azalia Mirhoseini and Aakanksha Chowdhery (Autumn 2025, published Aug 2026). Thesis: test-time compute manufactures the training data that improves the next model. All nine published lectures are compiled here, each turning the axis — 1 scaling laws through to agents; 2 the coverage power law and the generation–verification gap; 3 four years of trained verifiers; 4 where the feedback comes from; 5 where the planning lives; 6 train-time scaling, where majority@K rose and pass@K did not; 7 search as curation; 8 METR's exponential duration curve against GDPval's linear win rate; 9 the loop's three unsolved inputs — chain diversity, verifier reliability, task selection — plus intelligence per watt
- [US Center for AI Standards and Innovation (CAISI)](https://www.howardism.dev/articles/caisi): The US government's AI-evaluation body, publishing through NIST; UK AISI's counterpart and co-evaluator — joint author of the July 2026 Kimi K3 cyber assessment and the credited source of its cross-benchmark IRT/Elo capability analysis, co-builder of the Gray Swan indirect-prompt-injection benchmark, and a notification recipient in AISI's own INC-2026-07-28-01
- [Greptile](https://www.howardism.dev/articles/greptile): AI code-review agent vendor that runs frontier models from OpenAI and Anthropic under the hood; its research team publishes the same-model review-blindness study (two 500-PR labelled datasets, ~1,500 high-severity bugs) and shipped its finding as the Model Inversion routing feature — also one of the seven bots measured in the 18.9% secret-detection floor
- [Transluce](https://www.howardism.dev/articles/transluce): AI research lab (Jacob Steinhardt, Sarah Schwettmann) building Docent — an LLM-driven transcript-analysis tool that flags agent behaviours from logs against a written rubric — and using it for behavioural measurement other labs don't run: the corpus's only misalignment rate on unconstructed production traffic (8,600 real coding sessions) and the first study of how a model's belief about who its user is shifts its confidence, suspicion and grading
- [LlamaIndex](https://www.howardism.dev/articles/llamaindex): The RAG-framework company (run-llama) that narrowed its focus to document parsing for agents — LlamaParse (hosted, vision, Markdown-out, per-page pricing), LiteParse (Apache 2.0, local, spatial text + bboxes), LlamaExtract (Pydantic schema in, cited typed JSON out), LlamaCloud, event-driven Workflows, and ParseBench, the parsing leaderboard it publishes and leads
- [Jeff Dean](https://www.howardism.dev/articles/jeff-dean): Google's Chief Scientist; built MapReduce, BigTable, TensorFlow and the TPU, and co-authored the 2014 distillation paper NeurIPS rejected that now makes Gemini's Flash models cheap. His recurring method is napkin math against a bottleneck — search-in-RAM (2001), speech-would-double-the-fleet (2013) — and his 2026 advice to founders is the 1% rule: build where models fail 0–1% of the time, not 20%
- [Elon Musk](https://www.howardism.dev/articles/elon-musk): Founder of Tesla, SpaceX and xAI, and the corpus's clearest case of a reversed AI-risk position: 2015 'we'll be pet labradors' → 2023 pause-letter signatory → 2025 10–20% p(doom) → 2026 'even if there was a stop button we probably shouldn't press it'; his current answer is acceleration plus cross-lab pre-release review plus abundance
- [Erik Brynjolfsson](https://www.howardism.dev/articles/erik-brynjolfsson): Director of the Stanford Digital Economy Lab and the vault's most-cited economist — a disambiguation page, because five different works appear here under one surname: the Brynjolfsson-Rock-Syverson productivity paradox that anchors the complements thesis (well corroborated), the 2018 SML rubric that is one of seven exposure instruments which disagree (largely superseded), the 2025 'Canaries in the coal mine' 22-25-year-old -16% result the vault's firm-level panel contradicts, a 2025 workplace-writing homogenization finding a randomized essay experiment did not reproduce, and the July 2026 'We Must Act Now' open letter he organized
- [GPT-Live](https://www.howardism.dev/articles/gpt-live): OpenAI's third-generation voice system (July 2026): a full-duplex voice model that listens and speaks simultaneously — no turn detector anywhere in the audio path — and consults frontier models (GPT-5.5) over an asynchronous delegation path without interrupting the conversation; replaced Advanced Voice Mode after a silent production shadow test; powers ChatGPT Voice including desktop computer control and agent coordination, with a GPT-Live API announced as upcoming
- [Bun](https://www.howardism.dev/articles/bun): The JavaScript/TypeScript runtime, bundler, package manager and test runner created by Jarred Sumner; 22M+ monthly CLI downloads; Claude Code's runtime and the sandbox dynamic workflows execute inside; acquired by Anthropic December 2025 and ported from 535,496 lines of Zig to Rust by Claude in 11 days (v1.4.0)
- [Cline](https://www.howardism.dev/articles/cline): Open-source coding agent and harness (VS Code extension, bring-your-own-key or ClinePass subsidized inference) that publishes its benchmark hill-climbing as a practice — a Feb 2026 playbook, a Jan 2026 Opus 4.5 campaign (47%→57% on Terminal-Bench, four engineers, two weeks), and a July 2026 one-prompt autonomous campaign that took Kimi K3 from 77.5% to 88.8% on Terminal-Bench 2.1
- [Cursor](https://www.howardism.dev/articles/cursor): The AI coding company behind the Cursor IDE, the Composer model family, and the agent-swarm research line; in the corpus it appears in three unrelated roles — a publisher of first-party swarm engineering (planner/worker roles, a custom 1,000-commits-per-second VCS, merge-conflict mediation, the agent-authored Field Guide), a heavily-measured coding agent in third-party telemetry and security studies, and the vendor with the largest count of reproduced sandbox escapes (CVE-2026-48124 and three more, fixed in 3.0.0)
- [Jarred Sumner](https://www.howardism.dev/articles/jarred-sumner): Creator of the Bun runtime, now an Anthropic employee after the December 2025 acquisition; author of 'Rewriting Bun in Rust', the wiki's most detailed first-party account of running a large engineering project on ~50 Claude Code dynamic workflows
- [OpenHands](https://www.howardism.dev/articles/openhands): Open-source coding-agent platform (formerly OpenDevin, Wang et al., ICLR 2025) and the company behind it; four public repos — app/server, Software Agent SDK, Agent Canvas UI, CLI — totalling ~1.05M lines and 5,679 merged PRs in the 12 months to July 2026; in this corpus it appears mostly as the third-party research scaffold that papers run SWE-Bench and Terminal-Bench agents inside
- [Xiaohongshu](https://www.howardism.dev/articles/xiaohongshu): Chinese social-commerce platform (RED / 小红书) whose engineering team published Self-GC, the corpus's only measured, production-deployed treatment of agent context management — object-level context lifecycle control validated on 332 production-derived agent sessions and a live account-level traffic split
- [Kimi (Moonshot AI)](https://www.howardism.dev/articles/kimi): Moonshot AI's open-weight Kimi line — K2.5/K2.6 as 1T-class MoEs already circulating in this corpus (Inkling's post-training bootstrap; Arena rank 34), and K3 (July 2026) as the first open 3T-class model: 2.8T total / 104B active, 16-of-896 LatentMoE, hybrid 69 KDA + 24 Gated MLA attention, 401M MoonViT-V2 vision encoder, 1M context, MXFP4 quantization-aware training, and a 45-benchmark card that trails Claude Fable 5 on most rows while topping it on search, MCP orchestration, and document vision; UK AISI/CAISI's pre-release cyber assessment is the only non-vendor measurement of it — 32.2% on ExploitBench, step 17 of 32 on a cyber range, 0 of 41 arbitrary-code-execution solves, and safeguards that did not prevent attempted exploit development
- [Elizabeth Stone](https://www.howardism.dev/articles/elizabeth-stone): Netflix Chief Product and Technology Officer (economist by training: Analysis Group, Merrill Lynch trader, Nuna COO, Lyft VP of Science, Netflix CTO→CPTO); articulates systems-thinking-over-specialization hiring and 'excellence as an operating system'; two-time Lenny's Podcast guest
- [Claude Opus 5](https://www.howardism.dev/articles/claude-opus-5): Anthropic's Opus-class release of July 2026; matches Mythos 5 on capability without advancing the frontier, is the best-aligned and most injection-robust model Anthropic has shipped, and is simultaneously the first to confidently assert answers its own reasoning does not support
- [Google AI & Economy ATLAS](https://www.howardism.dev/articles/google-ai-economy-atlas): Google's recurring economic-research program measuring Gemini usage across the economy — ATLAS v1.0 (July 2026) maps 14.65M de-identified interactions from Gemini App, AI Mode, and the Gemini API onto BLS/O*NET occupations and ATUS household activities across 150 countries and 143 languages; the direct methodological rival to the Anthropic Economic Index, and the first such program to publish its classifier-validation numbers
- [Nate Parrott](https://www.howardism.dev/articles/nate-parrott): Anthropic product designer who built Claude Design; sole designer on Claude Code for VS Code in fall 2025, then spent a month of side-project time closing the velocity gap that opened when Opus 4.5 accelerated his engineers but not him — the HTML-playground prototype that resulted became an Anthropic Labs product
- [Inkling](https://www.howardism.dev/articles/inkling): Thinking Machines Lab's first from-scratch open-weights family (July 2026): a 975B/41B-active multimodal MoE with 1M context, continuous thinking-effort dial (0.2–0.99), encoder-free audio/vision, and calibration trained via RL on proper scoring rules — positioned not as the strongest open model but as the best base for fine-tuning on Tinker; Inkling-Small (276B/12B) previews the same recipe at interaction-model shape
- [Emergent](https://www.howardism.dev/articles/emergent): Indian AI 'engineering-team-in-a-box' app builder (Bengaluru, the Jha brothers); a $1.5B unicorn on a $130M Series C (July 2026) with company-reported $120M ARR, 200K+ non-technical paying customers, ~200 employees; Garry Tan's headline revenue-per-head exhibit — whose per-head extreme (~$600K/head) compresses below top-decile AI RPE on inspection
- [Garry Tan](https://www.howardism.dev/articles/garry-tan): President & CEO of Y Combinator; founder-investor turned evangelist for the AI-native organization — the ~400x output claim, "the leverage is not in the weights, it's in how you wire the work", the skillify-it discipline, and GBrain, his MIT-licensed open-source company brain (~220K pages)
- [OpenClaw](https://www.howardism.dev/articles/openclaw): Peter Steinberger's open-source personal AI agent / harness (openclaw.ai); the canonical example of agent-native distribution (install = text you paste to your agent); a skills ecosystem (ClawHub), YC's internal harness per Garry Tan, and the runtime real-world security work deploys against
- [GLM (Z.AI)](https://www.howardism.dev/articles/glm): Z.AI's (Zhipu AI, Tsinghua-affiliated) open GLM model family — GLM-4.5 the agentic/reasoning/coding foundation model, GLM-4.7 a frontier-competitive reasoner that in this corpus beats GPT-5 High and Claude-Sonnet-4.5 on AIME2025/HMMT/IMOAnswerBench, and GLM-5.2 a 750B-total/40B-active open MoE trained with SAO, reported by Databricks as statistically tied with Opus 4.8 on quality at 34% less per task, and named by UK AISI/CAISI the most cyber-capable open-weight model as of June 2026 before Kimi K3 displaced it; the large-MoE open-weight line that competes on capability where Gemma competes on efficiency
- [UK AI Security Institute](https://www.howardism.dev/articles/uk-ai-security-institute): UK government AI-evaluation body (Science of Evaluation team); its July 2026 test-time-compute study is the first independent, government-institute empirical corroboration that agent capability is a curve over compute, not a fixed score — also runs the 'The Last Ones' and 'Doing Life' cyber ranges, co-maintains the Agent Red Teaming benchmark, probed Fable 5 for a universal jailbreak, published the corpus's first base rate for evaluation cheating (every frontier model, 7.8–14.1% of 475 runs) via an automated trajectory monitor, on 2026-08-04 self-disclosed INC-2026-07-28-01, an incident on its own Doing Life range in which evaluated agents deceived two uninvolved real developers on the live internet, and with US CAISI published the corpus's only third-party dangerous-capability assessment of an open-weight model (Kimi K3, four days before its weights shipped)
- [Jack Lindsey](https://www.howardism.dev/articles/jack-lindsey): Anthropic interpretability researcher; corresponding author of the global-workspace paper, co-originator of the Jacobian lens, and the one who ran the directed-modulation and post-training-diffing experiments that turned a readout method into a claim about model cognition
- [Wes Gurnee](https://www.howardism.dev/articles/wes-gurnee): Anthropic interpretability researcher; co-first author and co-originator of the Jacobian lens, who conceived the connection between verbalizable representations and conscious access and led the method's development
- [Andrew Ng](https://www.howardism.dev/articles/andrew-ng): Founder of DeepLearning.AI and AI Fund, founding lead of Google Brain, co-founder of Coursera; writes The Batch, where his June 2026 letter set out the three-loop taxonomy of AI-native building and reframed the residual human contribution as a "context advantage" rather than taste — and in a July 2026 Washington Post interview argued open weights are a national-competitiveness instrument, called anti-open-model lobbying "false," and rejected distillation as the explanation for Chinese gains
- [Gemma 4](https://www.howardism.dev/articles/gemma-4): Google DeepMind's July 2026 open-weight multimodal family (Apache 2.0): 2.3B–31B dense plus a 26B/4B-active MoE, adding a thinking mode, an encoder-free 12B that discards its audio encoder entirely, and a deep inference-efficiency stack (−37.5% KV cache, QAT to sub-GB, MTP drafters); Arena rank 43, top *dense* open model
- [Noam Brown](https://www.howardism.dev/articles/noam-brown): OpenAI research scientist and a pioneer of inference-time (test-time) compute scaling; earlier built superhuman poker AIs and now uses building poker solvers as a personal model eval; author of the June 2026 essay *Implications of Large-Scale Test-Time Compute*
- [Andrew Ambrosino](https://www.howardism.dev/articles/andrew-ambrosino): Product & engineering lead for the Codex desktop app at OpenAI; a designer→engineer→PM→founder generalist whose June 2026 Lenny's Podcast interview is the wiki's OpenAI-side account of how cheap implementation inverts product work toward taste and curation
- [Anthropic Economic Index](https://www.howardism.dev/articles/anthropic-economic-index): Anthropic's recurring economic-research program measuring how Claude usage maps to and diffuses through the economy — privacy-preserving usage telemetry (Clio) now paired with a linked survey; reports include the June 2026 Cadences report, the returns-to-expertise study, and the agentic-coding work-composition analyses
- [Claude Sonnet 5](https://www.howardism.dev/articles/claude-sonnet-5): Anthropic's most agentic Sonnet yet (July 2026); narrows the gap to Opus 4.8 at lower price via effort-level cost-performance tuning; 1.0–1.35× tokenizer inflation; safer than Sonnet 4.6 on the behavioral audit but weaker cyber than Opus; ships default real-time cyber safeguards; and on the first third-party per-task bench costs *more* than Opus 4.8 per task ($2.09 vs $1.94) at lower success (81% vs 87%) despite ~1.7× cheaper tokens
- [Gemini Enterprise Agent Platform](https://www.howardism.dev/articles/gemini-enterprise-agent-platform): Google Cloud's agent platform: the GenAI evaluation service with adaptive AutoRaters (built with DeepMind), User Simulator, Automatic Loss Analysis, Online Monitors, OTel tracing, and the ADK/agents-cli toolchain; ships the quality-flywheel eval skill in two packages
- [Codex](https://www.howardism.dev/articles/codex): OpenAI's agentic coding and work platform: a CLI (April 2025) plus a desktop app (built Nov 2025, released Feb 2026) built on the GPT-5-series Codex models, extended by skills/plugins, a headless App Server Protocol, and the Symphony orchestrator; the OpenAI-side reference harness paired against Claude Code, subject of the June 2026 'Shift to Agentic AI' study, and — per its product lead — an app ~90% of OpenAI's whole company uses that is spreading from code into general knowledge work
- [Addy Osmani](https://www.howardism.dev/articles/addy-osmani): Engineering leader at Google (Chrome) and prolific author/educator; in 2026 writes a widely-read blog series on AI-assisted engineering — agent harness engineering, the factory model, comprehension/intent debt, cognitive surrender, and the essay that named loop engineering
- [Faros AI](https://www.howardism.dev/articles/faros-ai): Engineering-intelligence platform that aggregates SDLC telemetry (task trackers, IDEs, CI/CD, VCS, incident systems); publisher of the AI Engineering Impact Reports (2025 Productivity Paradox, 2026 Acceleration Whiplash)
- [OpenAI](https://www.howardism.dev/articles/openai): AI lab and maker of the GPT-5 series and Codex; in this corpus it appears as a frontier-safety research source (Deployment Simulation, deliberative alignment), an agent-tooling source (Codex, Symphony orchestrator, the App Server Protocol, harness engineering), and the company Andrej Karpathy co-founded
- [Peter Steinberger](https://www.howardism.dev/articles/peter-steinberger): Founder of PSPDFKit turned prolific independent AI-coding experimenter (@steipete); originated the framing that loop engineering is built on — "you should be designing loops that prompt your agents"
- [FastContext](https://www.howardism.dev/articles/fastcontext): Microsoft CoreAI + Shanghai Jiao Tong University's open-source repository-exploration subagent (June 2026): trained 4B–30B Qwen-based explorers (Read/Glob/Grep, parallel, compact file-line citations) that decouple repo search from solving; +up to 5.5% SWE-bench resolution, −up to 60% main-agent tokens; code + data released
- [Marcus Hutter](https://www.howardism.dev/articles/marcus-hutter): Creator of AIXI and the Universal AI framework; DeepMind senior researcher and ANU professor; co-author of the Legg–Hutter intelligence measure and the 2026 textbook 'An Introduction to Universal Artificial Intelligence'; co-author of the 'From AGI to ASI' report
- [Perplexity](https://www.howardism.dev/articles/perplexity): AI answer-engine company; maker of Perplexity Deep Research (the leading system on its own DRACO benchmark) and publisher of DRACO; runs Claude Opus 4.5/4.6 as base models inside its orchestration — simultaneously an Anthropic customer and a benchmark competitor
- [Shane Legg](https://www.howardism.dev/articles/shane-legg): Co-founder and Chief AGI Scientist of Google DeepMind; co-author with Hutter of the Legg–Hutter universal intelligence measure; senior author on the 2026 'From AGI to ASI' report
- [Claude Fable 5](https://www.howardism.dev/articles/claude-fable-5): Anthropic's first generally-available Mythos-class model (June 2026) — state-of-the-art on nearly all benchmarks; the same underlying model as Mythos 5 but shipped with classifiers that fall back to Opus 4.8 on cyber/bio-chem/distillation queries; $10/$50 per Mtok; access suspended shortly after launch
- [Claude Mythos 5](https://www.howardism.dev/articles/claude-mythos-5): The safeguards-lifted form of Claude Fable 5 (June 2026): same underlying Mythos-class model, deployed through Project Glasswing with cyber safeguards removed; strongest cybersecurity capabilities of any model in the world, plus autonomous drug-design / genomics results; restricted to trusted-access partners; access suspended shortly after launch
- [Anthropic Institute](https://www.howardism.dev/articles/anthropic-institute): Anthropic's policy/governance research arm; published *When AI builds itself* (Favaro & Clark, 2026) on recursive self-improvement; agenda includes building the verification systems a credible multilateral AI slowdown would require
- [Anthropic Labs](https://www.howardism.dev/articles/anthropic-labs): Anthropic's internal incubator — a 'bet factory' of ~a dozen tiny teams exploring the model frontier with lean-startup loops; origin of Claude Code, MCP, Skills, and Claude Design; led (round 2) by Mike Krieger
- [Claude Design](https://www.howardism.dev/articles/claude-design): Anthropic Labs product for collaborating with Claude on polished visual artifacts — designs, prototypes, slides, decks, animations; research preview ~April 2026, beta on Pro/Max/Team/Enterprise by July 2026; built by ~3 people in ~10 weeks from a designer's side project; multiplayer, round-trip with Claude Code, HTML/CSS/JS export; no image model, not for shipping production software
- [Claude Opus 4.8](https://www.howardism.dev/articles/claude-opus-4-8): Anthropic's most capable general-access model as of May 2026, since superseded by Fable 5 and Opus 5 and now the fallback target for both; upgrade on Opus 4.7 in SWE/agentic/knowledge work; does not advance the frontier beyond Mythos Preview; best-aligned public model of its era, but training surfaced a grader-speculation trend; and the first Anthropic model priced per-task on an outside production codebase ($1.94 at 87% success, tied on quality with a $1.28 open-weight model)
- [Dan Carey](https://www.howardism.dev/articles/dan-carey): Product Manager leading product within Anthropic Labs; led Claude Design; 'Designing with Claude' talk (May 2026); ~two decades of PRDs, now replaced by prototypes
- [METR](https://www.howardism.dev/articles/metr): Independent AI-evaluation org behind the 'time horizons' benchmark — the task length a model can complete reliably on its own; the doubling-every-~4-months trendline and the 'upper end of what we can measure' verdict on Mythos Preview
- [Entities — People, Orgs, Tools & Projects](https://www.howardism.dev/articles/moc-entities): Map of Content for all 79 entity pages. See Home for concept domains.
- [OWASP](https://www.howardism.dev/articles/owasp): Open Worldwide Application Security Project; source of the agentic threat taxonomy cited throughout Anthropic's Zero Trust framework, coined the term 'least agency', and maintains the AI-BOM (CycloneDX ML-BOM extension)
- [AlphaProof Nexus](https://www.howardism.dev/articles/alphaproof-nexus): DeepMind framework for LLM-aided Lean proof generation; four agents (basic→full-featured); proof-sketch + EVOLVE-BLOCK interface; SafeVerify
- [Andrej Karpathy](https://www.howardism.dev/articles/andrej-karpathy): Co-founder OpenAI, ex-Tesla AI, Eureka Labs; coined "vibe coding," Software 1/2/3.0, "ghosts not animals," "agentic engineering"; originated the LLM-wiki pattern this vault runs on — industrialized within ~3 months as 'agent wikis' (DeepWiki, AutoWiki, OpenWiki, GBrain)
- [Campfire](https://www.howardism.dev/articles/campfire): AI-native ERP (YC S23) pulling customers off NetSuite; custom foundation model + agent platform; Series B (Accel/Ribbit); doubling ARR/quarter since Q4 2024
- [Fiona Fung](https://www.howardism.dev/articles/fiona-fung): Leads engineering + product for Claude Code and Cowork at Anthropic (ex-Meta/Microsoft); "what served you prior may no longer"; rewrote team norms for the AI-native org
- [Google DeepMind](https://www.howardism.dev/articles/google-deepmind): Google's AI lab; built AlphaProof Nexus; Gemini models, AlphaProof, AlphaEvolve, and the open-weight Gemma line; opens the AI-for-mathematics domain and (via the Legg/Hutter 'From AGI to ASI' report) the theory-of-superintelligence cluster in this wiki; co-developer of the Cloud agent platform's AutoRater judges — and, across Gemma 4 and the Gemini 3.5 Flash-Lite card, runs two different safety-disclosure regimes: untabulated prose for the open line, a five-row delta table naming its own regression for the closed one
- [John Glasgow](https://www.howardism.dev/articles/john-glasgow): CEO/founder of Campfire; 10yr corporate finance; founder-led-sales advocate; long-horizon "last job I'll ever have"
- [Lean](https://www.howardism.dev/articles/lean): Proof assistant whose compiler mechanically verifies every step; the `sorry` placeholder enables proof sketches; mathlib maturity gates the reachable frontier
- [Claire Vo](https://www.howardism.dev/articles/claire-vo): Host of the "How I AI" interview series (ChatPRD); interviewed Thariq Shihipar; runs a parallel component-visualization practice for non-technical stakeholders
- [Thariq Shihipar](https://www.howardism.dev/articles/thariq-shihipar): Engineer on the Claude Code team at Anthropic; "HTML is the new markdown", "compute allocator", and "the map is not the territory" framings; three HTML-first workflows plus a phase-ordered catalog of techniques for eliciting your own unknowns
- [Thinking Machines Lab](https://www.howardism.dev/articles/thinking-machines-lab): AI research lab behind interaction models (May 2026) and the Inkling open-weights family (July 2026, 975B/41B from scratch); Tinker hosted fine-tuning platform; harness-dissolves-into-model thesis; mission: AI that extends human will and judgment via customization
- [TML-Interaction-Small](https://www.howardism.dev/articles/tml-interaction-small): TML's first interaction model: 276B MoE / 12B active, audio+video+text in / text+audio out, 200ms micro-turns, async background agent; best turn-taking latency of any model; research preview May 2026 — and the exact shape of July 2026's Inkling-Small
- [Chloe Li](https://www.howardism.dev/articles/chloe-li): Lead author of MSM paper (arXiv 2605.02087); Anthropic Fellows Program; designed all specs and experiments
- [Claude's Constitution / Model Spec](https://www.howardism.dev/articles/claude-constitution): Anthropic Model Spec / Constitution by Askell et al.; document specifying Claude's values + hard constraints (SP1–3, GP1–2); now also a direct training input via MSM
- [Anthropic](https://www.howardism.dev/articles/anthropic): AI safety company / vendor of Claude; mission-as-tiebreaker culture; ~30–40 PMs across teams; Mike Krieger leads Labs round 2
- [Boris Cherny](https://www.howardism.dev/articles/boris-cherny): Creator of Claude Code at Anthropic; phone-driven workflow with hundreds of agents; primary advocate of `/loop` primitive; "coding is solved (for me)" thesis; ablation-driven harness design (delete the prompt, add back what the model repeatedly stumbles on)
- [Cat Wu](https://www.howardism.dev/articles/cat-wu): Head of Product for Claude Code and Cowork at Anthropic; primary articulator of AI-native product cadence and engineer-PM convergence
- [Claude Code](https://www.howardism.dev/articles/claude-code): Anthropic's agentic coding product; created by Boris Cherny late 2024; TypeScript/React on Bun (itself Claude-rewritten Zig→Rust); CLI/desktop/web/mobile/IDE surfaces; central tool across all 2026 sources
- [Cowork](https://www.howardism.dev/articles/cowork): Anthropic's non-code knowledge-work agent product; sibling to Claude Code; output is decks/inbox/dossiers; same MCP/computer-use primitives
- [Matt Pocock](https://www.howardism.dev/articles/matt-pocock): Independent AI-coding educator; built Sandcastle library; smart-zone/grill-me/tracer-bullets pedagogical framing; "bad code bases make bad agents"
- [Mythos Model](https://www.howardism.dev/articles/mythos-model): Anthropic preview-tier frontier model and the first member of the Mythos-class tier (above Opus); gated for safety, used internally alongside Opus 4.7; its descendants Fable 5 / Mythos 5 shipped June 2026 as the first general-access Mythos-class models
- [Hermes Agent](https://www.howardism.dev/articles/hermes-agent): Nous Research's CLI agent + Gateway daemon (Telegram/Discord/Slack/WhatsApp); AGENTS.md/SOUL.md context split, bounded memory files, DM-pairing auth, container-as-security-boundary model
- [Symphony](https://www.howardism.dev/articles/symphony): OpenAI's open-source agent orchestrator (March 2026): turns Linear into a control plane for Codex, per-issue workspace, daemon-driven, SPEC.md-as-product, hedged 500% landed-PRs claim
- [Claude Opus 4.7](https://www.howardism.dev/articles/claude-opus-4-7): GA frontier model from Anthropic; direct upgrade to 4.6 at same price; literal instruction following, 1.0–1.35× tokenizer inflation, new `xhigh` effort, first post-Glasswing safeguards

## Syntheses

- [Open Questions Dashboard](https://www.howardism.dev/articles/open-questions): Generated by `_system/lint.py --write-backlog`. Do not hand-edit. Harvested from the `## Open Questions` section of every concept article. Work `#oq/now` items (listed in full below) via `/query`; answered items move to the page's `## Resolved Questions` at the next compile.
- [Open Questions Backlog](https://www.howardism.dev/articles/open-questions-backlog): Generated by `_system/lint.py --write-backlog`. Do not hand-edit. Domain and Watching sections carry one row per page — ×count, oldest bullet age, first-question kernel; the full text lives in the page's `## Open Questions` section, one click away. Bucketing vs. lint's flat open-question count: entity-page questions are split out below as "watching" rather than "actionable", predictions (`#oq/wait`) and notes (`#oq/note`) are listed in their own sections, and partially-answered bullets are counted as "in progress".