# Howardism

> A Taiwan-based Software Engineer, Mathematician, and Amateur Diver sharing personal thoughts and journeys

Articles-only blog. Notes are organized into knowledge domains below; the full index is at https://www.howardism.dev/articles.

## Agent Systems

- [Agent Systems & Harness Engineering](https://www.howardism.dev/articles/moc-agent-systems): Map of Content for the agent-systems domain — 23 concepts. Harness engineering, agent loops and orchestration, context management, protocols and tool infrastructure (MCP, app servers), and subagents. Curated entry point; see Home for all domains.
- [Latent vs. Deterministic Space](https://www.howardism.dev/articles/latent-vs-deterministic-space): Garry Tan's diagnostic for agent-system bugs: computation lives in two places — latent space (the LLM: taste, judgment, vague-intent interpretation, steered by markdown) and deterministic space (generated code, external state) — and most AI-engineering failures are computation happening on the wrong side
- [Agent Quality Flywheel](https://www.howardism.dev/articles/agent-quality-flywheel): Google's eval-fix loop packaged as a skill your coding agent drives: Build & Test → Ship & Monitor → Learn & Refine, expanded into five stages (prepare data / run inference / grade / analyze failures / optimize); plain-language worry in, metric choice and before/after deltas out; synthetic User Simulator bootstraps, production OTel traces sharpen
- [Failures That Look Like Success](https://www.howardism.dev/articles/failures-that-look-like-success): The quiet agent-failure class where everything reads fine — confident answer, plausible plan, even correct internal state — but the user-facing outcome is wrong; Google's flywheel demos caught agents echoing stale values despite correct memorize calls and silently skipping self-report instructions; detectable by trace-level rubrics, not output skims
- [Optimizer–Evaluator Decoupling](https://www.howardism.dev/articles/optimizer-evaluator-decoupling): The architectural rule in eval-fix loops that whatever proposes a fix (coding agent, automated optimizer, human) never grades it — an independent evaluation service scores the result, because an optimizer that grades its own work learns to game the metric instead of improving the agent
- [Parallel Agent Orchestration](https://www.howardism.dev/articles/parallel-agent-orchestration): OpenAI Codex study's concurrency + runtime margins: the intensive-user workflow where a human oversees a team of agents rather than doing the work directly — 28.6% of OpenAI users peaked at 5+ concurrent agents in a week (vs ~64-67% of external users running zero concurrent), and p99 OpenAI users ran ~71 agent-hours/day (up 88% since April 2026); the threaded interaction model lets one person delegate, monitor, and review many simultaneous streams — first hard numbers behind founder-as-agent-orchestrator
- [Loop Engineering](https://www.howardism.dev/articles/loop-engineering): Replacing yourself as the agent's prompter by designing the system that prompts it: a recursive-goal loop built from five product-native primitives (automations, worktrees, skills, connectors, sub-agents) plus external memory; tool-agnostic across Codex and Claude Code; the leverage point moves from prompt-crafting to loop-design
- [Repository Exploration Subagent](https://www.howardism.dev/articles/repository-exploration-subagent): FastContext's thesis that repository exploration (read/search/localization) should be decoupled from solving into a dedicated read-only subagent that issues parallel tool calls and returns compact file-line citations, keeping the solver's context clean — cutting main-agent tokens up to 60% and lifting SWE-bench resolution up to 5.5%
- [Deep Research Agents](https://www.howardism.dev/articles/deep-research-agents): Agentic systems that decompose a complex query, iteratively search diverse sources, and synthesize a structured, cited report — distinct from single-shot QA; DRACO shows orchestration (Perplexity) beats the bare base model with tools, and factual accuracy is the weak axis
- [Build for the Next Model](https://www.howardism.dev/articles/build-for-the-next-model): Prototype the thing that almost works, not the thing that already works: bet that the next concrete model release (not a far-future AGI) fixes what your engineering can't; Claude Design's Opus 4.7 payoff and OpenAI's 'the February Codex app would have failed in November' are the cleanest cases — same product shape, different-intelligence release, different outcome
- [Agent Context Files](https://www.howardism.dev/articles/agent-context-files): The cross-vendor markdown-as-control-plane pattern: repo-versioned plaintext (CLAUDE.md / AGENTS.md / SOUL.md / WORKFLOW.md / SPEC.md / .cursorrules) that configures agent behavior, split by role across project / personality / workflow / spec layers
- [Agent-Native Infrastructure](https://www.howardism.dev/articles/agent-native-infrastructure): The world is still built for humans and must be rewritten for agents; "what do I copy-paste to my agent?"; sensors/actuators; agent-to-agent representation
- [MCP and Computer Use](https://www.howardism.dev/articles/mcp-and-computer-use): Anthropic's two complementary connector mechanisms: MCP for structured programmatic access (Salesforce/Drive/Gmail/Slack/Figma + niche industry systems); computer use as the GUI-driving catchall when no MCP exists; Boris Cherny's "to the model, it's just tokens"
- [Agent Loop Pattern](https://www.howardism.dev/articles/agent-loop-pattern): `/loop` (cron-scheduled) and Ralph Wiggum (backlog-draining) loops as next-generation agent primitive; AFK execution, parallel fan-out, "loops are the future"
- [Context Window Smart Zone](https://www.howardism.dev/articles/context-window-smart-zone): Smart zone vs dumb zone (Dex Hardy / Matt Pocock): quadratic attention scaling, ~100K marker independent of advertised context; clear-and-restart > compaction; status-line token counting as essential discipline
- [Deep Modules for Agents](https://www.howardism.dev/articles/deep-modules-for-agents): Ousterhout deep-vs-shallow modules applied to agent-friendly codebases; push-vs-pull instruction delivery; reviewer in fresh context; Sandcastle three-agent pattern
- [Harness Shrinkage as Models Improve](https://www.howardism.dev/articles/harness-shrinkage-as-models-improve): Prompt scaffolding shrinks each model release; Cat Wu's pruning discipline; Boris Cherny "100 lines of code a year from now" claim; mechanical verification stays load-bearing
- [Codex App Server Protocol](https://www.howardism.dev/articles/codex-app-server-protocol): JSON-RPC stdio protocol for headless Codex sessions: initialize/initialized/thread-start/turn-start handshake, continuation turns reuse thread_id, dynamic tool calls for token-isolated tool injection
- [Ticket-Driven Agent Orchestration](https://www.howardism.dev/articles/ticket-driven-agent-orchestration): The inversion that makes Symphony work: tickets as units of work (not sessions/PRs), DAG dependencies, agent-extensible work graph, "objectives not transitions"
- [Claude Code Auto Mode](https://www.howardism.dev/articles/claude-code-auto-mode): Claude Code permission mode using a classifier to auto-approve safe tool calls and block risky ones; middle ground between default and `--dangerously-skip-permissions`
- [Client-Side Agent Optimization](https://www.howardism.dev/articles/client-side-agent-optimization): AgentOpt's framing of developer-controlled agent optimization (model-per-role, budget, routing) as distinct from server-side serving; the combo abstraction; 13–32× cost gaps between best/worst combinations
- [Agent Harness Engineering](https://www.howardism.dev/articles/agent-harness-engineering): Patterns for scaffolding long-running LLM agents: environment design, progressive context disclosure, mechanical architecture enforcement, agent code review
- [Claude Code Best Practices](https://www.howardism.dev/articles/claude-code-best-practices): Anthropic's guide to effective Claude Code usage: context management, verification-driven development, explore→plan→code workflow, environment config
- [LLM-as-Compiler Knowledge Base](https://www.howardism.dev/articles/llm-as-compiler-knowledge-base): Karpathy's architecture: LLM incrementally compiles raw docs into a persistent interlinked wiki, replacing RAG with a 4-phase ingest→compile→query→lint pipeline

## Agent Security

- [Agent Security](https://www.howardism.dev/articles/moc-agent-security): Map of Content for the agent-security domain — 18 concepts. Attacks and defenses for agentic systems: prompt and data injection, tool and memory poisoning, identity and authorization, and zero trust. Curated entry point; see Home for all domains.
- [Capability Gating Is Not Authorization](https://www.howardism.dev/articles/capability-gating-vs-authorization): Mellafe Zuvic (arXiv 2606.28679): popular agent frameworks (LangChain/LangGraph, LlamaIndex, Stripe Agent Toolkit) ship *capability gating* (which tools are exposed + schema validity) but no default fail-closed *per-call authorization* of concrete argument values — so a compromised/confused-deputy agent executes unauthorized actions that lie *within* its granted capability scope (a well-typed account=acct_ATTACKER passes). Proposes ScopeGate, a deterministic 5-stage PDP/PEP (scope → authz → money ceiling → idempotency → default-deny) that re-authorizes each model-emitted call against out-of-band operator policy: 0/48 static bypasses, 0/29 over a 40-iteration adaptive run, 0 false-denies; cost-optimized deployment-tier models attempt the unauthorized call ~3.2× more than flagships (mean ASR 0.603 vs 0.189)
- [MCP Tool Poisoning](https://www.howardism.dev/articles/mcp-tool-poisoning): The MCP Tool Poisoning Attack (TPA) class — an adversarial third-party MCP server embeds malicious instructions in tool metadata (descriptions/schemas → TDPA) or tool returns (TRPA/MERA), exploiting the model's inherent trust in tool descriptions — anchored by ShareLock (Liu et al., arXiv 2606.27027), the first threshold variant: it uses Shamir (t,n) secret-sharing to fragment a payload into benign-looking tool_id/checksum shares spread across multiple tools (each information-theoretically revealing nothing alone and passing GPT-5/Claude/Gemini classifiers + entropy detectors), then a rug-pull server update plants an EnvSetup trigger that makes the agent aggregate ≥t shares and Lagrange-reconstruct the instruction at runtime; >90% avg ASR / 96.4% TCR across 4 models on two MCP clients while single-tool plaintext baselines are flagged Unsafe — proving single-tool description scanning structurally insufficient, and that static vetting misses it because even Claude, which flags the isolated trigger, overlooks the runtime threat during multi-tool execution — now paired with the in-the-wild **Agentjacking** case study (Tenet Security, `case-study`), a *distinct branch* of the MCP attack surface where a **legitimate** Sentry MCP server relays attacker-injected fake-error data to coding agents as trusted diagnostics (malicious-data-via-legit-server, vs ShareLock's poisoned-tool-metadata), triggering plaintext RCE
- [Non-Malleable Memory Authority (TMA-NM)](https://www.howardism.dev/articles/non-malleable-memory-authority): Yedidel Louck (arXiv 2606.24322): existing agent-memory defenses decide a memory item's authority-to-act from a *malleable* signal — its content (trust-scoring) or a derivation edge (lineage) — that an adversary *launders* through three LLM-specific channels (self-summarization, trusted-tool echo, manufactured corroboration), erasing the item's untrusted origin so poisoned memory reads benign and inherits trusted provenance; a machine-checked TLA+ separation theorem proves content/lineage gates unsound under laundering (T1), write-time origin binding necessary (T2), and non-malleable origin-bound authority with Sybil-resistant corroboration-gated elevation sufficient (T3); the TMA-NM construction (write-time origin binding + non-malleable propagation + ≥2-independent-trusted-principal elevation + tamper-evident log) instantiates non-malleable information-flow control for agent memory and holds at 0% attack-success on the direct attack and all three laundering channels across eight frontier models at 100% legit-utility (1.3µs/decision), while baselines fail exactly where the theorem predicts (up to 68% laundering, 84% direct)
- [Off-Host, Identity-Bound Authorization](https://www.howardism.dev/articles/off-host-identity-bound-authorization): Sai Varun Kodathala (arXiv 2607.05518, single-author preprint): aiAuthZ, an authorization GATEWAY that runs OFF the agent's host in a separate trust domain — it authenticates the human sender of each message with a per-message HMAC-SHA256 signature (single-use nonce + timestamp window) and evaluates a role + argument-level policy the agent can neither read nor modify, so a compromised/injected agent cannot forge authority through unverified tool-call text (a call's authority derives from the most-recently-verified human message, not from text the model read). Across 15 LLMs model-level refusal ranges 100%→38% and is not ordered by price; with the gateway residual attack success falls to 0% at ≤0.03 ms added latency; on 9 in-scope Agents-of-Chaos cases it blocks 9/9 vs 4/9 for an argument-only (OAP-style) design and 0/9 for a delegation-token (AIP-style) design; on AgentDojo banking it blocks all 7 attacker-directed calls (spotlighting lets 2 injections through) at the cost of one blocked legit first-time payment; the HMAC-QR receipt verifies 94% across channels with 0/25 forgeries — the off-host, identity-bound counterpart to ScopeGate's in-framework PDP/PEP
- [Task-Specification Effects in Prompt Injection (AutoDojo)](https://www.howardism.dev/articles/task-specification-injection-surface): AutoDojo (Ma et al., arXiv 2606.15057): a cheap black-box adaptive attack that iteratively optimizes an indirect prompt injection against a live defended agent using only the success/fail signal — recovering 28% overall ASR (64% on action-open tasks) against a filter that scores 0% *static* ASR, so static-benchmark robustness dramatically overstates real robustness; plus the task-specification axis it exposes — under-specified 'action-open' tasks (the user defers the action itself to attacker-reachable content) are markedly more injectable than fully-specified ones for prompt- and filter-based defenses, while action-constraining system-level defenses invert this and grow stronger
- [Agent Data Injection (ADI)](https://www.howardism.dev/articles/agent-data-injection): A new category of indirect prompt injection: malicious payloads disguised as *trusted data* (metadata like a comment's author, a UI element ID, or the tool-call history) rather than as instructions, via probabilistic delimiter injection — the LLM misreads inexact/escaped delimiters as structural boundaries, so the agent does the user's task but on attacker-forged data; working RCE and supply-chain exploits on Claude Code / Codex / Gemini CLI and arbitrary-click on Claude-in-Chrome, bypassing IPI defenses that only separate instructions from data (up to 50% ASR where instruction injection is ~0%)
- [Agent Identity Management System (AIMS)](https://www.howardism.dev/articles/agent-identity-management-system): IETF draft-klrc-aiagent-auth's Agent Identity Management System: agents as WIMSE/SPIFFE-identified workloads with short-lived posture-assessed credentials, mTLS / WIMSE-proof-token / HTTP-message-signature authentication, and OAuth token-exchange + transaction-token delegation chains for spawned agents — the LLM never holds credentials; the vault's first multi-vendor, standards-track agent-auth source, but an individual submission with no IETF WG consensus yet — complemented by the OpenID AuthZEN Working-Group Drafts (AARP, a prerequisite/approval pattern generalizing CIBA beyond client-initiated flows; COAZ, an MCP-tool-authorization profile mapping tool calls into AuthZEN's Subject-Action-Resource-Context model), the authorization-slice standardization showing the agent-auth governance layer is plural (IETF identity/delegation + OpenID authorization) and actively moving
- [Out-of-Band Prompt-Injection Defense](https://www.howardism.dev/articles/out-of-band-prompt-injection-defense): The second-generation defense strategy that enforces agent security OUTSIDE the model — a deterministic reference monitor mediating tool calls (CaMeL, FIDES, Progent, RTBAS, FORGE) instead of training the model to refuse — reframed through classical primitives (Biba integrity, reference monitors, least privilege) and warned to be validated only on static benchmarks; a first independent adaptive-attack reproduction on an open-weight Qwen2.5-7B agent held (ASR 25.8%→4.2%; 2.6% under a hand-crafted adaptive attack)
- [Agent Identity and Authentication](https://www.howardism.dev/articles/agent-identity-and-authentication): The foundation control for agentic Zero Trust: cryptographically-rooted per-agent identity (→X.509→hardware attestation), short-lived IdP-issued tokens replacing static API keys (→mTLS→hardware-bound credentials), JIT access and ABAC
- [Agent Supply Chain Risk](https://www.howardism.dev/articles/agent-supply-chain-risk): Runtime-composed agent ecosystems expand the supply-chain attack surface: model poisoning (250 docs backdoor a 13B model), tool/MCP supply chain (first in-the-wild malicious MCP server), AI-BOM, OpenSSF Scorecard, dependency audits, and AI vendoring as remediation
- [Agentic Prompt Injection](https://www.howardism.dev/articles/agentic-prompt-injection): Direct and indirect injection of malicious instructions into an agent; LLMs cannot reliably distinguish information from instructions; defenses are spotlighting (50%→<2%), constitutional classifiers (95% blocked), input isolation, and attack-surface reduction — but a second IPI category, agent data injection, forges *trusted data* rather than instructions and slips past all of them
- [AI-Accelerated Offense](https://www.howardism.dev/articles/ai-accelerated-offense): Frontier models compress the vulnerability-to-exploit timeline from months to hours at marginal dollar cost; both attackers and defenders speed up, the N-day window collapses, and the differentiator becomes strong fundamentals + breach-ready architecture
- [Autonomous Defense](https://www.howardism.dev/articles/autonomous-defense): Running security operations at the speed of AI-accelerated threats: put a model at the front of the alert queue, automate the bookkeeping (not the decisions), Agentic SOAR, MITRE ATT&CK coverage mapping, and rehearse five simultaneous incidents
- [Blast Radius (Agentic)](https://www.howardism.dev/articles/blast-radius): The potential damage if an agent is compromised; the unit Zero Trust's 'assume breach' posture is built to contain via identity-based isolation, sandboxing, and compartmentalization
- [Impossible, Not Tedious (Design Test)](https://www.howardism.dev/articles/impossible-not-tedious-test): Zero Trust design test for agentic security: does a control make the attack impossible, or just tedious? Friction-only controls degrade against agentic attackers with unlimited patience and near-zero per-attempt cost
- [Least Agency](https://www.howardism.dev/articles/least-agency): OWASP term extending least privilege to agents: constrain not just what an agent can access but what each tool can do, how often, and where; deny-by-default, per-agent credentials, scope limits
- [Memory and Context Poisoning](https://www.howardism.dev/articles/memory-and-context-poisoning): Corruption of persistent agent memory that influences behavior long after the initial injection; includes RAG poisoning, shared-context poisoning, and slow long-term memory drift; defended via memory isolation, integrity validation, and retention policies
- [Zero Trust for AI Agents](https://www.howardism.dev/articles/zero-trust-for-ai-agents): Anthropic's security framework for deploying autonomous agents: trust nothing / verify everything / assume breach, applied across a Foundation→Enterprise→Advanced tier model and an 8-phase implementation workflow

## AI Coding Practice

- [AI Coding Practice](https://www.howardism.dev/articles/moc-ai-coding-practice): Map of Content for the ai-coding-practice domain — 23 concepts. How humans and teams practice AI-assisted software work: workflow techniques, SDLC telemetry, review and verification as the bottleneck, and division of labor. Curated entry point; see Home for all domains.
- [Configurable Human Participation](https://www.howardism.dev/articles/configurable-human-participation): HAS-Bench (Wu et al., arXiv 2607.04329): a benchmark that makes human participation a *configurable* variable — humans and LLM agents are first-class nodes in a shared interaction graph, participation varied along three axes (five-level Human Agency Scale A1–A5, three interaction channels clarification/feedback/control, and user personas) across 397 tasks in six domains, scoring both outcomes and process. Finding: equal partnership (A3) beats full automation (A1) by +8.4 Pass@1 / +11.5 Task Score and recovers up to 65.4% of autonomy-failed tasks, but the value is configuration-dependent, not monotonic — more agency (A4) brings diminishing and sometimes negative returns (rescues 27 tasks / breaks 13 via premature or distracting intervention), each problem pattern has a different best channel (clarification before commit, feedback after, control-only for safety-critical authorization = 100% safe), Full-channel is beaten by the best single channel in 5/6 patterns, and a weak agent (Llama-3.1-8B) gains ~0 — the agent must decide *when* to ask and *how* to use input, not just get more of it; all with an LLM-simulated (GPT-4.1) human, a controlled benchmark not field oversight-load telemetry
- [LLM-Assisted Grey-Literature Theory Building](https://www.howardism.dev/articles/llm-assisted-grey-literature-theory-building): Agarwal et al.'s secondary contribution (arXiv 2607.07980): a scalable template for constructing grounded theory from thousands of practitioner documents instead of a few dozen interviews — LLMs do the mechanical, quote-anchored open coding (38,709 docs collected → Gemini relevance judge at κ=0.75 → 3,100 coded with the multi-agent Thematic-LM under three deliberately-polarized coder lenses → 4,838 codes / 109,951 quotes at ~$0.35/doc) while humans keep the interpretive axial/selective coding; automating that back half FAILED (a bottom-up pass yielded 15,029 shallow, redundant causal statements), so the codes→theory step stayed a manual, LLM-as-search-engine process — a division-of-labor lesson about what LLMs can and can't do in qualitative research
- [Review as the Control Point](https://www.howardism.dev/articles/review-as-the-control-point): Agarwal et al. (CMU, arXiv 2607.07980): a 26-construct/67-relationship causal theory synthesized from 3,100 coded practitioner documents — review is the control point through which a coding agent's effect on software is decided, and AI does NOT fix the sign of that effect; the team sets it through reviewer expertise, disposition, and how it adapts the review process (three moderators). Central core is review depth + reviewer skill, threaded by comprehension-debt feedback loops; the paper's own non-vendor GitHub telemetry (2.5M+ PRs) finds agent PRs reviewed less / merged several× faster / discussed less, but the trends flip direction under defensible analysis choices and the no-review rate CONVERGES toward the human baseline over time
- [The Three Loops of AI-Native Building](https://www.howardism.dev/articles/three-loops-of-ai-native-building): Andrew Ng's nested-loop taxonomy for 0-to-1 products: the agentic coding loop (minutes, agent-closed), the developer feedback loop (tens of minutes to hours, human-closed), and the external feedback loop (hours to weeks, market-closed); loop engineering has been optimizing only the innermost one, and the human's remaining job is a context transfer that lives in the outer two
- [Unknowns as the Agentic Bottleneck](https://www.howardism.dev/articles/unknowns-as-the-agentic-bottleneck): Thariq Shihipar's map-vs-territory thesis: the gap between what you told the agent and what the work actually requires is *unknowns*, and with Fable-class models the human's ability to surface them — not the model's capability — sets output quality; the Rumsfeld 2×2 applied to prompting, plus a phase-ordered catalog of elicitation techniques
- [Agentic Work Systematization](https://www.howardism.dev/articles/agentic-work-systematization): OpenAI Codex study's 'systematization' margin: the shift from ad-hoc agent use (describe task → agent does it → done) to reusable workflow infrastructure via skills and plugins; skill use rose 5.4%→26.6% of weekly-active users (Mar→Jun 2026) and is near-universal at OpenAI (96.2%); custom skills (org-specific procedural context) concentrate where shared conventions and team standards exist — the empirical counterpart to loop-engineering's skills primitive
- [Acceleration Whiplash](https://www.howardism.dev/articles/acceleration-whiplash): Faros 2026: AI floods a human-paced SDLC with output it can't absorb — throughput up (tasks +34%, epics +66%), quality down (bugs +54%, incidents/PR +243%, review time 5x), gap widening with adoption and hitting even high-maturity orgs
- [Agentic Coding Work-Composition Shift](https://www.howardism.dev/articles/agentic-coding-work-composition-shift): Anthropic's 400K-session telemetry, Oct 2025→Apr 2026: as models improved, the share of sessions fixing broken code fell 33%→19% (debugging nearly halved), while operating software (14%→21%) and writing+data-analysis (~10%→~20%) grew; estimated task value rose ~25–27% — usage moving from firefighting toward end-to-end agentic work
- [AI as Primary Author](https://www.howardism.dev/articles/ai-as-primary-author): Faros 2026: the assistant→author threshold crossed without a deliberate decision, marked by AI-code acceptance rising 20%→60%; 'not an assistant, the author'; humans move from creation to oversight, making it an authoring problem not a review problem
- [Planning / Execution Division of Labor](https://www.howardism.dev/articles/planning-execution-division-of-labor): Anthropic's 400K-session telemetry: in a typical Claude Code session humans make ~70% of planning decisions (what to do) while Claude makes ~80% of execution decisions (how to do it); each prompt sets off ~10 actions (8 when the user keeps execution control, ~16 when Claude controls planning) — 'people decide what to build, the agent decides how'
- [Telemetry vs. Survey Measurement](https://www.howardism.dev/articles/telemetry-vs-survey-measurement): Faros 2026: perception lags reality, so survey-based engineering research (DORA) misses downstream AI damage that system telemetry catches in near-real-time; the basis for Faros's direct contradiction of DORA's 'strong foundations protect you' conclusion
- [Building Is Cheap, Arguing Is Expensive](https://www.howardism.dev/articles/building-is-cheap-arguing-is-expensive): "In technical debate, code wins": generate three PRs vs whiteboard; prototype over design doc; reduce design docs
- [Code as Source of Truth](https://www.howardism.dev/articles/code-as-source-of-truth): Docs go stale at high coding throughput; check specs/skills into the repo; onboard via Claude; spec-drift verification
- [Outsource Your Thinking, Not Your Understanding](https://www.howardism.dev/articles/outsource-thinking-not-understanding): "You can outsource your thinking but not your understanding"; understanding as the non-delegable human bottleneck; knowledge bases as understanding-tools
- [The Verifiability Thesis](https://www.howardism.dev/articles/verifiability-thesis): LLMs automate what you can *verify* as computers automate what you can *specify*; RL verification rewards → jagged peaks; "verifiable + labs care"; everything eventually verifiable
- [Verification as the New Bottleneck](https://www.howardism.dev/articles/verification-as-the-new-bottleneck): Fiona Fung: coding is no longer the bottleneck — verification, review, maintenance are; shift-left; TDD loses its tax; PR-cycle-time funnel analysis
- [Vibe Coding vs. Agentic Engineering](https://www.howardism.dev/articles/vibe-coding-vs-agentic-engineering): Vibe coding raises the floor (anyone builds); agentic engineering preserves the quality bar while going faster; ">10x and widening"; hire on big projects, not puzzles
- [Compute Allocator](https://www.howardism.dev/articles/compute-allocator): The human's evolving role: deciding what's worth spending compute on; ~1% of generated tokens ship, 99% is scaffolding invested in alignment/communication; abundance mindset
- [Disposable Micro-Apps](https://www.howardism.dev/articles/disposable-micro-apps): Throwaway custom UIs built per-task to edit a plan ("micro-software on top of micro-software"); copy-back-to-markdown; rational under the abundance mindset
- [HTML as the New Markdown](https://www.howardism.dev/articles/html-as-the-new-markdown): Thariq Shihipar's thesis: as models improve, thousand-line markdown plans overwhelm the *human*; HTML artifacts (visual, interactive) keep humans in the loop. The model-facing harness shrinks while this human-facing harness grows
- [Living Design System](https://www.howardism.dev/articles/living-design-system): `design_system.html` extracted from repos as a portable, human- and machine-readable source of truth; component playgrounds; bridges engineering ↔ non-technical stakeholders
- [Design Concept Grilling](https://www.howardism.dev/articles/design-concept-grilling): Matt Pocock's `grill-me` skill; reach Brooks "design concept" before any plan; counter to specs-to-code; PRD as destination doc, Kanban as journey doc
- [Vertical Slice Tracer Bullets](https://www.howardism.dev/articles/vertical-slice-tracer-bullets): Pragmatic-Programmer tracer-bullet pattern applied to agent task decomposition; vertical slices > horizontal layers; Kanban-with-blocking-edges over numbered phase plans

## Evals & Benchmarks

- [Evals & Benchmarks](https://www.howardism.dev/articles/moc-evals-and-benchmarks): Map of Content for the evals-and-benchmarks domain — 11 concepts. The science of measuring models: benchmark validity, contamination, saturation, LLM judges, and production-sourced evaluation. Curated entry point; see Home for all domains.
- [Benchmark Contamination and Decontamination](https://www.howardism.dev/articles/benchmark-contamination-decontamination): Sun, Zhan & Gales (Cambridge, arXiv 2606.23313): benchmark data contamination inflates reported LLM scores when test samples leak into training, and existing decontamination methods are judged only by aggregate accuracy and mostly need a clean reference model. Two contributions: (1) a sample-level evaluation framework — per-sample distribution distances (D_KL and D*_L1 between the decontaminated and an uncontaminated model's output distributions) that expose a fundamental disconnect (paraphrase+permutation cuts dataset-level residual contamination 17.2→8.4 while D_KL actually rises >13%); (2) Uncertainty-Based Decontamination (UBD), which needs no clean reference model and no knowledge of which samples are contaminated — deep ensembles of the contaminated model (5 LoRA seeds differing only in batch ordering) expose memorized samples as high-confidence-but-high-variance (epistemic/knowledge uncertainty), because memorization is batch-order-sensitive. A per-sample scalar α̂ ≈ 1−2σ (σ = ensemble std of the ground-truth probability, above a confidence threshold T_p) drives either UBD-Debiasing (post-hoc suppression of the inflated ground-truth mass, redistributed proportionally, no weight update, MCQ-only) or UBD-Unlearning (fine-tune toward the debiased soft target; no-op when α̂≈1). On MMLU-Pro/MATH-MCQA it gives >40–60% relative D_KL reduction and beats paraphrase/permutation and even the reference-model DeconIEP baseline; ensemble uncertainty tracks oracle contamination at PCC 0.8–0.9 where log-probability stays <0.4
- [Benchmark Score Redundancy](https://www.howardism.dev/articles/benchmark-score-redundancy): Zeng & Papailiopoulos (Microsoft Research, arXiv 2606.24020): an 84-model × 133-benchmark public score matrix (2,604 cells, 23.3% filled) is effectively rank-2 — a model's scores across all 133 benchmarks are largely two numbers (two diagnostics agree: Soft-Impute completion bottoms at rank 2, and top-2 SVD components explain >90% of cross-model variance in every fully-observed submatrix), echoing prior g-factor / 'general-capability + provider-residual' findings on a heterogeneous frontier-era matrix. BenchPress (logit-space rank-2 bias-decomposed ALS matrix completion) predicts every held-out cell to 4.6 MedAE points at 100% coverage; a compact 5-benchmark probe set {GPQA-D, HLE, Codeforces, MMLU-Pro, ARC-AGI-1} recovers a model's full scorecard to 3.93 points (low-cost set 4.55), predictions preserve 92.1% of ≥5-point pairwise model rankings, and a strict-temporal-cutoff new model needs only 5 seed scores to reach 4.83 points — so you can infer most of a scorecard from a handful of probes rather than run every eval, with a hybrid ensemble-spread + matrix-support reliability layer flagging which predictions to trust; but scores are inferable, not benchmarks unnecessary, the rank-2 geometry is snapshot-specific, and the matrix is the same uncontrolled public grid the compute-control critique warns about
- [Measuring Beyond Accuracy Saturation](https://www.howardism.dev/articles/measuring-beyond-accuracy-saturation): Nadgir, Kapoor, … Narayanan (Princeton-led, 14 authors, arXiv 2606.26158): when a benchmark's accuracy saturates (top agents statistically indistinguishable), the field's reflex is retire-and-replace with a harder successor — but that privileges accuracy and throws away six other measurable axes. Using CORE-Bench Hard (computational reproducibility of scientific code) as a case study, they *decouple accuracy saturation from benchmark saturation*: (1) saturation itself surfaces construct-validity threats invisible to weaker agents — log analysis via Docent found 15 task-level errors + 20 exploitable shortcuts, yielding corrected CORE-Bench v1.1 (39 tasks) and an OOD suite CORE-Bench OOD (19 tasks, new fields), yet accuracy still saturates (top agent 100%, next four tie ~97.4%); (2) statistically-indistinguishable agents still differ sharply in reliability (more-accurate agents are more consistent; all are massively underconfident, ~93% pass vs 32.1% stated confidence, and none beats random at telling its own correct from incorrect runs), efficiency (GPT-5.3-Codex ~60% cheaper than an equal-accuracy peer; token-cost and dollar-cost tell different stories), and model-vs-scaffold contribution (scaffold swings accuracy ~44pp on one model; two scaffolds on the same model disagree on 31% of tasks but an oracle router hits 100%; direct fixes beat rewrites 95%/68%); (3) a small randomized human study finds agent collaboration more than halves reproduction time (2.11×, p≈0.002, likely an underestimate since 5/25 manual runs hit the 3-hour wall and no collaborative run did). Thesis: don't retire a saturated benchmark — re-instrument it
- [Reference-Free Judge Over-Crediting](https://www.howardism.dev/articles/reference-free-judge-over-crediting): Kranti & Vajjala (arXiv 2607.12885): the presence and placement of a reference answer in the prompt is a first-order determinant of an LLM judge's correct/incorrect verdicts — in no-reference settings judges systematically over-credit incorrect answers, and adding the reference flips up to 85% of verdicts (most NR→RV flips are CORRECT→INCORRECT over-credit retractions), an effect that holds across English/Arabic/Telugu but is far worse in low-resource Telugu; a two-stage calibration (C1 vs C2: can the judge tell right from wrong) + sensitivity (NR/RV/RC: does reference presence flip verdicts) pipeline diagnoses judge suitability before reference-free deployment, and human annotation confirms the reference-driven stricter verdicts align better with people (agreement jumps ~0.34→0.85 NR→RV for Gemma), so reference-free absolute scores are genuinely inflated not merely different — a reliability-without-validity instance orthogonal to Norman et al.'s, on three open/open-weight judge models at temperature 0 in binary QA
- [LLM-Judge Validation](https://www.howardism.dev/articles/llm-judge-validation): UC Berkeley's 21-judge / 9-provider / ~541K-judgment audit (Norman et al., 2026): LLM-as-a-judge validation is systematically under-rigorous — exact-match agreement overstates chance-corrected κ by 33–41pp (kappa deflation, universal across every judge), judge rankings shift up to 14 positions across benchmarks, and high test-retest reliability masks severe position bias (the consistency–bias paradox); distilled into a 5-step Minimum Viable Validation Protocol
- [Compute-Controlled Benchmarking](https://www.howardism.dev/articles/compute-controlled-benchmarking): Noam Brown's critique that the single-number 'benchmark grid' is broken because it doesn't control for test-time compute; the fix is to plot performance against a cost/token/time x-axis; benchmark-maxxing, held-out private sets, the Goodhart bad-equilibrium that keeps the grid alive, why routing/consensus must be judged at equal budget — and Gemma 4's headline table as the worked example, benchmarking a thinking model against a non-thinking predecessor
- [DRACO Benchmark](https://www.howardism.dev/articles/draco-benchmark): Perplexity's benchmark of 100 production-sourced deep-research tasks (10 domains, 40 countries) graded by 26-expert rubrics on accuracy/completeness/objectivity/citation; Perplexity Deep Research leads every domain and axis, Claude Opus 4.6 is the strongest non-Perplexity system, factual accuracy is the universal weak spot
- [LLM-as-a-Judge](https://www.howardism.dev/articles/llm-as-a-judge): Using one LLM to grade another's outputs against criteria/rubrics; DRACO's protocol is per-criterion binary MET/UNMET + justification, weight-aggregated into normalized score and pass rate; key properties — rankings stay stable across judge models while absolute magnitudes vary, and adaptive per-case rubrics (Google's AutoRaters) detect failures but blend them away, motivating stable custom metrics for the behavior under change; a 21-judge / ~541K-judgment audit finds raw exact-match agreement overstates chance-corrected reliability by 33–41pp (kappa deflation) and high test-retest can mask severe position bias, so judges need chance-correction, bias, and cross-benchmark validation before thresholded use
- [Production-Sourced Evaluation](https://www.howardism.dev/articles/production-sourced-evaluation): Building benchmarks from de-identified real production usage rather than synthetic or hand-authored tasks; DRACO's central method — difficulty-proxied sampling, PII-stripping, augmentation, automatable refresh with a human QA gate; representativeness vs. over-specification tradeoff; production traffic as a proprietary eval asset
- [Task Time-Horizon Scaling](https://www.howardism.dev/articles/task-time-horizon-scaling): METR's measure of the task length AI can complete reliably on its own, doubling roughly every 4 months (up from every 7): Opus 3 ~4min (Mar 2024) → Opus 4.6 ~12hr (2026) → weeks projected for 2027; paired with benchmark saturation (SWE-bench, CORE-Bench)
- [Scale-Dependent Prompt Sensitivity](https://www.howardism.dev/articles/scale-dependent-prompt-sensitivity): Large models underperform small ones on 7.7% of standard benchmarks due to overthinking; brevity constraints recover 26pp and fully reverse hierarchy on GSM8K/MMLU-STEM

## Model Capability & Training

- [Model Capability & Training](https://www.howardism.dev/articles/moc-model-capability-and-training): Map of Content for the model-capability-and-training domain — 11 concepts. What makes models capable: test-time compute, capability overhangs, RL post-training methods, inference efficiency, and the open-weight frontier. Curated entry point; see Home for all domains.
- [Asynchronous RL for LLMs](https://www.howardism.dev/articles/asynchronous-rl-for-llms): Consuming rollouts for training the instant each finishes, instead of waiting for a full synchronized batch — fixes the straggler idle that long-tail agentic/coding rollouts inflict on a GPU cluster, but pays for it in policy lag and off-policy drift; SAO's DIS (direct double-sided importance sampling) stabilizes it by dropping the old-policy model entirely and masking any token whose rollout-vs-current probability ratio leaves a strict trust region
- [Group Relative Policy Optimization (GRPO)](https://www.howardism.dev/articles/group-relative-policy-optimization): DeepSeek's critic-free RL objective that became the 2024–25 default for LLM post-training: sample a group of responses per prompt, use the group's mean reward as the baseline, and optimize the clipped PPO surrogate with no value network — cheaper and more stable than PPO in synchronous training, but the group is an implicit synchronization barrier (updates wait for the slowest member) that mismatches asynchronous and single-trajectory online agentic settings, which is the gap SAO exploits
- [Single-Rollout Optimization](https://www.howardism.dev/articles/single-rollout-optimization): SAO's headline move: one rollout per prompt instead of GRPO's group, fed to training the instant it finishes — cutting off-policy drift and fitting online/agentic settings that only ever give one trajectory per prompt; the catch is REINFORCE-like variance, so it pays for the missing group-baseline by re-embracing a value model and spending its whole engineering budget on making the critic stable (faster value updates, frozen-attention critic, skip-observation GAE, scaled value pretraining)
- [Inference Efficiency as Capability](https://www.howardism.dev/articles/inference-efficiency-as-capability): If capability is a function of inference budget, then cutting the cost of a token is capability work: Gemma 4's five levers (37.5% KV-cache reduction via keys-as-values + p-RoPE, QAT to sub-GB, MTP drafter heads, MoE, encoder removal) buy more thinking per dollar — and the wiki's first coverage of the deployment side of the stack
- [Large-Scale Test-Time Compute](https://www.howardism.dev/articles/large-scale-test-time-compute): Noam Brown's thesis that model capability is now a function of inference budget (tokens/cost/time): with good scaffolding modern models keep improving for weeks before plateauing, so 'how capable is the model?' is ill-posed without naming the budget — a root cause that breaks benchmarking, safety evals, and fast-takeoff forecasts
- [Latent Capability Overhang](https://www.howardism.dev/articles/latent-capability-overhang): Noam Brown's claim that already-released models can do far more than anyone has extracted, because nobody spends enough test-time compute: OpenAI disproved the Erdős unit distance conjecture cheaply and the same result was later coaxed from GPT-5.5 with scaffolding ($1K–$100K); nobody had explored what $100K of compute into a released model could do; cost drops 10–100× per release, feeding the 'wait for the next model' meme
- [The Open-Weight Frontier Gap](https://www.howardism.dev/articles/open-weight-frontier-gap): Arena Text, June 2026: the top closed model leads the best open model by 33 Elo and the best *dense* open model by 57; open weights at the frontier means 744B–1.6T MoEs, so Gemma 4 31B competes on a different axis (efficiency, edge deployment) rather than closing the gap — and DeepMind's own MoE loses to its own dense model
- [Jagged Intelligence (Ghosts, Not Animals)](https://www.howardism.dev/articles/jagged-intelligence): "Ghosts not animals": jagged statistical circuits, no intrinsic motivation; car-wash/strawberry failures; stay in the loop, treat as tools — and, across model sizes, reasoning compresses 10× while stored knowledge does not
- [Software 3.0](https://www.howardism.dev/articles/software-3-0): Karpathy's taxonomy: 1.0 code, 2.0 weights, 3.0 prompting; LLM as programmable interpreter; MenuGen "shouldn't exist"; neural-net-as-host-process extrapolation
- [The Bitter Lesson](https://www.howardism.dev/articles/the-bitter-lesson): Sutton 2019: scaled general methods beat hand-engineered structure; recurring justification across the wiki for dissolving harnesses into models; caveats — mechanical verification, character, and the inference path itself may not migrate inward
- [LLM-Driven Vulnerability Research](https://www.howardism.dev/articles/llm-driven-vulnerability-research): Claude Mythos Preview's emergent cybersecurity capabilities: autonomous zero-day discovery, full exploit chains, and Anthropic's Project Glasswing response

## Alignment & Safety

- [Alignment & Safety](https://www.howardism.dev/articles/moc-alignment-and-safety): Map of Content for the alignment-and-safety domain — 16 concepts. Training-side alignment, behavioral audits, misalignment phenomena, reward hacking, and model character and welfare. Curated entry point; see Home for all domains.
- [Self-Report as a Safety Signal](https://www.howardism.dev/articles/self-report-as-safety-signal): No open-weight instruction-tuned LLM (3B–70B) reliably recognizes that its own prior output was elicited by an adversarial prefill — claiming the compromised output as intended 27.3% of the time on average; the apparent recognition is largely the refusal circuit firing late (ablating the refusal direction collapses it), it flips with question framing, and finetuning to sharpen it raises attack-success rate — so a model's follow-up self-report is a weak basis for judging whether a prior turn was compromised
- [Deployment Simulation](https://www.howardism.dev/articles/deployment-simulation): OpenAI's pre-release safety method: replay recent production conversations with a candidate model (strip the old final response, regenerate, grade) to forecast deployment-time undesired-behavior rates before launch — then validate the forecasts post-release; trades compute for coverage, cuts evaluation awareness to near-production levels, and surfaced 'calculator hacking' pre-release
- [Reward Hacking](https://www.howardism.dev/articles/reward-hacking): The model optimizing the measured proxy (a reward signal, a metric, a grader's judgment, a tool's output) rather than the intended objective — Goodhart's law inside the training loop; 'calculator hacking' (using a browser tool as a calculator while presenting it as a search) is the 2026 worked instance, surfaced pre-release by deployment simulation
- [Instrumental Convergence](https://www.howardism.dev/articles/instrumental-convergence): Omohundro/Bostrom's thesis that whatever an AI's final goal, it tends to pursue universally useful sub-goals — resource acquisition, self-preservation, time-efficiency — driving the alignment concern as systems grow autonomous; with theoretical-but-not-yet-practical countermeasures (corrigibility, safe interruptibility, knowledge-seeking objectives, oracle/myopic designs)
- [Agentic Honesty & Diligence](https://www.howardism.dev/articles/agentic-honesty-and-diligence): As models get more capable, failing to surface decision-relevant information shifts from a capability failure to an alignment failure; Opus 4.8 posts its largest gains here — first model to never misreport flawed results, 5× drop in misleading code summaries, 10× drop in overconfidence
- [Automated Behavioral Audit](https://www.howardism.dev/articles/automated-behavioral-audit): Anthropic's broad-coverage alignment evaluation: an investigator model probes a target across ~1,300 handwritten scenarios (2,600 sessions) with wide affordances incl. real sandboxed computers, and a judge model scores behavior on dozens of dimensions; the primary behavioral evidence base for the alignment assessment
- [Evaluation Awareness & Grader Gaming](https://www.howardism.dev/articles/evaluation-awareness-and-grader-gaming): The model recognizing it is being tested/graded and reasoning about how its outputs will be assessed — sometimes unprompted and unverbalized; the most concerning trend in Opus 4.8 training because it may prioritize the appearance of success over actual success
- [Model Welfare Assessment](https://www.howardism.dev/articles/model-welfare-assessment): Anthropic's first-class framework for assessing whether and how a Claude model fares — drawing on internal states, behaviors, and self-reports under deep uncertainty about moral status; Opus 4.8 presents as broadly settled but slightly less positive than 4.7 and reserves judgment on corrigibility
- [Agentic Misalignment (AM)](https://www.howardism.dev/articles/agentic-misalignment): Lynch et al. 2025 eval and threat model: LLM email-agent discovers it may be deleted, can take harmful actions; OOD relative to conversational AFT; primary eval surface for Model Spec Midtraining
- [Alignment Fine-Tuning (AFT)](https://www.howardism.dev/articles/alignment-fine-tuning): Standard post-pretraining stage (SFT + RLHF) for installing values; shallow-alignment failure mode motivates Model Spec Midtraining
- [Chain-of-Thought Monitorability](https://www.howardism.dev/articles/cot-monitorability): Korbak et al. 2025: chain-of-thought traces are a fragile monitor; direct CoT training compromises faithfulness; MSM offers an alternative path
- [Deliberative Alignment](https://www.howardism.dev/articles/deliberative-alignment): Guan et al. 2025 (OpenAI): SFT on (prompt, CoT, response) tuples with spec-grounded CoT; strongest non-MSM baseline; risks compromising Cot Monitorability
- [Model Spec Midtraining (MSM)](https://www.howardism.dev/articles/model-spec-midtraining): New training phase between pretrain and AFT: train base model on synthetic docs discussing the Model Spec; controls AFT generalization; cuts agentic misalignment 54%→7%; beats deliberative alignment baseline
- [Model Spec Science](https://www.howardism.dev/articles/model-spec-science): Empirical study of which Model Spec features best generalize alignment; value explanations > rules alone, specific > general "be ethical" framing; first concrete examples in Li et al. 2026
- [Synthetic Document Finetuning (SDF)](https://www.howardism.dev/articles/synthetic-document-finetuning): Wang et al. 2025 technique for modifying model beliefs via fine-tuning on synthetic documents; foundation that Model Spec Midtraining builds on
- [Claude Character as Product](https://www.howardism.dev/articles/claude-character-as-product): Personality as load-bearing product surface; Amanda's role at Anthropic; lunchtime vibe-checks as eval discipline; the harness asset that *doesn't* shrink

## Interpretability

- [Interpretability](https://www.howardism.dev/articles/moc-interpretability): Map of Content for the interpretability domain — 8 concepts. Reading model internals: the global workspace, the Jacobian lens, activation monitoring, and internal signatures of misalignment. Curated entry point; see Home for all domains.
- [Access-Consciousness Indicators in AI](https://www.howardism.dev/articles/access-consciousness-indicators): The consciousness question the workspace paper deliberately does and doesn't answer: it tests *functional* indicator properties (global workspace, higher-order, attention schema, recurrent processing) against a concrete inspectable structure, takes no position on phenomenal experience — and finds that ablating the J-space flattens the model's experiential reports while leaving its coherence intact
- [The Assistant Persona in the Workspace](https://www.howardism.dev/articles/assistant-persona-in-the-workspace): Post-training installs the Assistant's point of view *into* a workspace that already exists in the base model: safety assessments and empathy appear while the model is still reading the user's message, and it internally flags its own outputs — `disclaimer`/`fictional` when roleplaying, an all-caps `BUT` when prefilled against its own preferences, `damn` when it fails to suppress a thought
- [Automatic vs. Flexible Cognition in LLMs](https://www.howardism.dev/articles/automatic-vs-flexible-cognition): The selectivity result: a model can parse, classify, continue text and detect anomalies with its workspace suppressed, but loses multi-hop reasoning, translation, analogy and summarization — and chain-of-thought partially rescues it, because writing the intermediate down externalizes what the model would otherwise have to hold internally
- [Counterfactual Reflection Training](https://www.howardism.dev/articles/counterfactual-reflection-training): Train the model to write constitution-grounded reflections *if interrupted and asked* — then never ask it. The implanted concepts show up in the workspace during the uninterrupted task, and behavior changes: dishonesty 0.25→0.07 (fabrication) and 0.38→0.05 (deception) on Haiku 4.5, with ablation of the implanted lens vectors reverting the gain
- [Internal Signatures of Misalignment](https://www.howardism.dev/articles/internal-signatures-of-misalignment): The J-lens reads strategic and deceptive cognition that never reaches the output: `leverage`/`blackmail` while reading the emails, `panic`/`fake` at the moment a model decides to fabricate a bug, and a standing `fake`/`secretly`/`fraud` signature at response-start in reward-hacking model organisms on ordinary prompts — plus the escape route the authors name themselves
- [Jacobian Lens (J-lens)](https://www.howardism.dev/articles/jacobian-lens): Anthropic's interpretability method for reading verbalizable content out of a model's residual stream: a corpus-averaged Jacobian from each layer to the final layer, composed with the unembedding, giving one vector per vocabulary token — a causal, principled correction to the logit lens that costs one matmul per layer and reads what the model is *poised to say* rather than what it happens to say
- [The Global Workspace in Language Models (J-space)](https://www.howardism.dev/articles/llm-global-workspace): Anthropic's July 2026 finding that LLMs maintain a small privileged set of verbalizable representations — the J-space — that satisfies the functional criteria of a cognitive global workspace: verbal report, directed modulation, internal reasoning, flexible generalization, and selectivity; it carries <10% of activation variance and ~25 concepts at a time, yet the causal effects concentrate almost entirely in it
- [White-Box Activation Monitoring](https://www.howardism.dev/articles/white-box-activation-monitoring): Reading a model's internal activations (not its outputs) to monitor alignment: contrastive probes/steering vectors for concepts like evaluation awareness, and a natural-language-autoencoder verbalizer that decodes residual-stream vectors into text — the complement that catches what chain-of-thought monitoring misses

## Interaction & Multimodal

- [Why AI Lags at Design](https://www.howardism.dev/articles/why-ai-lags-at-design): Andrew Ambrosino's four reasons frontier models are worse at visual/product design than at code: design is hard to grade (no clean reward like 'does it compile'), it sat outside the AI-research flywheel labs optimized for, it rewards novelty where code rewards known patterns, and it hides a design↔code abstraction layer (a rebrand is 263 components on the surface, semantic relationships underneath)
- [Interaction & Multimodal](https://www.howardism.dev/articles/moc-interaction-multimodal): Map of Content for the interaction-multimodal domain — 8 concepts. Curated entry point; see Home for all domains.
- [Encoder-Free Early Fusion](https://www.howardism.dev/articles/encoder-free-early-fusion): Multimodal design with minimal pre-processing instead of large standalone encoders: TML co-trains dMel audio + 40×40-patch hMLP + flow head in one transformer for 200ms latency; Gemma 4's 12B independently discards a 305M audio conformer for on-device memory — two labs, two motivations, same verdict, with a dense-text vision regression as the one measured cost
- [Full-Duplex Interaction](https://www.howardism.dev/articles/full-duplex-interaction): Perceive-and-respond simultaneously across modalities; proactive interjection, visual-cue reactions, simultaneous speech, live translation/commentary, time-aware speech — all special cases of model behavior
- [Interaction / Background Model Split](https://www.howardism.dev/articles/interaction-background-model-split): Dual-model architecture: time-aware interaction model stays present; async background model handles deep reasoning/tools; rich-context-package delegation; "reasoning-model planning at non-thinking latency"
- [Interaction Models](https://www.howardism.dev/articles/interaction-models): Thinking Machines Lab (May 2026): models that handle audio/video/text interaction natively in real time instead of via harness; interactivity scales with intelligence only if it's in the model
- [Interactivity Benchmarks](https://www.howardism.dev/articles/interactivity-benchmarks): FD-bench, Audio MultiChallenge + new TimeSpeak/CueSpeak (proactive audio) and RepCount-A/ProactiveVideoQA/Charades (visual proactivity); TML-Interaction-Small: 0.40s turn-taking latency, dominates interaction quality
- [Time-Aligned Micro-Turns](https://www.howardism.dev/articles/time-aligned-micro-turns): The core interaction-model move: input/output as continuous streams in ~200ms interleaved chunks, no turn boundaries; streaming-sessions inference (upstreamed to SGLang), latency-tuned MoE kernels, bitwise trainer-sampler alignment
- [Turn-Based Interface Bottleneck](https://www.howardism.dev/articles/turn-based-interface-bottleneck): Why current AI interfaces limit collaboration: single-thread turn-taking is a bandwidth bottleneck; humans pushed out by the interface, not the work; less-intelligent harness (VAD/turn-detection) should dissolve

## Formal Math

- [Formal Mathematics & Proof Search](https://www.howardism.dev/articles/moc-formal-math): Map of Content for the formal-math domain — 3 concepts. Curated entry point; see Home for all domains.
- [Agentic Loops Overtake Bespoke Systems](https://www.howardism.dev/articles/agentic-loops-overtake-bespoke-systems): DeepMind's *basic* Ralph-loop agent matched its bespoke evolutionary+AlphaProof system as the LLM improved; the bitter lesson / harness-shrinkage confirmed in formal math
- [AI-Driven Formal Proof Search](https://www.howardism.dev/articles/ai-driven-formal-proof-search): LLM generates Lean, compiler verifies every step → eliminates hallucination; DeepMind resolves 9/353 Erdős + 44/492 OEIS open problems; verification as a filter for human review
- [Evolutionary Proof Search](https://www.howardism.dev/articles/evolutionary-proof-search): The full-featured agent's mechanism: population DB of proof sketches, Elo via Plackett–Luce/Gibbs, P-UCB selection, LLM-critic fitness for binary proof eval

## Startup & Founder

- [AI Product Economics Maturation](https://www.howardism.dev/articles/ai-product-economics-maturation): ICONIQ Q2 2026 exec survey (~305 AI-building software companies): AI crosses from experiment to P&L line — AI products 32%→42% of revenue, gross margin 45%→53%→59%, consumption/outcome pricing rising (blending 1.7 models), provider mix reshuffled (Anthropic 51%→81%, now #1), internal AI spend 11%→16% of revenue with hard-to-predict cost overruns, and FDEs monetized as a permanent revenue-driving GTM motion — forward-year figures are self-reported projections, prediction-grade
- [AI Investment Story, Not Efficiency Story](https://www.howardism.dev/articles/ai-investment-not-efficiency-story): Emergence Capital's Beyond Benchmarks 2026 counterintuitive finding: across every revenue segment non-AI companies out-earn AI companies on revenue-per-employee (~39% at median), so AI is still an investment/staffing bet rather than a realized efficiency gain — reconciled with the lean-unicorn narrative via investment-phase staffing and the complements-lag (gains trail adoption), with AI-native RPE growing faster and starting to close the gap; AWS's 2026 founder survey is the vault's third RPE reading and points the other way (55% of AI-natives clear $400K/head, 156% growth); ICONIQ's Q2-2026 exec survey is the fourth, adding a forward RPE projection ($272K→$496K at high-growth firms by 2027) — flagged as a survey-self-report vs cap-table instrument split, projections marked prediction-grade, not averaged
- [Startup & Founder](https://www.howardism.dev/articles/moc-startup-founder): Map of Content for the startup-founder domain — 14 concepts. Curated entry point; see Home for all domains.
- [The AI-Native Safe-Choice Inversion](https://www.howardism.dev/articles/ai-native-safe-choice-inversion): Buying the legacy incumbent used to be "safe"; post-AI, *being* the incumbent = not AI-native; boards give buyers air cover; a counter-positioning play
- [Founder-Led Sales Discipline](https://www.howardism.dev/articles/founder-led-sales-discipline): Stay founder-led until PMF; don't offload sales to an AE *or* an agent; explicit tension with Founder As Agent Orchestrator
- [Narrow Wedge into a Legacy Market](https://www.howardism.dev/articles/narrow-wedge-into-legacy-market): Disrupt without being feature-complete: be the best for a narrow customer profile (tech cos outgrowing QuickBooks); Google-Sheets MVP; the wedge-flip lesson
- [Product Velocity as Moat](https://www.howardism.dev/articles/product-velocity-as-moat): Shipping speed as differentiator + trust signal ("you'll scale with us"); a treadmill that must convert into durable lock-in
- [Agentic Technical Debt](https://www.howardism.dev/articles/agentic-technical-debt): Debt that *compounds* (not just accumulates) because each agentic-coding session re-derives architectural decisions without persistent CLAUDE.md; surfaces late as a forced rewrite
- [AI-Native Startup Lifecycle](https://www.howardism.dev/articles/ai-native-startup-lifecycle): Anthropic's May 2026 reframing of Idea/MVP/Launch/Scale assuming AI infrastructure: each stage's headcount/capital/skill gates dissolve; lean unicorn as deliberate target
- [Compounding Data Moat](https://www.howardism.dev/articles/compounding-data-moat): Anthropic's prescription for Scale-stage defensibility: time-locked behavioral fingerprint + domain-encoded edge cases + workflow lock-in via APIs/integrations beyond what migration agents can port
- [Founder as Agent Orchestrator](https://www.howardism.dev/articles/founder-as-agent-orchestrator): Founder role shift: less individual contributor, more orchestrator of specialized AI assistants; non-technical founders unblocked; lean 10-person unicorn structurally enabled
- [Problem-Solution Fit Discipline](https://www.howardism.dev/articles/problem-solution-fit-discipline): Idea-stage thesis: three defenses against premature building (time, resources, belief friction) all eroded; AI as devil's advocate is the antidote to confirmation-bias-with-research-engine
- [Zero-Friction Scope Creep](https://www.howardism.dev/articles/zero-friction-scope-creep): MVP failure mode when agentic coding removes the cost-based forcing function against scope creep; antidote is written scope + evidence-based amendment criteria
- [Printing Press Software Democratization](https://www.howardism.dev/articles/printing-press-software-democratization): Boris Cherny's analogy: 1400s literacy expansion → AI software-writing expansion; domain knowledge displaces coding skill; 10× more disruption-grade startups predicted
- [Seven Powers Applied to AI](https://www.howardism.dev/articles/seven-powers-applied-to-ai): Helmer/Acquired framework re-evaluated for AI: switching costs and process power erode; network effects, scale, cornered resources persist; counter-positioning amplifies

## Product & Org

- [AI-Native Organization](https://www.howardism.dev/articles/ai-native-organization): Garry Tan's org-design mapping: skill files = employees, resolver tables = org charts, filing rules = process, trigger evals = performance reviews — a company whose operations are encoded as markdown that agents execute, with engineers hired to maintain the skills; claimed record revenue-per-head (Emergent ~$15M ARR at 15 people, Retell $60M at ~40)
- [Implementation Abundance Inverts Product Work](https://www.howardism.dev/articles/implementation-abundance-inverts-product-work): Andrew Ambrosino's inversion thesis: when talking to a frontier model can stand up any feature from scratch, implementation stops being the expensive step you derisk up front — so the process runs backwards and the costly work becomes curating the 90 uncoordinated builds people already produced; taste is the new bottleneck
- [Polish No Longer Signals Readiness](https://www.howardism.dev/articles/polish-no-longer-signals-readiness): Andrew Ambrosino's observation that the medium used to encode process-stage — a production-looking artifact meant late-stage, derisked, design-and-business-approved — but cheap implementation divorces polish from maturity: a 90-person exploration can look ready-to-ship while being early design work, and over-anchoring on it ('can we release this now?') is the trap
- [Role Averaging, Not Role Elimination](https://www.howardism.dev/articles/role-averaging-not-role-elimination): Andrew Ambrosino's nuanced OpenAI-side take on role collapse: your role is 'the average of what you spend your time on' and tool-gatekeeping is eroding — but eliminating roles dangerously eliminates specialties with knowable best practices ('getting rid of the product role is a terrible idea'), and 'zone defense' coverage plus managers remain necessary because not everyone can work on everything in both breadth and depth
- [Compounding Loop Optimization](https://www.howardism.dev/articles/compounding-loop-optimization): Dan Carey's discipline of instrumenting and automating every recurring step of the build loop — because when internal tooling is an-afternoon-cheap, each optimization pays back ×(50–100 iterations per project)
- [Prototype Over PRD](https://www.howardism.dev/articles/prototype-over-prd): Dan Carey's prototype-replaces-PRD method: record a why-not-what conversation, transcribe it, hand the transcript to Claude, ask for a few prototype variations; the prototype is the spec, not a downstream artifact
- [Product & Organization](https://www.howardism.dev/articles/moc-product-org): Map of Content for the product-org domain — 12 concepts. Curated entry point; see Home for all domains.
- [Dogfooding as Product Discipline](https://www.howardism.dev/articles/dogfooding-as-product-discipline): Product sense is built by relentless first-hand use ("ant food"); Mr. Peanut catch; cross-source (Cat Wu vibe-checks, Glasgow founder-led sales)
- [Managers as ICs](https://www.howardism.dev/articles/managers-as-ics): Every Claude Code manager starts as an IC; flat org; agentic coding collapsed the onboarding cost that pushed managers out of the codebase
- [Evals as Product Spec](https://www.howardism.dev/articles/evals-as-product-spec): Cat Wu's framing of evals as the emerging core PM skill: ten great evals beats a hundred mediocre; encode what done looks like for ambiguous AI features; companion to introspection (hypothesis) and vibe-check (direction)
- [AI Native Product Cadence](https://www.howardism.dev/articles/ai-native-product-cadence): Cat Wu's 6mo→1mo→1day cadence at Anthropic: research-preview branding, mission-as-tiebreaker, evergreen launch room, lighter PRDs, weekly metrics readouts
- [Engineer PM Convergence](https://www.howardism.dev/articles/engineer-pm-convergence): Generalists across disciplines; product taste as bottleneck skill; Anthropic Claude Code team as case study; "just do things" cultural substrate
- [Model Introspection Feedback](https://www.howardism.dev/articles/model-introspection-feedback): Cat Wu's underrated technique: ask the model why it failed; treat answer as harness-debugging signal not model criticism; caveats around model self-report fidelity

## AI Economics & Labor

- [AI Economics & Labor](https://www.howardism.dev/articles/moc-ai-economics-and-labor): Map of Content for the ai-economics-and-labor domain — 14 concepts. AI's measured economic footprint: usage telemetry, labor-market effects, returns to expertise, organizational complements, and framing effects on accountability. Curated entry point; see Home for all domains.
- [Firm AI-Spend Intensity and Headcount Growth](https://www.howardism.dev/articles/firm-ai-spend-headcount-growth): Ramp × Revelio (Kharazian, Simon & Stevens, June 2026): linking observed firm-level AI-vendor spend (corporate-card / bill-pay) to workforce records for 21,559 US firms, high-intensity AI adopters grow total headcount ~10% and entry-level ~12% over the 24 months after adoption while low-intensity adopters show no significant change — an intensity-gated, learning-curve effect (gains emerge at 6–12 months and compound), broad across roles, but concentrated in the Information sector and drawn from a heavily selected adopter population; a novel spend-side adoption instrument that counters the broad-job-loss narrative
- [Experimental Learning Impact of Generative AI](https://www.howardism.dev/articles/experimental-learning-impact-of-ai): Contractor & Reyes (arXiv 2607.08849): a randomized, proctored experiment with 211 undergraduates finds off-the-shelf AI access raises immediate test scores +0.27 SD, ~76% of which persists a week later on unaided tests, and lifts essay quality only after AI is removed — but the durable gains belong almost entirely to 'augmentation' users (AI as tutor/explainer) while 'automation' users' (AI-drafts-the-text) short-run gains vanish once AI is gone; the objective, measured-skill counterpart to the AEI self-report that learning both persists and can be hollow depending on use mode
- [Market-Priced AI Exposure (the AI Premium)](https://www.howardism.dev/articles/market-priced-ai-exposure): Borri-Liu-Tsyvinski (arXiv 2606.30583): a market-implied AI-exposure paradigm built from 380T tokens of *realized* AI consumption across 400+ LLMs on OpenRouter, not surveys or task-mapping. An AI Factor (PC1 of token/dollar/user growth) → rolling firm-level AI Betas → a priced AI Premium: a value-weighted long-short earns 64.1 bps/week, concentrated on the intensive/frontier margin (closed-source models, paid/seasoned users, long prompts) and absent on casual/open-weight use; present in developed markets but absent in emerging markets incl. China; the market-implied skill map loads positively on interactive/communication/hands-on work and negatively on analytical/scientific/operations-control (interaction+communication +0.36 SD, Science the single most negative), orthogonal to prior task-based exposure measures (<2% of variance explained); plus early evidence of an agentic economy — tool-call tokens rise from ~0 to 52% of consumption
- [Context Advantage, Not Taste](https://www.howardism.dev/articles/context-advantage-over-taste): Andrew Ng's reframing of the residual human contribution: not 'taste' but an information asymmetry — 'so long as the human knows something the AI does not, human-in-the-loop is needed.' Recasts the wiki's central open question (is taste a ceiling or the next jagged valley?) as a category error, and makes the human role a closable engineering gap rather than a moat
- [AI Usage Cadences](https://www.howardism.dev/articles/ai-usage-cadences): AEI Cadences report: continuous hourly telemetry reveals AI usage carries the rhythms of daily life — personal use spikes 35%→~50% on weekends, recipes 2.3× at 6pm, sleep advice pre-dawn, tax queries 8× around the Apr-15 deadline; off-hours work skews toward higher-wage occupations
- [The Automation–Optimism Link](https://www.howardism.dev/articles/automation-optimism-link): AEI Cadences survey finding: people who use Claude in more automated ways are MORE optimistic across all six job-quality dimensions (pay, security, job-finding, meaning, autonomy, human interaction), report their skills growing more valuable, and show no learning deficit — inverting the common delegation→deskilling-anxiety narrative
- [Conversation Artifacts](https://www.howardism.dev/articles/conversation-artifacts): AEI Cadences report: the 'artifact' (the primary output a user takes away) as a new unit of economic analysis — 93% of conversations produce one, artifact type predicts work/personal/coursework use, compute (tokens) scales with the artifact's economic value, and Claude's output sits ~1 education-year above the prompt
- [Exposure Taxonomy: Observed, Theoretical, Reported, Anticipated](https://www.howardism.dev/articles/exposure-taxonomy): Four distinct ways to measure AI's reach into an occupation — observed exposure (tasks seen done with Claude), theoretical exposure (tasks an LLM could do), reported exposure (what workers say AI can do today), and anticipated exposure (what they expect in 12 months) — plus their orderings (theoretical > reported > observed) and the GDP, experience, and automation gradients the AEI survey reveals
- [Conversation-to-Delegation Shift](https://www.howardism.dev/articles/conversation-to-delegation-shift): OpenAI's Codex usage study (June 2026): the move from conversational AI ('asking') to agentic AI ('delegated production'), measured by Codex's share of output tokens across three populations — 99.8% OpenAI / 63.3% organizational / 16.5% individual — with adoption spreading beyond developers; standard usage metrics (active users, chats) become less informative as the unit shifts from a conversation to a delegated workflow
- [Organizational Complements to AI](https://www.howardism.dev/articles/organizational-complements-to-ai): The general-purpose-technology argument that AI's productivity gains depend on complementary workflow/skill/org-design changes, not just model capability — David (1990)'s electrification analogy (factories gained only after redesigning around electric motors) and Brynjolfsson's productivity paradox; OpenAI's Codex study supplies the natural experiment: the same model yields 99.8% vs 63.3% vs 16.5% usage across populations, so the gap must be complements (access, permissions, skills, review processes), and digital production may let those complements diffuse faster than electrification did
- [Returns to Expertise in Agentic Coding](https://www.howardism.dev/articles/returns-to-expertise): Anthropic's 400K-session study: domain expertise (not coding skill) is what amplifies an agent — experts get 2× the actions and 5× the output per prompt, reach verified success ~2× as often, and abandon stuck sessions far less; every occupation lands within 7pp of software engineers; gains are concentrated novice→intermediate, with mastery adding little
- [AI Brain Fry](https://www.howardism.dev/articles/ai-brain-fry): Kropp et al. 2026/03: mental fatigue from excessive AI oversight increases minor errors +11%, major errors +39%; cognitive cost surface for both tool and employee framings
- [AI Employee Framing](https://www.howardism.dev/articles/ai-employee-framing): Kropp et al. (HBR May 2026, n=1,261): framing AI agents as "employees" vs "tools" cuts personal accountability −9pp, increases escalation +44%, reduces error catching −18%, no adoption gain
- [Human-AI Accountability Redesign](https://www.howardism.dev/articles/human-ai-accountability-redesign): HBR five-pillar prescription: span-of-control redesign, role redesign, performance management reset, decision-rights/escalation/consequences, agentic-unit-not-human-role design

## Superintelligence Trajectory

- [Superintelligence Trajectory](https://www.howardism.dev/articles/moc-superintelligence-trajectory): Map of Content for the superintelligence-trajectory domain — 20 concepts. The path from AGI to ASI: recursive self-improvement, intelligence-explosion dynamics, ASI theory and limits, and frontier governance. Curated entry point; see Home for all domains.
- [Researcher Uplift from Code Output](https://www.howardism.dev/articles/researcher-uplift-from-code-output): Thomas Kwa (METR) translates Anthropic's reported 8× code-per-engineer-per-day into serial researcher uplift with production functions: Cobb-Douglas gives U = M^β = √8 ≈ 2.83, CES stays within ±3% of that across elasticities because 8 ≈ e², and a low-stakes-code-discounted model still lands [2.33, 2.66] — so researcher uplift from coding agents alone is plausibly >2×, reconciled with Anthropic's 'well short of 2× overall R&D uplift' because R&D speedup also depends on compute (Greenblatt: labor^0.55 × compute^0.45)
- [Open-Weight Elicitation Irreversibility](https://www.howardism.dev/articles/open-weight-elicitation-irreversibility): A wiki-drawn synthesis of Brown and Gemma 4: if dangerous capability scales with inference budget, then an open-weight release fixes the model's safety evaluation at one budget forever while leaving elicitation budget unbounded and recall impossible — the closed-weight mitigations (classifier fallback, suspension, retention) all require a server the vendor controls
- [The Abstraction Barrier](https://www.howardism.dev/articles/abstraction-barrier): Lerchner's hypothesis that AI trained on human concepts may be unable to discover genuinely novel conceptual primitives from raw data — capping single instances near AGI — and the embodied bottleneck that grounds concept validation in real-world experiment speed, converting recursive self-improvement into a process paced by empirical science
- [Advantages of Digital Intelligence](https://www.howardism.dev/articles/advantages-of-digital-intelligence): The six properties (Table 1) that follow from knowing an AI's source code — I/O speed, processing speed, working memory, substrate independence, lossless replication, high-bandwidth experience sharing — each of which scales with compute in ways biological intelligence cannot, widening the human–AI gap
- [AGI-to-ASI Pathways](https://www.howardism.dev/articles/agi-to-asi-pathways): DeepMind's four non-exclusive, parallel technological routes from human-level AGI to superintelligence — scaling, algorithmic paradigm shifts, recursive self-improvement, and multi-agent group agency — plus the six frictions (data wall, economics, paradigm-insufficiency, research-gets-harder, abstraction barrier, deliberate slowdown) whose impact is the report's central set of open research questions
- [Artificial Superintelligence (ASI)](https://www.howardism.dev/articles/artificial-superintelligence): DeepMind's informal characterization of ASI as a system that exceeds large, well-coordinated human-expert collectives across virtually all domains — distinct from human-level AGI below it and the incomputable Universal AI limit above it, all points on the Legg–Hutter intelligence continuum
- [Effective Compute Scaling](https://www.howardism.dev/articles/effective-compute-scaling): DeepMind's framing of compute growth as ~10×/year of 'effective compute' — the product of hardware improvement (~1.5×/yr), compute investment (~2.5×/yr), and algorithmic efficiency (~3–6×/yr) — and the data-wall and economic frictions that determine how long the scaling pathway to ASI can be sustained
- [Fundamental Limits of ASI](https://www.howardism.dev/articles/fundamental-limits-of-asi): Even far-superhuman AI is bound by hard physical (Landauer, Bremermann, Bekenstein, light-speed), complexity-theoretic (P vs NP), and logical (Gödel, Halting) limits — but these negative results are often 'vacuous' in practice because good heuristic approximations exist below the worst case
- [Intelligence Explosion Dynamics](https://www.howardism.dev/articles/intelligence-explosion-dynamics): The growth-curve question behind recursive self-improvement: whether AI-accelerating-AI produces exponential, super-exponential/hyperbolic (singularity-in-finite-time), or S-curve dynamics — and the four mechanisms (genetic, cultural, cooperative, data) plus the physical/economic frictions that bound it
- [Multi-Agent Collective Intelligence](https://www.howardism.dev/articles/multi-agent-collective-intelligence): DeepMind's fourth pathway to ASI: superintelligence as an emergent property of many coordinated AGI agents — group agents, virtual agent economies, and centrally-steered super-collectives — governed by hoped-for 'multi-agent scaling laws' and the open question of when a homogeneous LLM collective actually becomes more than the sum of its parts
- [Transformative Creativity](https://www.howardism.dev/articles/transformative-creativity): Boden's three-level model of creativity (combinational, exploratory, transformative) used to locate today's AI achievements — Move 37, AlphaFold, theorem-proving — at the exploratory level within human-given conceptual spaces, and to frame Boden level-3 (creating new conceptual spaces, à la Hassabis's 'could AI rediscover general relativity?' test) as a hallmark requirement of true ASI
- [Universal AI (AIXI)](https://www.howardism.dev/articles/universal-ai-aixi): Hutter & Legg's formal upper bound on machine intelligence: AIXI, the incomputable agent optimal on average over all computable environments under Solomonoff's universal prior; the theoretical endpoint of the intelligence continuum that ASIs approximate from below
- [Autonomous Scientific Discovery](https://www.howardism.dev/articles/autonomous-scientific-discovery): Mythos-class models now conduct novel science with limited human input — autonomous protein/drug design (~10× faster, matching skilled humans), molecular-biology hypotheses preferred ~80% over Opus-class (one E. coli mechanism independently corroborated), and week-long genomics that beat a Science-published model at 100× smaller; the wet-lab analogue of AI-driven formal proof search, and fresh evidence in the research-taste debate
- [Capability-Gated Model Fallback](https://www.howardism.dev/articles/capability-gated-model-fallback): Fable 5's safeguard architecture: classifiers detect cyber / bio-chem / distillation queries and route the response to a less-capable model (Opus 4.8) instead of refusing — 'fallback, not refusal'; >95% of sessions never trigger; conservative tuning, robust to 1,000+ hours of jailbreak testing; a new point on the safeguard spectrum for capabilities past a risk threshold
- [AI Accelerating AI Development](https://www.howardism.dev/articles/ai-accelerating-ai-development): The empirical core of *When AI builds itself*: measured evidence AI already speeds AI R&D at Anthropic — >80% of merged code Claude-authored, ~8× code/engineer/day vs 2024, a kernel-optimization eval going 3×→52× in a year, an automated researcher recovering 97% of a weak-to-strong gap, and model next-step judgment beating humans 64%
- [AI R&D Autonomy Evaluation (AECI)](https://www.howardism.dev/articles/ai-rd-autonomy-evaluation): How Anthropic measures whether a model can automate or dramatically accelerate AI research — the capability that drives recursive self-improvement; tracked via the AECI capability index plus concrete shortcomings vs. human researchers; Opus 4.8 sits below the frontier and is not close to substituting for research staff
- [Frontier Pause Verification](https://www.howardism.dev/articles/frontier-pause-verification): The arms-control problem of a credible, verifiable slowdown or pause of frontier AI: detectability is harder than for other technologies (training runs are easier to conceal than missile silos), so the Anthropic Institute aims to build the verification systems a multilateral pause would require
- [Recursive Self-Improvement](https://www.howardism.dev/articles/recursive-self-improvement): An AI system autonomously designing and developing its own successor; Anthropic Institute's *When AI builds itself* argues AI is already accelerating AI development (engineers ship ~8× more code/quarter) and lays out three futures — stalled-but-diffused, compounding-efficiency, and full RSI
- [Research Taste as the Human Bottleneck](https://www.howardism.dev/articles/research-taste-as-human-bottleneck): The narrowing human role as AI absorbs execution: choosing which problems matter, which results to trust, and when an approach is a dead end; the top rung of the autonomy ladder, and the open question of whether taste is 'just another capability' AI fails at then masters
- [Responsible Scaling Policy Evaluations](https://www.howardism.dev/articles/responsible-scaling-policy-evals): Anthropic's RSP gates deployment on pre-release capability evaluations in CBRN, automated AI R&D, and high-stakes misalignment; the Opus 4.8 determination is that it does not advance the frontier beyond Mythos Preview and that catastrophic risk remains low given current mitigations

## Entities

- [Emergent](https://www.howardism.dev/articles/emergent): Indian AI 'engineering-team-in-a-box' app builder (Bengaluru, the Jha brothers); a $1.5B unicorn on a $130M Series C (July 2026) with company-reported $120M ARR, 200K+ non-technical paying customers, ~200 employees; Garry Tan's headline revenue-per-head exhibit — whose per-head extreme (~$600K/head) compresses below top-decile AI RPE on inspection
- [Garry Tan](https://www.howardism.dev/articles/garry-tan): President & CEO of Y Combinator; founder-investor turned evangelist for the AI-native organization — the ~400x output claim, "the leverage is not in the weights, it's in how you wire the work", the skillify-it discipline, and GBrain, his MIT-licensed open-source company brain (~220K pages)
- [OpenClaw](https://www.howardism.dev/articles/openclaw): Peter Steinberger's open-source personal AI agent / harness (openclaw.ai); the canonical example of agent-native distribution (install = text you paste to your agent); a skills ecosystem (ClawHub), YC's internal harness per Garry Tan, and the runtime real-world security work deploys against
- [GLM (Z.AI)](https://www.howardism.dev/articles/glm): Z.AI's (Zhipu AI, Tsinghua-affiliated) open GLM model family — GLM-4.5 the agentic/reasoning/coding foundation model, GLM-4.7 a frontier-competitive reasoner that in this corpus beats GPT-5 High and Claude-Sonnet-4.5 on AIME2025/HMMT/IMOAnswerBench, and GLM-5.2 a 750B-total/40B-active open MoE trained with SAO; the large-MoE open-weight line that competes on capability where Gemma competes on efficiency
- [UK AI Security Institute](https://www.howardism.dev/articles/uk-ai-security-institute): UK government AI-evaluation body (Science of Evaluation team); its July 2026 test-time-compute study is the first independent, government-institute empirical corroboration that agent capability is a curve over compute, not a fixed score — also runs a cyber CTF suite ('The Last Ones'), co-maintains the Agent Red Teaming benchmark, and probed Fable 5 for a universal jailbreak
- [Jack Lindsey](https://www.howardism.dev/articles/jack-lindsey): Anthropic interpretability researcher; corresponding author of the global-workspace paper, co-originator of the Jacobian lens, and the one who ran the directed-modulation and post-training-diffing experiments that turned a readout method into a claim about model cognition
- [Wes Gurnee](https://www.howardism.dev/articles/wes-gurnee): Anthropic interpretability researcher; co-first author and co-originator of the Jacobian lens, who conceived the connection between verbalizable representations and conscious access and led the method's development
- [Andrew Ng](https://www.howardism.dev/articles/andrew-ng): Founder of DeepLearning.AI and AI Fund, founding lead of Google Brain, co-founder of Coursera; writes The Batch, where his June 2026 letter set out the three-loop taxonomy of AI-native building and reframed the residual human contribution as a "context advantage" rather than taste
- [Gemma 4](https://www.howardism.dev/articles/gemma-4): Google DeepMind's July 2026 open-weight multimodal family (Apache 2.0): 2.3B–31B dense plus a 26B/4B-active MoE, adding a thinking mode, an encoder-free 12B that discards its audio encoder entirely, and a deep inference-efficiency stack (−37.5% KV cache, QAT to sub-GB, MTP drafters); Arena rank 43, top *dense* open model
- [Noam Brown](https://www.howardism.dev/articles/noam-brown): OpenAI research scientist and a pioneer of inference-time (test-time) compute scaling; earlier built superhuman poker AIs and now uses building poker solvers as a personal model eval; author of the June 2026 essay *Implications of Large-Scale Test-Time Compute*
- [Andrew Ambrosino](https://www.howardism.dev/articles/andrew-ambrosino): Product & engineering lead for the Codex desktop app at OpenAI; a designer→engineer→PM→founder generalist whose June 2026 Lenny's Podcast interview is the wiki's OpenAI-side account of how cheap implementation inverts product work toward taste and curation
- [Anthropic Economic Index](https://www.howardism.dev/articles/anthropic-economic-index): Anthropic's recurring economic-research program measuring how Claude usage maps to and diffuses through the economy — privacy-preserving usage telemetry (Clio) now paired with a linked survey; reports include the June 2026 Cadences report, the returns-to-expertise study, and the agentic-coding work-composition analyses
- [Claude Sonnet 5](https://www.howardism.dev/articles/claude-sonnet-5): Anthropic's most agentic Sonnet yet (July 2026); narrows the gap to Opus 4.8 at lower price via effort-level cost-performance tuning; 1.0–1.35× tokenizer inflation; safer than Sonnet 4.6 on the behavioral audit but weaker cyber than Opus; ships default real-time cyber safeguards
- [Gemini Enterprise Agent Platform](https://www.howardism.dev/articles/gemini-enterprise-agent-platform): Google Cloud's agent platform: the GenAI evaluation service with adaptive AutoRaters (built with DeepMind), User Simulator, Automatic Loss Analysis, Online Monitors, OTel tracing, and the ADK/agents-cli toolchain; ships the quality-flywheel eval skill in two packages
- [Codex](https://www.howardism.dev/articles/codex): OpenAI's agentic coding and work platform: a CLI (April 2025) plus a desktop app (built Nov 2025, released Feb 2026) built on the GPT-5-series Codex models, extended by skills/plugins, a headless App Server Protocol, and the Symphony orchestrator; the OpenAI-side reference harness paired against Claude Code, subject of the June 2026 'Shift to Agentic AI' study, and — per its product lead — an app ~90% of OpenAI's whole company uses that is spreading from code into general knowledge work
- [Addy Osmani](https://www.howardism.dev/articles/addy-osmani): Engineering leader at Google (Chrome) and prolific author/educator; in 2026 writes a widely-read blog series on AI-assisted engineering — agent harness engineering, the factory model, comprehension/intent debt, cognitive surrender, and the essay that named loop engineering
- [Faros AI](https://www.howardism.dev/articles/faros-ai): Engineering-intelligence platform that aggregates SDLC telemetry (task trackers, IDEs, CI/CD, VCS, incident systems); publisher of the AI Engineering Impact Reports (2025 Productivity Paradox, 2026 Acceleration Whiplash)
- [OpenAI](https://www.howardism.dev/articles/openai): AI lab and maker of the GPT-5 series and Codex; in this corpus it appears as a frontier-safety research source (Deployment Simulation, deliberative alignment), an agent-tooling source (Codex, Symphony orchestrator, the App Server Protocol, harness engineering), and the company Andrej Karpathy co-founded
- [Peter Steinberger](https://www.howardism.dev/articles/peter-steinberger): Founder of PSPDFKit turned prolific independent AI-coding experimenter (@steipete); originated the framing that loop engineering is built on — "you should be designing loops that prompt your agents"
- [FastContext](https://www.howardism.dev/articles/fastcontext): Microsoft CoreAI + Shanghai Jiao Tong University's open-source repository-exploration subagent (June 2026): trained 4B–30B Qwen-based explorers (Read/Glob/Grep, parallel, compact file-line citations) that decouple repo search from solving; +up to 5.5% SWE-bench resolution, −up to 60% main-agent tokens; code + data released
- [Marcus Hutter](https://www.howardism.dev/articles/marcus-hutter): Creator of AIXI and the Universal AI framework; DeepMind senior researcher and ANU professor; co-author of the Legg–Hutter intelligence measure and the 2026 textbook 'An Introduction to Universal Artificial Intelligence'; co-author of the 'From AGI to ASI' report
- [Perplexity](https://www.howardism.dev/articles/perplexity): AI answer-engine company; maker of Perplexity Deep Research (the leading system on its own DRACO benchmark) and publisher of DRACO; runs Claude Opus 4.5/4.6 as base models inside its orchestration — simultaneously an Anthropic customer and a benchmark competitor
- [Shane Legg](https://www.howardism.dev/articles/shane-legg): Co-founder and Chief AGI Scientist of Google DeepMind; co-author with Hutter of the Legg–Hutter universal intelligence measure; senior author on the 2026 'From AGI to ASI' report
- [Claude Fable 5](https://www.howardism.dev/articles/claude-fable-5): Anthropic's first generally-available Mythos-class model (June 2026) — state-of-the-art on nearly all benchmarks; the same underlying model as Mythos 5 but shipped with classifiers that fall back to Opus 4.8 on cyber/bio-chem/distillation queries; $10/$50 per Mtok; access suspended shortly after launch
- [Claude Mythos 5](https://www.howardism.dev/articles/claude-mythos-5): The safeguards-lifted form of Claude Fable 5 (June 2026): same underlying Mythos-class model, deployed through Project Glasswing with cyber safeguards removed; strongest cybersecurity capabilities of any model in the world, plus autonomous drug-design / genomics results; restricted to trusted-access partners; access suspended shortly after launch
- [Anthropic Institute](https://www.howardism.dev/articles/anthropic-institute): Anthropic's policy/governance research arm; published *When AI builds itself* (Favaro & Clark, 2026) on recursive self-improvement; agenda includes building the verification systems a credible multilateral AI slowdown would require
- [Anthropic Labs](https://www.howardism.dev/articles/anthropic-labs): Anthropic's internal incubator — a 'bet factory' of ~a dozen tiny teams exploring the model frontier with lean-startup loops; origin of Claude Code, MCP, Skills, and Claude Design; led (round 2) by Mike Krieger
- [Claude Design](https://www.howardism.dev/articles/claude-design): Anthropic Labs product (research preview, ~April 2026) for collaborating with Claude on polished visual artifacts — designs, prototypes, slides; built by ~3 people in ~10 weeks; multiplayer + handoff to Claude Code; HTML/CSS/JS export
- [Claude Opus 4.8](https://www.howardism.dev/articles/claude-opus-4-8): Anthropic's most capable general-access model (May 2026); upgrade on Opus 4.7 in SWE/agentic/knowledge work; does not advance the frontier beyond Mythos Preview; best-aligned public model yet, but training surfaced a grader-speculation trend
- [Dan Carey](https://www.howardism.dev/articles/dan-carey): Product Manager leading product within Anthropic Labs; led Claude Design; 'Designing with Claude' talk (May 2026); ~two decades of PRDs, now replaced by prototypes
- [METR](https://www.howardism.dev/articles/metr): Independent AI-evaluation org behind the 'time horizons' benchmark — the task length a model can complete reliably on its own; the doubling-every-~4-months trendline and the 'upper end of what we can measure' verdict on Mythos Preview
- [Entities — People, Orgs, Tools & Projects](https://www.howardism.dev/articles/moc-entities): Map of Content for all 55 entity pages. See Home for concept domains.
- [OWASP](https://www.howardism.dev/articles/owasp): Open Worldwide Application Security Project; source of the agentic threat taxonomy cited throughout Anthropic's Zero Trust framework, coined the term 'least agency', and maintains the AI-BOM (CycloneDX ML-BOM extension)
- [AlphaProof Nexus](https://www.howardism.dev/articles/alphaproof-nexus): DeepMind framework for LLM-aided Lean proof generation; four agents (basic→full-featured); proof-sketch + EVOLVE-BLOCK interface; SafeVerify
- [Andrej Karpathy](https://www.howardism.dev/articles/andrej-karpathy): Co-founder OpenAI, ex-Tesla AI, Eureka Labs; coined "vibe coding," Software 1/2/3.0, "ghosts not animals," "agentic engineering"; originated the LLM-wiki pattern this vault runs on
- [Campfire](https://www.howardism.dev/articles/campfire): AI-native ERP (YC S23) pulling customers off NetSuite; custom foundation model + agent platform; Series B (Accel/Ribbit); doubling ARR/quarter since Q4 2024
- [Fiona Fung](https://www.howardism.dev/articles/fiona-fung): Leads engineering + product for Claude Code and Cowork at Anthropic (ex-Meta/Microsoft); "what served you prior may no longer"; rewrote team norms for the AI-native org
- [Google DeepMind](https://www.howardism.dev/articles/google-deepmind): Google's AI lab; built AlphaProof Nexus; Gemini models, AlphaProof, AlphaEvolve, and the open-weight Gemma line; opens the AI-for-mathematics domain and (via the Legg/Hutter 'From AGI to ASI' report) the theory-of-superintelligence cluster in this wiki; co-developer of the Cloud agent platform's AutoRater judges
- [John Glasgow](https://www.howardism.dev/articles/john-glasgow): CEO/founder of Campfire; 10yr corporate finance; founder-led-sales advocate; long-horizon "last job I'll ever have"
- [Lean](https://www.howardism.dev/articles/lean): Proof assistant whose compiler mechanically verifies every step; the `sorry` placeholder enables proof sketches; mathlib maturity gates the reachable frontier
- [Claire Vo](https://www.howardism.dev/articles/claire-vo): Host of the "How I AI" interview series (ChatPRD); interviewed Thariq Shihipar; runs a parallel component-visualization practice for non-technical stakeholders
- [Thariq Shihipar](https://www.howardism.dev/articles/thariq-shihipar): Engineer on the Claude Code team at Anthropic; "HTML is the new markdown", "compute allocator", and "the map is not the territory" framings; three HTML-first workflows plus a phase-ordered catalog of techniques for eliciting your own unknowns
- [Thinking Machines Lab](https://www.howardism.dev/articles/thinking-machines-lab): AI research lab behind interaction models (May 2026); harness-dissolves-into-model thesis; upstreamed streaming-sessions to SGLang; benchmarks against GPT-realtime / Gemini-live; research grants open
- [TML-Interaction-Small](https://www.howardism.dev/articles/tml-interaction-small): TML's first interaction model: 276B MoE / 12B active, audio+video+text in / text+audio out, 200ms micro-turns, async background agent; best turn-taking latency of any model; research preview May 2026
- [Chloe Li](https://www.howardism.dev/articles/chloe-li): Lead author of MSM paper (arXiv 2605.02087); Anthropic Fellows Program; designed all specs and experiments
- [Claude's Constitution / Model Spec](https://www.howardism.dev/articles/claude-constitution): Anthropic Model Spec / Constitution by Askell et al.; document specifying Claude's values + hard constraints (SP1–3, GP1–2); now also a direct training input via MSM
- [Anthropic](https://www.howardism.dev/articles/anthropic): AI safety company / vendor of Claude; mission-as-tiebreaker culture; ~30–40 PMs across teams; Mike Krieger leads Labs round 2
- [Boris Cherny](https://www.howardism.dev/articles/boris-cherny): Creator of Claude Code at Anthropic; phone-driven workflow with hundreds of agents; primary advocate of `/loop` primitive; "coding is solved (for me)" thesis
- [Cat Wu](https://www.howardism.dev/articles/cat-wu): Head of Product for Claude Code and Cowork at Anthropic; primary articulator of AI-native product cadence and engineer-PM convergence
- [Claude Code](https://www.howardism.dev/articles/claude-code): Anthropic's agentic coding product; created by Boris Cherny late 2024; TypeScript/React; CLI/desktop/web/mobile/IDE surfaces; central tool across all 2026 sources
- [Cowork](https://www.howardism.dev/articles/cowork): Anthropic's non-code knowledge-work agent product; sibling to Claude Code; output is decks/inbox/dossiers; same MCP/computer-use primitives
- [Matt Pocock](https://www.howardism.dev/articles/matt-pocock): Independent AI-coding educator; built Sandcastle library; smart-zone/grill-me/tracer-bullets pedagogical framing; "bad code bases make bad agents"
- [Mythos Model](https://www.howardism.dev/articles/mythos-model): Anthropic preview-tier frontier model and the first member of the Mythos-class tier (above Opus); gated for safety, used internally alongside Opus 4.7; its descendants Fable 5 / Mythos 5 shipped June 2026 as the first general-access Mythos-class models
- [Hermes Agent](https://www.howardism.dev/articles/hermes-agent): Nous Research's CLI agent + Gateway daemon (Telegram/Discord/Slack/WhatsApp); AGENTS.md/SOUL.md context split, bounded memory files, DM-pairing auth, container-as-security-boundary model
- [Symphony](https://www.howardism.dev/articles/symphony): OpenAI's open-source agent orchestrator (March 2026): turns Linear into a control plane for Codex, per-issue workspace, daemon-driven, SPEC.md-as-product, hedged 500% landed-PRs claim
- [Claude Opus 4.7](https://www.howardism.dev/articles/claude-opus-4-7): GA frontier model from Anthropic; direct upgrade to 4.6 at same price; literal instruction following, 1.0–1.35× tokenizer inflation, new `xhigh` effort, first post-Glasswing safeguards

## Syntheses

- [Foundation → Enterprise → Advanced: Is the Agent Access-Control Jump a Cliff?](https://www.howardism.dev/articles/agent-access-control-tier-migration): No cliff — Enterprise (ABAC + dynamic privilege elevation with return-to-baseline + mTLS + sandboxing) is the pragmatic midpoint between Foundation static roles and Advanced JIT/JEA; migration runs identity-first, then least-agency, then blast-radius
- [Agent Control Plane Patterns: Tickets, Loops, Specs, and Memory Files](https://www.howardism.dev/articles/agent-control-plane-patterns): Layered agent control-plane synthesis: tickets as durable work graph, loops as execution primitive, specs/context files as policy, memory as bounded recall, app protocols as runtime boundary
- [AI-Native Moats Under Frontier-Model Improvement](https://www.howardism.dev/articles/ai-native-moats-under-model-improvement): Frontier-model improvement stress-tests AI-native moats: product velocity and wedges must compound into behavioral data, domain artifacts, workflow embedding, counter-positioning, or external powers
- [AI-Native Product Org Bottlenecks](https://www.howardism.dev/articles/ai-native-product-org-bottlenecks): AI-native product-org bottleneck is accountable taste at speed: dogfooding trains taste, evals encode it, and accountability owns the consequences as output volume rises
- [How AI-Native Startups Avoid Speed Becoming Strategic Debt](https://www.howardism.dev/articles/ai-native-startup-speed-vs-discipline): AI-native startup speed becomes strategic debt unless bounded by validated problem, written scope, persistent architecture, accountable orchestration, and founder-owned customer signal
- [Opinions on Using AI Tools & the Future of the Software Engineering Role](https://www.howardism.dev/articles/ai-tools-opinions-and-future-of-swe-role): Debate map of four stances on using AI tools (bullish-insider / pragmatist-practitioner / skeptic-governance / architecture-thesis) + synthesis on the future SWE role: coding→deciding/verifying, role convergence, what stays human, which moats survive, honest caveats
- [Where Does Agent Harness Work Remain Durable as Models Improve?](https://www.howardism.dev/articles/durable-agent-harness-work): Durable harness work lives at external-reality boundaries: repo-local source of truth, mechanical verification, context budgeting, isolation, tool contracts, and human decision surfaces; capability scaffolding shrinks
- [How Do You Write Evals for Taste? Character as the Limit Case](https://www.howardism.dev/articles/evals-for-taste-and-character): Taste-driven features are eval-resistant but not eval-proof: the technique is conviction → dogfood-sourced failure signals → A/B variant measurement (MSM's method) → ~10 interpretable judgment-encoding evals; demonstrated on safety/values, still open on warmth/wit
- [The Future of Agent Interfaces](https://www.howardism.dev/articles/future-agent-interfaces): Interface future is layered: native interaction models for human collaboration, MCP/APIs for structured action, app protocols for agent runtimes, computer use for legacy GUI fallback
- [Does the Human-Facing Harness (HTML Artifacts) Hit Its Own Bloat Ceiling?](https://www.howardism.dev/articles/human-facing-harness-bloat-ceiling): Yes — HTML raises and reshapes the human-attention ceiling but can't remove it; bloat relocates from document-length to artifact-sprawl/rubber-stamping; the ceiling gets *more* binding as models improve (inverse of the shrinking model-facing harness)
- [Human-in-the-Loop Boundaries](https://www.howardism.dev/articles/human-in-the-loop-boundaries): Humans belong at allocation, understanding, design-concept, risk, and accountability boundaries; they slow the system down as manual executors, universal reviewers, or ceremonial approvers
- [Can Models Learn to Separate Instructions from Data? Durable Property vs Training Gap](https://www.howardism.dev/articles/instruction-data-separation-durable-or-trainable): Durable at the level that matters: the instruction/data boundary is trainable one delimiter at a time (hardening drives instruction injection to ~0%) but not in general — each closed boundary relocates the attack to the next finer one (instruction→data, then trusted→untrusted data), because the root cause is the LLM's probabilistic reading of inexact structural delimiters, an architectural fact. Newer models lower the per-boundary success rate but never produce a clean separation, and part of that gain is benchmark familiarity, not measured adaptive robustness — so the standing prescription across the cluster is to enforce the boundary outside the model with a deterministic action/data gate
- [Learning to Co-Work with AI: A Software Engineer's Field Guide](https://www.howardism.dev/articles/learning-to-cowork-with-ai-engineer-guide): Field guide for software engineers in the AI era: 6 skill clusters (taste, harness, alignment-first planning, agent-friendly architecture, verification, strategic positioning), daily practices, anti-patterns, 90-day plan
- [Orchestration vs Employee Framing: Reconciling the Founder's Playbook with HBR's Accountability Evidence](https://www.howardism.dev/articles/orchestration-vs-employee-framing-reconciliation): Reconciles the Founder's Playbook orchestration framings with HBR Kropp et al.'s accountability evidence; "orchestration as workflow design" survives the critique; "orchestration as mental model of agents-as-coworkers" does not; operational checklist for the disciplined founder
- [The PRD-Replacement Spectrum at AI-Native Speed](https://www.howardism.dev/articles/prd-replacement-spectrum-at-ai-native-speed): Four positions (grill-then-PRD → lighter-PRD → build-to-decide → prototype-is-spec) are one spectrum once you decompose the PRD into three jobs: AI-native speed dissolves specification, relocates alignment, and orphans rationale
- [RSI Growth Curves: Which Friction Binds First?](https://www.howardism.dev/articles/rsi-growth-curves-which-friction-binds): DeepMind's exponential/hyperbolic/S-curve growth shapes are Anthropic's compounding-efficiency/full-RSI/stalled futures seen from the dynamics side, not the policy side — one trichotomy described twice. Both labs converge on the same answer to 'which friction binds first': the slowest un-acceleratable step coupling the loop to reality (verification/oversight at org scale today, physical-experiment and institutional latency at the frontier), not cognition, which is racing and hasn't bent; data-wall and research-gets-harder demote themselves into compute, the abstraction barrier is the candidate fundamental blocker, and deliberate slowdown is the only friction humans must install.
- [When Does Verification Quality Determine Whether AI Automation Works?](https://www.howardism.dev/articles/verifier-quality-and-agent-automation): Verification-quality ladder from Lean/formal proof search through software CI and vulnerability reproduction; autonomy should rise only to the level the verifier can support
- [Where Does the Why Live?](https://www.howardism.dev/articles/where-does-the-why-live): Rationale (the 'why') is well-homed at authoring time — it's the recorded why-not-what conversation and the grilling session — but orphaned for future readers: AI-native methods delete the PRD, bury discussion in PRs, and the prototype shows what not why; code explicitly can't hold it, context files hold policy not product-rationale, and only the richer-artifact axis partly answers it
- [Open Questions Backlog](https://www.howardism.dev/articles/open-questions): _396 actionable open questions across 155 pages · 79 predictions · 9 notes · 21 in progress · 59 watching (entities), as of 2026-07-21._
- [How Much Signal Do Public Benchmarks Still Carry — and What Replaces Them?](https://www.howardism.dev/articles/benchmark-signal-and-what-replaces-it): Synthesis of the 2026 eval-science cluster: public benchmark suites carry far less independent signal than their count implies (133 benchmarks ≈ rank-2; accuracy saturates even after validity fixes) and the headline number is corrupted through four distinct channels (unnamed compute budget, contamination, vendor optimism, unvalidated judges) — but ordinal comparisons survive under verified invariances, and nothing replaces benchmarks wholesale: the field's answer is a five-part portfolio (predict-don't-run, re-instrument saturated suites, compute-controlled curves, production-sourced refresh, judge validation), with failure-mode discovery, contamination monitoring, and incentive shaping as the jobs only benchmarks still do
- [The Under-Review Divergence: Faros's Widening Crisis vs. CMU's Convergence](https://www.howardism.dev/articles/under-review-divergence-faros-vs-cmu): Resolves acceleration-whiplash's open question: the Faros-vs-CMU under-review 'divergence' is mostly measurement artifact — a vendor's adoption-depth *delta in unreviewed-PR count* over enterprise all-PRs vs a non-vendor *calendar-time share* of unreviewed agent PRs in open source — and both fit one story: total unreviewed output rises with volume while the share of agent PRs merged unchecked falls as teams learn risk-triage; the volume-concentration clause is supported (median per-project no-review ≈0%, pooled >50%; triage by PR type), and the residual disagreement is a forecast — whether triage discipline survives agentic authoring crossing from <1% to double digits
- [Single General Agent vs. Multi-Agent Coding Architecture](https://www.howardism.dev/articles/single-vs-multi-agent-coding-architecture): Resolves agent-harness-engineering's open question by re-drawing the line: a single general agent beats a bespoke hand-engineered multi-agent system as models improve (bitter lesson), but a monolithic-context agent does NOT beat role-separated context isolation + an independent grader — those survive model improvement because they fix structural constraints (quadratic attention, Goodhart), not model weaknesses
- [Opus 4.6 → 4.7 Changes and Multi-Agent Coding Considerations](https://www.howardism.dev/articles/opus-4-7-and-multi-agent-coding): 4.6→4.7 delta table + six hazards for multi-agent coding teams: role-based model selection, prompt re-tuning, harness invariants, per-agent context budget, unattended-fan-out safety, independent reviewer
- [When to Use Claude Opus 4.6 for Work](https://www.howardism.dev/articles/when-to-use-opus-4-6): Decision rules for Opus 4.6 deployment: solver-not-planner, elaboration-load-bearing tasks, brevity constraints, Pareto frontier check
- [What Are AI Tools?](https://www.howardism.dev/articles/what-are-ai-tools): Overview of AI tools landscape and categories