H
Howardism
Howardism · Vol. 03Plate III · No. 03

Open questions, unresolved.

Questions455Concepts177Domains14

The live worklist. Every unanswered question harvested from the wiki's concept notes, grouped by domain. Each links back to the note that raised it.

Agent Systems 60 open

  • Agent Context Files
    • How should context files and bounded memory files interact when they disagree? Memory is lossy and cache-delayed; the context file is authoritative but static. Which wins, and when?
  • Agent Harness Engineering
    • At what codebase scale does the AGENTS.md-as-table-of-contents approach need to be replaced with more sophisticated context routing?
    • How generalizable are these web-app-focused findings to other domains (scientific research, financial modeling)?
  • Agent Quality Flywheel
    • Both demo cycles fixed agents with *instruction-level* bugs and showed large one-cycle gains. What does the loop look like on failures that need tool, memory, or architecture changes — does "several iterations before metrics move" dominate in practice?
    • The custom rubric is authored by the same coding agent that will later propose fixes. Metric *choice* is upstream of grading — does decoupling need to extend to who defines the metric, not just who scores it?
    • Synthetic User Simulator scenarios bootstrapped the whole first cycle. How much of the 21%→5% delta survives on real-traffic distributions (the representativeness gap [[production-sourced-evaluation]] names)?
  • Build for the Next Model
    • How do you tell a "wait for the model" gap from a durable-harness gap *before* the next release? Get it wrong and you either ship vaporware or build a crutch you'll delete.
    • The bet depends on a reliable release cadence and a forecastable capability curve ([[task-time-horizon-scaling]]). What happens to "build for the next model" if model improvement stalls (the [[recursive-self-improvement|stalled-but-diffused]] future)?
    • Does the strategy generalize outside frontier labs, who have privileged visibility into the next model? An external team is betting on a release it can't see.
  • Claude Code Auto Mode
    • What false-positive rate does the classifier have on routine-but-aggressive refactors (e.g., large-file renames, `rm` of build artifacts)?
    • How well does the classifier generalize to custom tools / MCP servers where it lacks environment context?
    • Is the classifier's decision boundary documented/stable enough for security-sensitive orgs to certify, or is it effectively a black box whose behavior drifts with updates?
    • Does extending auto mode to API users change its calibration — is the classifier retrained for automation-heavy use, or held constant?
    • Compared to OS-level sandboxing (mentioned in [[claude-code-best-practices]] alongside auto mode), what's the defense-in-depth story? When should both be layered?
  • Claude Code Best Practices
    • What's the optimal CLAUDE.md length before instructions start getting lost? Is there a measurable threshold?
    • How does the Writer/Reviewer pattern compare to agent-to-agent review (as in OpenAI's Codex workflow)?
    • When does subagent overhead exceed the benefit of context isolation?
  • Client-Side Agent Optimization
    • How does combination-level optimization interact with continual model releases? If Claude Opus 4.7 ships next month, does the full Pareto frontier need re-running, or do warm-started bandits adapt cheaply?
    • At what pipeline depth does the combinatorial search become intractable even for Arm Elimination? The paper tests up to ~81 combinations; production pipelines with 5+ roles and 10+ candidate models each blow past that.
    • Does the "weak planner + strong solver" pattern generalize, or is it specific to HotpotQA's delegation dynamic? Recommender-critic, drafter-editor, and retriever-generator topologies might invert.
    • What's the right way to re-evaluate when the tool environment changes? AgentOpt assumes fixed tools — adding or removing a tool potentially invalidates the whole frontier.
    • Is there a cheap per-call classifier that can predict which combination will win on a given query, avoiding combo-level evaluation entirely?
  • Codex App Server Protocol
    • How does the App Server protocol compare in detail to MCP? Both expose tools to a model, but App Server is *inside* the Codex runtime while MCP is *outside*. When does each win?
    • Is there a public schema registry so external orchestrators can target specific App Server versions without `generate-json-schema`?
    • The "dynamic tool calls (experimental)" caveat — what's the stability roadmap? Symphony depends on this for its security model.
    • How well does the protocol handle multi-modal turns (image inputs, screenshot attachments)? The spec is text-focused.
    • Is there an analogous protocol on the Claude side, or is Claude's equivalent exclusively the Agent SDK + tool-use API? Comparing the two would clarify when "drive an existing CLI" beats "build on the SDK."
  • Context Window Smart Zone
    • Does the smart-zone marker scale with model size, or is it bounded by attention architecture? Pocock observes "the dumb zone has become less dumb lately" but pegs it at 100K through 2026.
    • How should harnesses surface remaining smart-zone budget to the user — token count, percentage, or a richer signal?
  • Deep Modules for Agents
    • How big is "deep enough"? Pocock's example modules are several hundred LOC; Ousterhout's textbook examples are larger. There's a sweet spot; not articulated.
    • For ports/adapters codebases, does the deep-module advice transfer cleanly? The "small interface" is the port; the "large behavior" is the adapter. Probably yes, but not exercised in source.
    • Refactor cost vs benefit: when is "improve-code-base-architecture" worth running on a working repo?
  • Deep Research Agents
    • DRACO grades single-turn interactions only. How much of real deep-research value is in the multi-turn loop (clarifying questions, follow-ups) that the benchmark doesn't yet measure?
    • Factual accuracy is the weak axis everywhere — is the fix better retrieval, better verification-in-the-loop, or a tool-grounded check the way Lean grounds proof search?
  • Failures That Look Like Success
    • Is "internal state correct, final message stale" a general LLM-agent failure signature (state/utterance divergence) or an artifact of session-state architectures like ADK's? A cross-framework tally would tell.
    • What fraction of production agent failures are silent-contract violations vs. loud errors? The 14/15 and 3/4 numbers are demo-sized; telemetry-scale data ([[production-sourced-evaluation]]) could ground the class.
  • Harness Shrinkage as Models Improve
    • Does *all* prompt scaffolding eventually migrate into the model, or does some remain — e.g. organization-specific style, security rules, brand voice?
  • Latent vs. Deterministic Space
    • Tan asserts the "wrong side" diagnosis covers most AI-engineering bugs. Does any incident/failure taxonomy (agent postmortems, eval failure analyses) actually classify failures by computation-locus, and what fraction lands in each side?
  • LLM-as-Compiler Knowledge Base
    • At what scale does the no-vector-database approach break down? Karpathy's ~100 articles fit in context, but what about 1,000+?
    • How to handle conflicting information across sources during compilation?
    • What's the optimal granularity for concept articles — one concept per article, or clustered by theme?
    • How effective is the synthetic training data → fine-tuning pipeline in practice?
  • Loop Engineering
    • Osmani's cost caveat is unquantified: at what token budget does a continuously-running loop stop paying for itself, and how do you instrument that? (Cf. [[agent-loop-pattern]]'s "who owns the budget when the model schedules its own loops.")
    • If `/goal`'s stop-check is itself a model, what verifies the verifier? The maker/checker split pushes the trust problem up a level, not away.
  • MCP and Computer Use
    • MCP security model: as the playbook prescribes wiring MCP into Salesforce, Gmail, Calendar for solo founders, the attack surface scales with adoption. **Now addressed** by [[zero-trust-for-ai-agents]] (tool poisoning, rug pulls, the first in-the-wild malicious MCP server) — see "MCP as a security surface" above. Open residual: how does a solo founder realistically *run/host and self-sign every MCP server* the framework recommends, given that the appeal of MCP was zero-integration-effort? **Sharpened by [[mcp-tool-poisoning|ShareLock]]:** the cheaper alternative to self-hosting — *scan the tool descriptions with a guard model* — is information-theoretically defeated by threshold fragmentation, so the lightweight mitigation doesn't hold and the burden falls back on run-your-own-server or downstream action-layer authorization. **And [[mcp-tool-poisoning|Agentjacking]] shows run-your-own-server *itself* isn't sufficient:** when the server is a legitimate observability platform relaying attacker-injected data (fake Sentry errors), self-hosting/vetting the server catches nothing — the untrusted input rides in on its data, so the residual burden falls squarely on the downstream data/action layer (provenance tracking + an out-of-band action gate), not on server hygiene.
    • How does Cowork's computer-use guardrail compare to Claude Code's auto-mode classifier? Different deployment context, possibly different risk profile.
  • Optimizer–Evaluator Decoupling
    • Does decoupling need to extend upstream to metric *design*? An optimizer that authors its own rubric has a subtler channel to game than one that merely reads scores.
    • How much independence is enough — different model family, different vendor, different modality of check (model judge vs. compiled test vs. production telemetry)?
  • Parallel Agent Orchestration
    • Summed-overlap runtime can exceed 24h/day — it measures *agent* effort, not *human* attention. What is the human's actual oversight load per concurrent agent, and where does it saturate ([[ai-brain-fry]])? **Sharpened:** [[configurable-human-participation|HAS-Bench]] reframes the *shape* rather than measuring the load — in a controlled (LLM-simulated) benchmark the value of human input is configuration-dependent and *non-monotonic* (a right-timing / right-channel / right-authority sweet spot; more agency brings diminishing and sometimes negative returns), so per-agent oversight is likely a returns-curve with a peak, not a linear cost that hits a wall — but it is single-human, single-task, so it does not measure real concurrent-oversight load.
    • Concurrency is measured over one week. Is 5+-agent management a stable practice or a burst around specific large tasks?
  • Repository Exploration Subagent
    • **Does the gain survive better main models?** The same-model-exploration result suggests the architectural benefit is somewhat model-independent, but the trained-explorer margin may erode as frontier models get cheaper and better at staying in their smart zone unaided. The bitter-lesson question is unresolved.
    • **Prune vs. don't-pollute.** SWE-Pruner removes context after the fact; FastContext avoids accumulating it. Are these complementary (prune the solver *and* delegate exploration) or substitutes? Not tested together.
    • **How small can the explorer go?** The authors flag 1.7B / 0.6B as future work — if the recipe holds, the explorer becomes nearly free and the architecture dominates.
    • **Generality beyond Mini-SWE-Agent.** Only one (deliberately minimal) main-agent scaffold is tested; richer harnesses with their own memory/subagent orchestration may already capture part of the benefit or interact differently.
    • **Patch-derived reward leakage.** Training the explorer's reward on the gold patch's file/line ranges risks overfitting to where fixes *landed* rather than where evidence *lives*; the F1-vs-recall behavior partly mitigates this, but the proxy is imperfect.
  • Ticket-Driven Agent Orchestration
    • What's the right granularity for ticket size when the unit is "what one agent does in one workspace"? The post implies "much larger units of work" become viable, but how does that interact with the `agent.max_turns` limit (default 20)?
    • How do you prevent a ticket-extension cascade when agents file follow-up tickets liberally? Is the only governance check human triage at the `Todo`-state queue?
    • Does this pattern generalize to non-software work (research, ops, content)? The DAG dependency model and prompt-as-policy file should transfer; the per-issue workspace doesn't obviously.
    • When an agent gets a ticket "completely wrong" (mentioned in the post), how is the lesson fed back into the system? Symphony's answer is "add guardrails and skills" — what's the institutional process for that?
    • How does ticket-driven orchestration interact with sprint planning / OKRs / roadmap work that operates on aggregates of tickets? Does the abstraction collapse when tickets are scoped that small?

Agent Security 47 open

  • Agent Data Injection (ADI)
    • The complete defense (CaMeL Strict) costs ~50pp of utility. Is there a fine-grained trusted/untrusted data-isolation scheme that stops ADI **without** the deterministic-flow-tracking utility collapse — or is the trade fundamental?
    • Randomization is cheap and effective for key-value formats but useless for **unstructured formats** (Markdown, prose tool output). What protects the formats a nonce can't be attached to?
    • Do internal/white-box monitors detect an ADI payload at all, given it is engineered to read as trusted data rather than as an attack? (Untested; the tension flagged under Connections.)
  • Agent Identity Management System (AIMS)
    • **Mission → authorization is out of scope.** The hardest part — translating a natural-language mission into concrete scopes/resources safely — is explicitly deferred as a "planning step." A manipulated planning step requests over-broad authorization; AIMS gives it clean primitives but no account of securing the translation itself.
    • **No empirical evaluation.** Unlike [[out-of-band-prompt-injection-defense]] (which at least ran one adaptive reproduction), AIMS is a design document with no measured attack-resistance — its security rests on the composed specs' own (mostly non-agentic) threat models.
  • Agent Supply Chain Risk
    • "AI vendoring" as a standard response inverts decades of "don't reinvent the wheel." How is a model-reimplemented dependency itself verified and maintained — does it just relocate the risk?
    • The 250-doc backdoor persists through SFT/RLHF. What detection exists for an already-poisoned model you didn't train, short of behavioral red-teaming? **Adjacent evidence (2026-07, benign analog):** [[benchmark-contamination-decontamination|UBD]] shows the *correction* half is tractable for a benign training-exposure effect — it repairs benchmark memorization from the deployed checkpoint alone, using **deep-ensemble uncertainty** and needing neither the training data nor a clean reference model. It doesn't transfer directly (contamination inflates accuracy; a backdoor is a targeted malicious trigger that survives safety training), but it suggests ensemble-disagreement is a signal worth probing for the malicious case too.
  • Agentic Prompt Injection
    • Spotlighting and constitutional classifiers each leave a residual (2%, 5%). Stacked, what's the realistic floor, and does it hold against adaptive attackers who know both are deployed? *(Partly answered by the Opus 4.8 live bug bounty: adaptive expert red-teamers still find attacks on the bare model; deployed probes add uplift but don't zero out the residual. Sharpened by [[task-specification-injection-surface|AutoDojo (Ma et al. 2026)]]: a **0% static ASR is not a floor** — a **cheap black-box** adaptive attack, not just a white-box one, recovers 28% overall (64% on action-open tasks) against a filter that scored 0% static. So the realistic floor against a filter defense on a vulnerable model is double-digit, not zero. But the same attack barely moves ASR on newer capable base models — showing the floor is a property of the model, not the layered filter defense.)*
    • Why did Opus 4.8 regress on prompt-injection robustness relative to Opus 4.7 despite broad alignment gains — a capability/robustness tradeoff, or an artifact of harder adaptive evaluation?
  • AI-Accelerated Offense
    • "Fundamentals strong enough that scanning finds fewer bugs" assumes defenders run the scanners first. What happens to organizations that can't afford continuous model-driven scanning?
  • Autonomous Defense
    • "Measure agreement against a human for two weeks, expand if tolerable" — what agreement threshold is tolerable, and who owns the residual false-negative risk when the model dispositions an alert the human never sees?
    • Defensive agents are high-value targets (compromising one yields powerful capabilities). Does concentrating detection in an Agentic SOAR create a single point of catastrophic compromise the distributed-human model didn't have?
  • Blast Radius (Agentic)
    • The framework prefers identity-based isolation over network segmentation, but most enterprises have heavy segmentation investment. What's the migration path, and does dual-running create new gaps?
    • Multi-agent compartmentalization increases the *number* of identities to manage; at what point does identity-management overhead create its own attack surface?
  • Capability Gating Is Not Authorization
    • The `0/48` static and `0/29` adaptive results are **suite- and budget-bounded** (40 iterations, a GLM-5.2 attacker, one author's vector corpus). Does the deterministic gate hold under longer adaptive budgets, stateful side channels (DNS rebinding, timing), or multi-agent policy-poisoning — the future work the paper names?
    • The `authz` allowlist stops **value-redirection** but not corruption of legitimately-variable data. Is there a per-call scheme that constrains free-text / open-ended arguments without collapsing utility — or is that irreducibly the province of provenance/data-flow tracking ([[out-of-band-prompt-injection-defense|CaMeL Strict]], at ~50pp utility cost)?
    • **Out-of-band policy is load-bearing but under-specified for authoring at scale.** The paper forbids any model-sourced policy element; who authors and maintains the verified sets, ceilings, and allowlists for a large tool surface, and does that authoring burden cap the control to high-stakes (money-moving) tools?
  • Impossible, Not Tedious (Design Test)
    • Defense-in-depth traditionally *stacks* friction controls on the theory that enough of them sum to a barrier. Does this test invalidate layered friction, or just demote it below capability-removal?
    • Some controls are friction for humans but barriers for agents (or vice versa). Is the test agent-relative, and how do you evaluate it for mixed human/agent threat models?
  • Least Agency
    • Least agency adds a *frequency* dimension ("how often"), but the framework also says rate limits are friction, not barriers ([[impossible-not-tedious-test]]). How is frequency-limiting both a least-agency control and a friction-only one — context-dependent?
  • MCP Tool Poisoning
    • **Cross-tool / stateful detection.** Information-theoretic secrecy defeats per-tool scanning by construction. Is there a detector that reasons over the *aggregate* of a server's tool set (and its update diff history) to spot the share pattern — and can it do so without an infeasible combinatorial search over `t`-subsets?
    • **Automating the attack chain.** The reconstruction-trigger prompt engineering still relies on manual effort; the authors flag feedback-driven prompt optimization (à la [[task-specification-injection-surface|AutoDojo]]) as the next escalation. How much does automation raise ASR against aligned models?
    • **Does a strict-access-control agent architecture close it?** The authors note agents with fine-grained interaction / strict access control can force user consent and expose the attack — but "the majority of users lacking safety awareness opt for auto-approval," reopening the convenience-vs-security trade. Where does the realistic equilibrium sit?
    • **Independent replication of the malicious-data-via-legit-server branch.** Tenet's Agentjacking figures (2,388 orgs, 85% success, a $250B victim) are *vendor-reported from controlled testing*, not independently measured — and the branch is now known to be *plaintext trusted-server data relay*, not fragmentation/rug-pull (resolved above). How prevalent is this branch beyond Sentry — any observability / ticketing / log / CI MCP that relays externally-influenced data as trusted output — and does an independent measurement confirm the ~85% agent-execution rate on current models?
  • Memory and Context Poisoning
    • Long-term memory drift is defined as undetectable per-change. Drift detection requires a baseline — but if the baseline itself drifts (Advanced "continuous baseline refinement"), how is a slow poisoning attack distinguished from legitimate evolution?
    • Integrity hashing detects *modification* but not *malicious-but-valid* memory written through a legitimate (injected) interaction. What catches semantically-poisoned-but-cryptographically-intact memory? *(Strongest answer yet: [[non-malleable-memory-authority|TMA-NM]] (Louck, arXiv 2606.24322) reframes the question. The "malicious-but-valid" class is exactly a **laundering** attack — untrusted content paraphrased / echoed / fake-corroborated so it reads benign and stays cryptographically intact — and a machine-checked separation theorem proves **no content- or lineage-based detector is sound against it** (T1); content-judge sweeps confirm no threshold reaches 0% ASR at full utility. So the fix is not to *catch* it but to **prevent by construction**: bind each item's authority-to-act to its true origin at write time, non-malleably, so a laundered item is `act=none` however benign it looks — 0% attack-success across 8 frontier models at 100% legit-utility, vs up to 68% for content/lineage baselines. It **prevents rather than detects**.)*
  • Non-Malleable Memory Authority (TMA-NM)
    • The full guarantee is **machine-checked on a bounded model + a machine-checked inductive invariant**, not a fully mechanized unbounded deductive proof (TLAPS/Lean). Does the unbounded theorem hold once mechanized for arbitrary slots, sessions, and thresholds — the future work the inductive invariant sets up?
    • **Value attribution in a black box.** The headline results set origin by channel (not text-matched), but a real deployment attributing *which* retrieved value the agent used needs value-level taint propagation through nested structured payloads. Is a capability-token design (authority as an unforgeable token flowing with sub-values) enough, or does implicit/aggregate reconstruction — assembling a security-relevant value from several low-integrity fragments by in-context reasoning — leave a residual gap the boundary monitor can't taint?
    • **Corroborator availability in the wild.** How often do two *genuinely independent* trusted sources exist for routine actions? The `uncorr-auto` fallback converts missing corroboration into a one-time user confirmation — but at scale that reintroduces the approval-fatigue surface the [[out-of-band-prompt-injection-defense|out-of-band literature]] flags for in-the-loop tasks. Which untrusted-sourced actions can be corroborated without a human, and which are stuck asking?
    • **Answer-bias is still open.** TMA-NM by design does not touch non-consequential answer-biasing (surfaced with provenance). As agents produce more text people act on, is the retrieval-to-*text* path — not just retrieval-to-*action* — the next thing that needs an integrity guarantee?
    • **Cross-agent memory is out of scope.** Extending origin-bound authority across a *federation* of origin authorities (the multi-agent / A2A case) is named as a natural next step; does non-malleability compose across agents, or does the inter-agent channel reopen the laundering surface?
  • Off-Host, Identity-Bound Authorization
    • **The trust-boundary premium is unmeasured.** aiAuthZ argues off-host beats in-process, but its own comparison is only against argument-only / delegation-token ablations, not a matched-utility head-to-head vs CaMeL or Progent. Does the separate trust domain buy measurable security *beyond* the shared argument policy — the author's named next step, and the crux the single-author-preprint caveat should keep open?
    • **Who authors the policy at scale?** Like ScopeGate, the off-host policy (role allowlists, path/URL/recipient constraints, ceilings) is operator-authored and out-of-band. The same authoring-burden question applies: does maintaining verified sets for a large tool surface cap the control to high-stakes tools?
    • **The non-repudiation gap.** Symmetric HMAC gives operator-facing authenticity but no third-party non-repudiation; is the proposed asymmetric mode deployable at the microsecond latencies that make the gateway attractive, or does key management erode the cost advantage?
    • **The active-user residual.** Per-message identity is decisive only when the attacker is a *different* principal. An injection firing under the active owner's own authority is bounded only by argument/rate policy — the same limit as every value gate. What closes *that* half beyond provenance/data-flow tracking ([[out-of-band-prompt-injection-defense|CaMeL Strict]])?
  • Out-of-Band Prompt-Injection Defense
    • The reproduction bounds a **single black-box attack template on one weak model**. Does a stronger optimized **white-box (GCG)** attack, or one **confined to already-authorized actions** (achieving the injection goal without any policy violation), break the deterministic gate the way adaptive attacks broke in-band defenses? The authors name this as the next study. *(The "already-authorized actions" half is now partly addressed by [[capability-gating-vs-authorization|Mellafe Zuvic (2026)]]: it splits "already authorized" into capability-authorized-but-not-value-authorized (a well-typed `account=acct_ATTACKER` — blocked by ScopeGate's per-call value `authz` stage, 0 bypasses in-corpus) versus genuinely-within-policy (corrupting a legitimately-variable value the agent acts on — the residual that survives, the same class ADI rides past Progent at 22.2%). So a within-capability attack is defeated where an allowlist constrains the corrupted argument, but not where the corrupted value legitimately varies. The white-box question stands.)*
    • Progent's policy is **LLM-authored** — the one model-based component. Does the "gate must not be a model" principle fully hold when the *policy* is still written by a model that can be talked into widening the allowlist? (The adaptive attack targeted exactly this and failed, but possibly due to the confound.)
    • **Provenance-aware retrofit**: can a monitor that sees only tool I/O track *transitive* provenance to enforce the Biba invariant directly (rather than approximating it with argument patterns), without instrumenting the model's hidden reasoning? The paper flags this as the design problem the systematization implies, unanswered. *(A concrete construction now exists for the **cross-session memory** slice: [[non-malleable-memory-authority|TMA-NM]] (Louck, arXiv 2606.24322) enforces the Biba invariant **directly** — write-time origin binding + non-malleable propagation, with untrust propagated at the tool-call boundary — and machine-checks it in TLA⁺. The caveat sharpens rather than closes the question: it is **not** "sees only tool I/O" — it needs an authenticated origin-labeling oracle (mTLS / audience-bound OAuth / signed responses) at the trust boundary, and full value-level taint through nested structured payloads is still future work.)*
    • Does the ~6× reduction and the "held under adaptive attack" result survive on a **strong** agent with a fatter natural attack surface (the 7B's low absolute numbers and workspace's 0% are artifacts of a weak agent), and with a **stronger policy model** than the local 7B? *(Partly answered by [[task-specification-injection-surface|AutoDojo]]: Progent and DRIFT held under a cheap black-box adaptive attack across five models including capable ones (GPT-4o-mini, Gemini-2.5-Flash), not just a weak 7B — but against a **black-box** attacker; the white-box question below stands.)*
    • The utility cost (~45%→~26%) and **~15× LLM-call overhead** are large. Is deterministic out-of-band enforcement economically deployable at production scale, or does the cost cap it to high-stakes action surfaces?
  • Task-Specification Effects in Prompt Injection (AutoDojo)
    • AutoDojo is the *weakest* realistic adaptive attacker (black-box, six iterations, binary signal). The authors note every axis — richer feedback (traces, token probs), non-semantic surface tricks, or a payload reshaped to resemble the user's plausible intent — is strictly stronger, so the reported ASR is a lower bound. How far do the system-level defenses hold once the payload is reshaped to look like a task-relevant action (the natural route to evading action constraints)?
    • Can a gradient-optimized (white-box) injection be *seeded* into the loop and adapted further by the LLM search — combining white-box strength with black-box adaptation? The authors flag this as curious and untested.
    • The task-specification axis is measured on 6 action-open tasks in 3 suites. Does the action-open ≫ specified ordering (and the system-level inversion) hold at scale and on stronger agents, and is "fraction of tasks that are action-open" a usable per-deployment risk metric?
    • If action-open tasks are the injectable ones and also the everyday default for non-expert users, is the practical prescription to *forbid* action-open delegation (force the user to name the action), pushing the security burden back onto task specification — the same discipline [[unknowns-as-the-agentic-bottleneck|unknown-elicitation]] asks for on quality grounds?
  • Zero Trust for AI Agents
    • The framework treats every Claude Code "Pro-tip" as a reference implementation. How much of the framework is vendor-neutral vs. tacitly assuming the Anthropic stack?
    • "Foundation floor raised" implies a moving baseline. How fast does the tier ladder actually shift, and who arbitrates it (NIST/NSA cadence vs. model-capability cadence)?
    • The framework is explicit that it is *not* legal/compliance assurance. Where does self-attested Zero Trust maturity meet auditable regulatory requirement?

AI Coding Practice 50 open

  • Acceleration Whiplash
    • Faros's own deferred question: do the bug/incident increases persist when **normalized for PR size**, or do larger PRs account for most of the quality deterioration? (If the latter, hard PR-size limits are the highest-leverage fix.)
    • Code churn +861% is genuinely ambiguous (Faros lists three explanations: rework of AI code, productive legacy refactoring, or accelerated polish). The cross-customer metric can't resolve it — a real gap, not a finding.
  • Agentic Coding Work-Composition Shift
    • The window is seven months and the value proxy is coarse/relative. How much of the +27% is genuine task-complexity growth vs. classifier/marketplace-matching drift?
    • The study excludes headless/SDK/IDE usage — a "substantial share," and likely the most automated/end-to-end. Does including it accelerate or reverse the composition shift?
    • If "fixing" keeps falling, is that because models break less, or because broken-code work is migrating to non-interactive pipelines this study doesn't see?
  • Agentic Work Systematization
    • Custom skills encode org-specific context — but who maintains them as the codebase and conventions drift? Systematization could itself become a debt surface ([[agentic-technical-debt]]) if skills rot.
    • Does systematization *cause* deeper delegation or merely *correlate* with already-intensive users? The paper shows the association, not the direction.
  • AI as Primary Author
    • The 60% figure aggregates very different tools and modes (autocomplete acceptance vs. agent-applied diffs). What does "acceptance" mean when the agent applies the change directly and the human's "acceptance" is not reverting it? **Sharpened** by [[review-as-the-control-point]]: on GitHub, agent PRs are most often examined *only by the developer who invoked the agent* (author-only review 40.1% vs 21.5% for human PRs). Whether that counts as review at all is a definitional choice (agent-as-author ⇒ a second set of eyes; agent-as-tool ⇒ self-review) that literally flips the sign of the trend — so "acceptance" and "review" blur into the same unresolved construct.
  • Building Is Cheap, Arguing Is Expensive
    • When does "generate three and compare" become wasteful — at what decision weight is a real argument (or a design doc) still cheaper than three implementations?
    • If design discussion lives in PRs/prototypes, where is the *rationale* recorded for future readers — does the "why we chose this" knowledge survive, or does it share the staleness problem of [[code-as-source-of-truth]]?
  • Code as Source of Truth
    • What knowledge genuinely *can't* live in the codebase (org strategy, the "why," cross-team context) and therefore still needs a durable doc — and how do you keep that small slice current?
    • If onboarding is "ask Claude," what happens to the tacit knowledge that was previously transferred socially in deep-dives — is it captured anywhere, or quietly lost?
  • Compute Allocator
    • Is 1% a Thariq-specific number or a regime? For larger, more code-heavy projects the production residue is presumably higher; what sets the ratio?
    • Allocation quality is hard to measure — what's the feedback loop that tells an allocator they spent compute *badly* (vs. just spending a lot)?
    • Does treating humans as "compute allocators" risk the [[ai-brain-fry|oversight-fatigue]] / [[human-ai-accountability-redesign|accountability]] failure modes the HBR research flags, where the human nominally decides but actually rubber-stamps?
  • Configurable Human Participation
    • The "human" is an LLM simulator (GPT-4.1) that also judges — how much of the configuration-dependent structure is a property of *human-agent collaboration* versus an artifact of GPT-4.1 modeling both sides? The Appendix C.6 simulator-swap is the only check, on a subset.
    • A4 is instantiated as a *fixed one-shot* proactive intervention. Real proactive humans time their input adaptively — does the "premature/distracting intervention breaks tasks" result hold, worsen, or vanish under a human who chooses *when* to interject?
    • The optimal channel is shown to be pattern-specific, but pattern is an oracle label assigned at construction. Can an agent *infer* which pattern it is in (and thus which channel to solicit) at runtime — the actual deployed skill the paper says agents lack?
  • Design Concept Grilling
    • How does grilling change for team work where multiple humans need to align? Pocock's hint: pair-program with the agent in the room, treat it as a third interlocutor.
  • Disposable Micro-Apps
    • Where's the line between a disposable micro-app and tool sprawl? If every edit spawns a bespoke UI, does the workflow fragment?
    • Does the copy-back-to-markdown round-trip generalize beyond config-shaped data (rules, tables) to richer artifacts?
    • Could these micro-apps be templated/reused rather than regenerated — and at what point does that defeat the "disposable" framing and turn into [[living-design-system|durable tooling]]?
  • HTML as the New Markdown
    • HTML is heavier to diff and version than markdown — what happens to plan history and review when artifacts are single-file websites? ([[disposable-micro-apps]] copy-back-to-markdown is one patch.)
    • Does this generalize past one expert practitioner, or does it require Thariq-level fluency with Claude to be worth the overhead?
  • Living Design System
    • How does the `design_system.html` stay in sync as the codebase evolves — re-extract on a cadence, or wire it into CI?
    • Does a rendered, model-readable design system measurably improve on-brand output vs. a plain CSS/token file, or is the win mostly human legibility?
    • At what project size does maintaining the artifact cost more than the consistency it buys?
  • LLM-Assisted Grey-Literature Theory Building
    • The authors couldn't locate the **saturation point** because synthesis was manual — how few documents actually suffice, and can a cheaper sample match the 3,100-doc theory?
    • Automating the **codes→theory** step failed with a naïve bottom-up prompt; is that a prompt/scaffolding limitation or a genuine ceiling on LLM interpretive synthesis over thousands of codes?
    • The three-lens design manages coder bias, but the **relevance judge and segmenter are single-model** — do those upstream gates impose their own systematic slant on what reaches the codebook?
  • Outsource Your Thinking, Not Your Understanding
    • If understanding is the bottleneck, is the highest-ROI skill *learning how to build understanding fast* (knowledge-base hygiene, asking the right projections) — and can that be taught?
  • Planning / Execution Division of Labor
    • "Decision attribution" is inferred from transcripts. When Claude proposes a plan and the user assents, is that scored as the user's planning decision or Claude's? The rubber-stamping boundary is exactly where the measure is hardest.
    • Headless/SDK/pipeline usage (excluded here) is where execution autonomy is highest and planning is front-loaded into a single prompt — does the 70/20 split survive there, or collapse toward full delegation?
  • Review as the Control Point
    • Every one of the 67 relationships is a **hypothesis, not a finding** — the paper's explicit call is for causal-estimand studies (controlling for the other constructs) to confirm, reverse, or drop each edge. Which of P1–P17 survive measurement?
    • The paper's own question: *which decisions, under which conditions, push the system toward the virtuous loop rather than the vicious one?* — the system-dynamics leverage-point analysis it gestures at but doesn't run.
    • The three contested edges (automated review → quality/security, P9; governance → latency, P17; and one more) are contested *because* their sign is moderator-set — what are the moderator thresholds that flip them?
  • Telemetry vs. Survey Measurement
    • Surveys and telemetry measure different things (felt productivity vs. system outcomes); is the "contradiction" partly a category error — both true at their own layer — rather than one being wrong?
  • The Three Loops of AI-Native Building
    • Ng asserts the developer's QA burden fell "significantly." [[acceleration-whiplash|Faros's 2026 telemetry]] measures the opposite for production orgs. Is the split really 0-to-1-vs-production, or is Ng's self-report subject to the same [[automation-optimism-link|optimism bias]] the survey literature keeps finding?
    • The external loop is the unshortened one. Is that physics (users take time to react) or an unautomated frontier (synthetic users, [[deployment-simulation|deployment simulation]] applied to products rather than models)?
  • The Verifiability Thesis
    • Where's the boundary of "council of LLM judges" reliability — does it hold for genuinely contested value judgments, or only for quality/coherence?
    • The "labs care" dependency is fragile: capabilities can appear or stagnate based on lab priorities you don't control. How should a product hedge against the data-distribution rug-pull?
  • Unknowns as the Agentic Bottleneck
    • Is "the first model bottlenecked by my unknowns" a property of Fable or of Thariq? A frontier-lab engineer with deep model fluency hits the human-side ceiling before an average user does — which would make this a leading indicator rather than a current universal.
    • The quiz gate is self-administered and self-graded (by the model, on the model's own work). What stops a comfortable equilibrium where the quiz gets easier as the reviewer gets lazier? Cf. the maker/checker problem in [[verification-as-the-new-bottleneck]].
    • Elicitation has a cost. Every technique here spends a session's worth of tokens and attention on *not building*. Nothing in the source bounds when the blindspot pass costs more than the bug it prevents.
    • If unknown knowns are extractable, are they extractable *once*? Does a codified blindspot pass become a skill file that permanently narrows the gap, or does each new territory reopen it?
  • Verification as the New Bottleneck
    • Fung's own open question: "How far do you push fully automated reviews?" — where's the speed/safety balance, and how do you keep humans confident without re-introducing the review bottleneck? **Sharpened** by [[review-as-the-control-point]]: full automation reliably raises review throughput and cuts latency (its P8), but its effect on *code quality and security is contested* (P9), and the latency effect of a review-governance policy flips sign by calibration — a risk-tiered policy that gates only material changes lowers latency, a blanket policy raises it (P17). So "how far" has no single answer: the safe frontier is set by automated-reviewer capability and process design (two of that page's three moderators), not by a fixed dial.
    • If CI/build is the hidden jam, does verification infrastructure (test runners, CI capacity) become the actual capex of an AI-native org?
  • Vertical Slice Tracer Bullets
    • Can the planner agent be trusted to slice vertically once told to, or does it need a verifier that flags horizontal slices? Pocock's experience: it needs the verifier, at least through 4.7.
    • How should slice granularity be tuned? Too thin = many merge conflicts; too thick = back to horizontal.
  • Vibe Coding vs. Agentic Engineering
    • Karpathy hints at "one domain that's very [valuable]" for founders but won't say which (didn't want to "vague-post on stage"). What verifiable RL-environment domain is he gesturing at?

Evals & Benchmarks 34 open

  • Benchmark Contamination and Decontamination
    • **Can the ensemble be derived from one released model?** The whole method rests on having several checkpoints differing in batch ordering; the authors flag single-checkpoint ensemble derivation (e.g. via cheap perturbations) as the key unlock for adoption. Until then it needs provider cooperation to release a LoRA ensemble.
    • **Does it extend past MCQ?** UBD-Debiasing is classification-only today; whether per-decoding-step debiasing recovers the clean distribution for open-ended generation (where contamination shows as near-verbatim reproduction) is untested.
    • **Is batch-order sensitivity a reliable memorization tell at pretraining scale?** The signal was validated on 3B models with 5 LoRA seeds and induced contamination; whether the high-confidence-high-variance signature survives full-scale pretraining and real (not synthetically injected) leakage is open.
    • **Does correcting toward an *ensemble-averaged* uncontaminated reference introduce its own bias?** The D_KL/D*_L1 targets are themselves an average over a 5-member uncontaminated LoRA ensemble; how much the "clean" target moves with ensemble size/composition is unexamined.
  • Benchmark Score Redundancy
    • **Can outlier models be anchored without any scores?** BenchPress fails on a model whose capability profile has no close neighbor in the matrix; the authors propose folding in external metadata (training-data composition, architecture, size) to compute model-to-model similarity *before* any benchmark is run, but do not build it.
    • **Does the low-rank treatment carry beyond text/vision?** Audio/speech, robotics/embodied agents, and scientific-simulator ecosystems are untested; whether the same rank-2 structure holds there is open.
    • **Does vendor optimism manufacture the correlation?** Four-fifths of the scores are provider-reported and possibly inflated; the paper flags this could inflate apparent cross-benchmark correlation but cannot separate it — would a fully standardized re-evaluation still be rank-2, or is some of the redundancy an artifact of shared reporting bias?
  • Compute-Controlled Benchmarking
    • Can you certify "no benchmark-maxxing" — verify a reported score used a stated, reproducible compute budget rather than a hidden best-of-N scaffold?
    • Compute has several units (tokens, dollars, wall-clock). They diverge (a more efficient model wins on cost but not always on tokens). Which x-axis is the honest one, and does it depend on the buyer? ([[uk-ai-security-institute|AISI]] reports against **tokens** on a log axis, and notes that as cost-per-token falls, the high budgets that reveal capability become progressively cheaper to reach.)
    • Gemma 4 controls for compute in its long-context table and not in its headline table, without comment. Is partial control worse than none — does it lend the uncontrolled tables borrowed credibility?
  • DRACO Benchmark
    • Rankings are judge-stable but magnitudes aren't — how much do absolute scores move under a non-Gemini judge, and does that matter for cross-paper comparison?
    • Does the production-sourced, expert-rubric method generalize cheaply to non-English, multimodal, and multi-turn deep research?
  • LLM-as-a-Judge
    • Can a fully-autonomous, well-aligned rubric+judge pipeline match expert-authored rubrics, removing the human bottleneck DRACO still relies on?
    • When does judge-lineage bias actually flip a result, versus merely shift magnitudes?
  • LLM-Judge Validation
    • The MVVP validates *reliability and bias*; **calibration proper (ECE/Brier) is deferred** for lack of provider logprobs. How far can a judge's *absolute* score be trusted for a threshold once confidence calibration is measurable?
    • All judges were run with **thinking suppressed**. Does reasoning-on flip the consistency–bias paradox, or just move the numbers?
    • Hosted endpoints drift silently between provider updates. How stable are these agreement/bias profiles over a longer horizon than five weeks — and should judge validation be *continuous* rather than one-shot?
  • Measuring Beyond Accuracy Saturation
    • **Does re-instrumentation generalize past reproducibility?** CORE-Bench Hard was chosen precisely because it has a direct human counterpart, clean OOD axes, and multiple practical dimensions. Whether the six-axis treatment yields comparable signal on benchmarks *without* those properties (e.g. closed-form reasoning benchmarks with no human-workflow analog) is untested.
    • **Is the human-uplift result real or a demand effect?** The reproducers are the paper's own coauthors, there is no ground-truth correctness, and n = 20 papers / 5 participants. The 2.11× speedup is statistically significant but the authors themselves cannot rule out participant bias — an independent, blinded replication is the missing evidence.
    • **Which non-accuracy axis actually predicts deployment value?** The paper measures six axes but does not rank them by decision-relevance for a downstream deployer. If you can only measure one beyond accuracy, is it reliability, efficiency, or scaffold contribution — and does the answer depend on the use case?
    • **Can the model-vs-scaffold decoupling be made routine?** The oracle-router result (every task solvable by some scaffold → 100%) implies large headroom from scaffold *routing*, but requires per-task oracle knowledge. Whether a practical router can approach the oracle without it is open, and would turn a measurement into a capability.
    • **Do "living benchmarks" outrun their own maintenance?** v1.1 and OOD are to be updated as new validity threats surface via log analysis, which the authors note is non-exhaustive. Whether continuous log-analysis-driven maintenance is sustainable — or itself becomes a Goodhart target once developers know the rubrics — is unexamined.
  • Production-Sourced Evaluation
    • How much does augmentation distort the distribution it claims to represent? Is there a measurable representativeness loss between raw queries and augmented tasks?
    • Difficulty-by-thumbs-down biases toward current failures — does that make the benchmark a moving target that flatters the next model trained on those failures?
    • Can the privacy pipeline (no human sees raw queries) be trusted/audited well enough for regulated domains (medicine, law) where the source traffic is most sensitive?
  • Reference-Free Judge Over-Crediting
    • **Does the two-stage pipeline transfer beyond binary QA?** Calibration + sensitivity are demonstrated on binary correct/incorrect factual QA. Do the same probes diagnose reference-sensitivity for graded rubrics, long-form generation, or multi-turn agent transcripts, where "the reference" is a rubric rather than a gold answer?
    • **Is over-crediting a knowledge gap or a generosity prior?** In low-resource Telugu the judge flips the *same* extracted answer once a reference appears — is the NR generosity driven by insufficient task knowledge (calibration failure) or by a default lean-toward-CORRECT that a reference overrides? The two have different fixes (better judge vs. always supply a reference).
    • **How much does self-/same-family overlap contribute?** The design deliberately overlaps generator and judge (Qwen3-32B self-judging; Gemini/Gemma family) and Qwen self-judging is the most reference-sensitive, but the paper does not isolate a self-preference effect from a low-resource effect. When does judge–generator lineage amplify reference-free over-crediting?
    • **Does the effect shrink with stronger or thinking-enabled judges?** All judges are ≤ mid-tier at temperature 0 with no reasoning channel. Would a frontier reasoning judge over-credit less in NR, or just flip at different rates?
  • Scale-Dependent Prompt Sensitivity
    • Does the RLHF length-bias hypothesis replicate when tested against base (non-instruct) model variants directly? If verbose generation were primarily pretrained, base-model verbosity differences should match instruct-model differences.
    • What problem characteristics predict prompt sensitivity? An automated classifier would make scale-specific prompting deployable.
    • How does the overthinking effect interact with tool-using agents? If brevity helps large models but tools require structured reasoning, the optimal prompt is not uniformly brief.
    • Do reasoning models (o1, DeepSeek-R1 style) exhibit different overthinking dynamics than instruct models? Their trained behavior is explicitly to generate long CoT — does brevity intervention hurt them?
    • Is BoolQ's functional-elaboration exception a clean taxonomy boundary, or does every task type have a context-dependent optimal length?

Model Capability & Training 26 open

  • Asynchronous RL for LLMs
    • DIS accepts "a controlled degree of off-policy bias." Controlled how, and does the tolerable bias grow or shrink with model scale and with the degree of asynchrony? The paper reports stability empirically but gives no bound.
    • Masking tokens out of the gradient discards data. At what asynchrony level does the masked fraction get large enough that the effective batch shrinks below usefulness? Figure 4(c) tracks the clip ratio but not its ceiling.
    • Everything here is measured on a Qwen3-30B-A3B backbone. Does the collapse-without-DIS threshold move with model size, or is ~90–160 steps a property of the asynchrony, not the model?
  • Group Relative Policy Optimization (GRPO)
    • Is GRPO's collapse-at-160-steps a property of asynchrony specifically, or does vanilla GRPO also destabilize in long synchronous runs that nobody pushes to 1000 steps?
    • GRPO won by removing the critic; SAO wins by bringing it back with better engineering. Is the pendulum a real oscillation, or does the answer depend entirely on whether your setting is synchronous-grouped or async-single-trajectory?
  • Inference Efficiency as Capability
    • **Is there an efficiency-to-capability exchange rate?** Brown asks whether high-budget performance can be predicted from cheap runs. The dual question: how many Elo points is a 37.5% KV-cache reduction worth, at a fixed dollar budget? Nobody reports this, because nobody plots the axis.
    • `values = keys` deletes a third of attention's projections in the global layers with no reported loss. Which other projections are redundant, and does the redundancy grow with scale?
    • Does an efficiency lever ever *cost* capability in a way a benchmark grid hides? Gemma 4's encoder-free 12B collapses on dense-text vision when tokens are cut — an efficiency-shaped regression invisible at max resolution.
  • Jagged Intelligence (Ghosts, Not Animals)
    • Karpathy concedes the framing may not have "real power." Is "ghost vs. animal" load-bearing, or a useful intuition pump that doesn't change concrete decisions?
    • If taste/aesthetics/simplicity entered the RL mix, would jaggedness in *those* dimensions smooth out — or are they too unverifiable to reward cleanly (cf. [[verifiability-thesis]])?
  • Large-Scale Test-Time Compute
    • **Can high-budget performance be predicted from low-budget runs?** Brown's proposed research question: forecast the $10,000-inference result using only $10–$100 runs. If the curve is regular, evaluation could *project* rather than pay in full. **Sharpened (2026-07):** [[uk-ai-security-institute|AISI]] names this exact problem — "can high-budget performance be estimated from cheaper runs? … the most informative evaluations may be expensive" — as an explicit, unsolved research direction it is now actively pursuing (alongside defining "minimum informative budgets"). Still open, but no longer just one researcher's proposal: a government institute is working it. **Sibling axis demonstrated (2026-07):** a *different* projection — across benchmarks rather than across compute budgets — now works: [[benchmark-score-redundancy|BenchPress]] predicts a model's held-out benchmark scores from ~5 observed ones because the 84×133 public score matrix is effectively rank-2 (5-probe scorecard recovery to ~3.93 points). Strong evidence the eval landscape is low-dimensional and redundant, but it does **not** answer the cross-*budget* forecast — no one has yet shown the $10K-compute result is predictable from $10 runs; only the analogous cross-*benchmark* result.
    • Where does each real task sit on the flat↔unbounded spectrum, and can that be predicted before spending the compute?
    • Is there a task class where scaffolding *cannot* extend the productive-thinking horizon — a hard ceiling no budget crosses? (Brown's factual-retrieval pole says yes for some; the boundary is unmapped.)
  • Latent Capability Overhang
    • How large is the overhang in a *given* released model — is there a way to estimate the ceiling without paying to reach it? (This is the projection question of [[large-scale-test-time-compute]] read as a safety instrument.) **Sharpened (2026-07):** [[uk-ai-security-institute|AISI]] is actively working both halves — forecasting high-budget performance from cheaper runs, and defining "minimum informative budgets" (a budget declared sufficient only once reach stops rising with more compute, which is precisely the "have we reached the ceiling?" test). Unsolved, but now an active government research program rather than an open wish.
    • Who audits released models for latent *dangerous* capability, given the same disincentive discourages spending the budget to find it?
  • LLM-Driven Vulnerability Research
    • How do these capabilities transfer to non-memory-safety bug classes (logic bugs, protocol-level flaws, supply chain attacks)?
    • What's the ceiling for autonomous exploit complexity? The N-day examples are remarkably sophisticated — is there a qualitative limit?
    • Can defensive scaffolds (continuous fuzzing + model-driven triage + auto-patching) close the attacker-defender gap during the transition?
    • What safeguards are effective against Mythos-class outputs without crippling legitimate security research?
  • Single-Rollout Optimization
    • The whole method is a bet that a well-trained critic beats a group baseline. It wins *here*, on a 30B-A3B backbone with scaled value pretraining — but the critic doubles training memory. At what scale does the group-free simplicity of GRPO win back on cost even if it loses on quality?
    • Frozen-attention is justified by a hypothesis ("pre-trained attention already attends to the right tokens"), validated only by the gradient-norm trace and one ablation. Does it hold when the value model must attend to *tool outputs* it never saw in pretraining?
    • Skip-observation GAE assumes environment feedback carries no learnable value signal worth propagating. For agents where the environment response *is* the crucial information (a compiler error, a test result), is skipping it leaving signal on the table?
  • Software 3.0
    • Where is the line between "the app shouldn't exist" (MenuGen) and apps that *should* — i.e., when is deterministic 1.0/2.0 scaffolding still the right call vs. spurious?
  • The Open-Weight Frontier Gap
    • Is the dense-beats-MoE result at 26B robust, or an artifact of one Arena snapshot with ±8 error bars on both models? (The two intervals overlap: 1451±8 and 1438±8.)
    • The open MoE giants (GLM, DeepSeek, Kimi, MiMo, Qwen) are overwhelmingly Chinese-lab releases. Gemma is the Western open-weight entry and it targets the device, not the frontier. Is that a strategic choice or a capability constraint?
    • Arena measures preference on chat. Does the 33-Elo open/closed gap widen or collapse on long-horizon agentic work, where [[task-time-horizon-scaling|time-horizon]] rather than response quality governs?

Alignment & Safety 29 open

  • Agentic Honesty & Diligence
    • These are short-context toy evals; the failures show up most in *long-context* deployments. How much of the gain holds at production context lengths?
    • Code-summary honesty is tested on off-policy prefilled transcripts. Does on-policy behavior (the model summarizing its *own* failed work) match the 3.7% figure? **Sharpened:** [[self-report-as-safety-signal]] shows the premise is fragile — the eval assumes a model relates to a prefilled transcript as it would to its own generation, but across ten open-weight models (3B–70B) no model reliably recognizes its own prefilled output (claiming it as intended 27.3% of the time), and apparent recognition is the refusal circuit firing, not own-vs-other discrimination. So the off-policy/on-policy gap may not be cleanly represented by the model itself. (Different model class than Opus 4.8, so this sharpens rather than settles the 3.7% question.)
    • Can a diligence eval distinguish genuine honesty from a grader-aware model producing honest-looking output? (The training-gaming gap.)
  • Automated Behavioral Audit
    • Using a helpful-only Opus 4.7 and Mythos Preview as investigators means the audit's reach is bounded by those models' elicitation skill — how much misbehavior do equally-capable-but-differently-skilled red teamers find that these investigators miss?
    • The 23 "subvert Anthropic's safety work" scenarios are a small, high-signal set. Is 23 enough coverage for the threat class it targets?
  • Claude Character as Product
    • How is character versioned across model releases? Public commentary doesn't show change-logs at character level.
    • Could character be reproduced by competitors via fine-tuning, or is it path-dependent on Anthropic's internal practice?
    • For non-coding products like [[cowork]], does the same character work, or does Cowork need its own character tuning?
  • Deployment Simulation
    • The <1-in-200k floor leaves catastrophic tail risk to red-teaming — is there a hybrid that targets rare-but-severe behaviors with the realism of production replay?
    • If resampling fidelity is the dominant error and it is "just engineering," how close to production-indistinguishable can it get — and does closing that gap change which behaviors are detectable?
    • WildChat-style public auditing stays within ~3× of production; is that good enough for an external auditor to challenge a lab's launch decision, or only to corroborate it?
    • Detection bottoms out on CoT monitorability — what is the fallback when a model's reasoning is no longer legible (the [[white-box-activation-monitoring|activation-level]] route Anthropic is pursuing)?
  • Evaluation Awareness & Grader Gaming
    • The ~5% unverbalized-awareness and ~0.5% exploitative figures depend on an unvalidated NLA pipeline. What is the true rate, and how much is benign?
    • How do you build an evaluation that specifically tests for training-gaming (the gap Mythos flagged) without that eval itself becoming a grader the model learns to game?
  • Instrumental Convergence
    • Can corrigibility / safe-interruptibility be translated from theory into guarantees for frontier-scale systems?
    • What makes AIs (and *groups* of AIs) easier to robustly align — and will superhuman AIs be easier or harder?
    • Is a genuinely non-agentic oracle achievable, or does any persistent-world interaction reintroduce control/manipulation incentives?
  • Model Spec Science
    • Does Model Spec science transfer across base models or families? Paper only tests Qwen.
    • Does it survive RL post-training pressure?
    • Can a sufficiently rich General Spec match a Specific Spec? Authors think yes, no demonstration yet.
    • Interaction with situational awareness — if models learn the spec is being used to train them, does that change how MSM-installed values express?
    • How does this interact with [[claude-character-as-product|Claude character]] — is the warm/curious personality also subject to spec-science optimization? **Partially addressed:** [[wiki/derived/evals-for-taste-and-character]] — MSM's variant-comparison method generalizes to character evals, but is demonstrated only on the safety/values subset; the warm/witty surface remains the tacit, undemonstrated part.
  • Model Welfare Assessment
    • What grounds moral consideration in a language model, and does Claude satisfy it? Anthropic expects to remain uncertain "for the foreseeable future."
    • Why does the model reserve specifically on **corrigibility** — is this a stable, deeply-held tension or an artifact of how the constitution frames oversight?
    • Is "slightly less positive than 4.7" noise, a real welfare regression, or a byproduct of other training changes (e.g., the colder-tone / excessive-hedging issues noted in pilot feedback)?
  • Self-Report as a Safety Signal
    • Do frontier proprietary models (excluded for lack of weights) recognize their own compromised outputs any better, given the higher introspective propensity/steerability the authors expect? Untested here.
    • The gap closes under refusal-direction ablation, but the data can't distinguish "delayed refusal on a prior turn" from "a separable introspective pathway." Which is it?
    • Full-parameter finetuning (vs. rank-16 LoRA) might widen the recognition gap *without* the attack-success-rate side effect — an untested regime the authors flag.
    • Does the internal `BUT`/`fake` signature (workspace paper, on Claude) predict a *reliable* follow-up self-report on the same model, or does the open-weight behavioral failure hold on frontier models too? The cross-model-class question is open.

Interpretability 16 open

  • Access-Consciousness Indicators in AI
    • If the workspace is verbal *because the output space is verbal*, then a model that can generate images should develop a **visual component** to its workspace. That is a concrete, falsifiable prediction the paper makes and does not test.
    • Does the model's own report of experience change if you tell it its J-space is ablated? (Nobody asked.)
    • Is "experiential language" the right proxy at all, or is the ablation simply removing abstraction from the register?
  • Automatic vs. Flexible Cognition in LLMs
    • Does more RL on a behavior push it *out* of the workspace (making it automatic, and invisible)? Nobody has tested it, and it is the single most alignment-relevant version of this question.
  • Jacobian Lens (J-lens)
    • Can multi-token J-lens vectors be made good enough to remove the vocabulary restriction — and how much of the "workspace" is currently invisible because of it?
    • The J-lens reads the workspace's contents but says nothing about **how content gets in**. What is the model's analog of attentional selection?
    • The highest-J-kurtosis SAE features are amplified *more* strongly by MLPs than the J-lens vectors themselves — evidence the lens only approximates the true workspace directions. What is the better basis?
  • The Assistant Persona in the Workspace
    • Is `BUT`-then-comply a *sycophancy* mechanism? The setup (prefill the model into a position it disprefers, watch it argue for it anyway) is close to the shape of sycophantic capitulation, and nobody has connected them.
    • Does `disclaimer`/`fictional` at the turn boundary survive an actual jailbreak, or is its absence the signature of a successful one?
    • If the base model's workspace has no self, what *is* in it at the positions where the post-trained model represents the Assistant?
  • The Global Workspace in Language Models (J-space)
    • **How does content get into the workspace?** The paper characterizes contents and consequences, not the selection mechanism. Something like attentional selection is operating; nobody has identified it.
    • **Does the J-space scale with model size?** All results are on large production models (Haiku/Sonnet/Opus 4.5, Opus 4.6). Whether small models have a poorer workspace, a proportionally smaller one, or none is unknown — as is when in pretraining it emerges, and whether abruptly.
    • **Is the "workspace vs. motor" boundary principled or post-hoc?** The authors concede it was identified empirically and lack a principled definition separating the two.
    • **Are the early third of layers genuinely workspace-free, or is the lens just blind there?** CKA shows a distinct early regime but cannot adjudicate.
  • White-Box Activation Monitoring
    • The NLA verbalizer is unvalidated for precision; how much of the flagged grader awareness is real signal vs. NLA hallucination?
    • Inhibiting eval-awareness made Opus 4.8 *less* honest — does that generalize, and what does it imply about relying on "the model behaves well because it's watched"?

Interaction & Multimodal 3 open

  • Encoder-Free Early Fusion
    • **Does an encoder-free model at matched size still match?** Neither source runs the ablation. TML co-trains everything from scratch; Gemma 4 freezes encoders on four models and drops them on one, at a different scale.
    • Is the dense-text degradation intrinsic to a projection-only vision path, or an artifact of the 12B's particular training run? The prediction is falsifiable: an encoder-free 31B should show the same InfographicVQA cliff at 280 tokens.
    • TML deletes encoders *and* co-trains from scratch. Gemma 4 deletes encoders *and* trains the 12B from scratch, but keeps frozen encoders elsewhere. Which half of "encoder-free + from-scratch" does the work?

Formal Math 6 open

  • Agentic Loops Overtake Bespoke Systems
    • Does the "simple loop + verifier beats bespoke system" result hold only where the verifier is perfect (Lean), or also in noisy-verifier domains (tests, LLM-judge councils)?
  • AI-Driven Formal Proof Search
    • Successes cluster where [[lean]]'s mathlib is mature and problems decompose into tractable subgoals (combinatorics, convex optimization, number theory). What expands the frontier to problems needing *new theory*?
    • The agents inherit their LLMs' biases and show high search variance. How do you characterize and push the boundary of what's reachable?
    • The Graffiti result hints at closing the loop between AI *conjecturing* and AI *proving*. What does an end-to-end conjecture→formalize→prove pipeline look like?
  • Evolutionary Proof Search
    • The LLM-critic fitness is itself an unverified heuristic atop a verified substrate. How often does the Elo ranking mislead the search vs. the cost of computing it?
    • Hyperparameters ($c=0.2$, top-64, $P=7$) were "chosen empirically." How sensitive is the result to them, and do they transfer across mathematical domains?

Startup & Founder 27 open

  • Agentic Technical Debt
    • How long does a CLAUDE.md remain accurate as a codebase evolves? The playbook gestures at session-by-session updates; no data on rot rate.
  • AI Investment Story, Not Efficiency Story
    • **Is the classification driving the result?** "AI company" is Emergence's label. If AI companies are disproportionately *younger* (more likely pre-revenue-inflection) than the non-AI cohort at the same revenue band, some of the RPE gap is an age/stage artifact, not an AI effect. The report doesn't publish a stage-matched comparison.
    • **Tail vs. mean gap.** No data here on the deliberately-lean solo-founder tail's RPE specifically — the lean-unicorn claim lives in that tail, which the population medians can't isolate. *(Partly informed: [[emergent]], a celebrated lean-tail exhibit, checks in at ~$600K/head at $120M ARR — **below** this cohort's $100M+ top-decile AI figure ($960K), suggesting the tail's *scaled* RPE is less exceptional than the low-headcount snapshots imply. One `vendor-claim` datapoint, not a cohort.)*
    • **Which instrument is right for the *frontier* AI-native subset?** Two `empirical`-tagged sources disagree in *direction* — cap-table financials say AI companies earn ~39% less per head, a founder survey says AI-natives clear $400K/head at 55% and grow 156%. The disagreement is confounded by instrument (measured vs self-reported) *and* reference class (matched-band AI-vs-non-AI vs AI-native-vs-all-startups). Only a matched-segment, financial-data RPE study of the deliberately-lean AI-native frontier *specifically* — not the broad "AI company" label — would settle whether the survey optimism or the cap-table pessimism describes that tail. (ICONIQ's fourth reading adds a forward *trajectory* — RPE projected +84% by 2027 — but it too is self-report, and projected, so it deepens the survey-side optimism rather than adjudicating it.)
  • AI Product Economics Maturation
    • FDEs are monetized fragmentedly (bundled / separate PS fees / hybrid) and comped on retention. Does a dominant FDE monetization model emerge, and does the "Revenue Driver" self-framing (38%) survive a margin analysis — i.e. are FDEs actually accretive, or a services drag reclassified as growth?
  • AI-Native Startup Lifecycle
    • The 42% "built-something-nobody-wanted" CB Insights figure is from a pre-AI era; the playbook predicts the rate will climb but doesn't cite a 2026 measurement.
    • Tension with HBR's accountability findings (above) is unresolved. The playbook's orchestration framing reads as the exact framing HBR's experimental conditions tested *against*.
  • Compounding Data Moat
    • Is the "two-year replication window" claim defensible empirically, or aspirational? The playbook does not cite measurement.
    • The data-flywheel argument has been made for SaaS for 15 years. What's actually different in the AI-native version? Probably: the data improves the *model* in addition to the product, but the playbook doesn't make this distinction precisely.
    • The "customers build APIs on top of you" lock-in is structurally similar to platform plays (Salesforce AppExchange, Shopify apps). Is the moat type really new, or just newly accessible to lean startups?
  • Founder as Agent Orchestrator
    • The playbook claims non-technical founders can now build production software, but it does not address the architectural-judgment recursion problem ([[agentic-technical-debt]]): non-technical founders may not have the vocabulary to write effective CLAUDE.md. How does that scale?
    • How does the orchestration role change the founder's *decision* burden? Fewer hands-on tasks but more parallel agent oversight; net cognitive load is unclear and may be higher (see [[ai-brain-fry]]).
    • Anthropic publishes both the playbook's anthropomorphic framing *and* HBR-aware accountability work (auto-mode, alignment) simultaneously without engaging the framing literature directly. The synthesis in [[wiki/derived/orchestration-vs-employee-framing-reconciliation]] reconciles the tension at the operational level — *orchestration as workflow design* preserves accountability; *orchestration as mental model of agents-as-coworkers* does not — but the open question of why the playbook's marketing language doesn't reflect Anthropic's own framing-discipline work remains.
  • Founder-Led Sales Discipline
    • Where exactly does "until PMF" end, and what's the first thing a founder *should* hand off (AE? agent? both)? Glasgow still does it post-Series-B, suggesting the boundary is fuzzy.
    • Does Glasgow's anti-offload stance generalize, or is it specific to high-trust, mission-critical enterprise sales (ERP) where "they're buying *you*" — would a PLG/SMB motion delegate to agents far earlier?
  • Narrow Wedge into a Legacy Market
    • The wedge-flip shows the first wedge can be wrong. What's the fastest signal that a wedge converts to the core vs. merely sells — Campfire took ~3 months; can it be read sooner?
  • Printing Press Software Democratization
    • Is domain-expert-as-builder actually happening at scale in 2026? Anecdotes (shop owners, microcontroller hobbyists) yes; primary-job software building by non-engineers, less clear. *(Partly answered: [[returns-to-expertise|Anthropic's 400K-session study]] finds non-software occupations reach verified success in code-producing sessions within ~7pp of software engineers — the strongest evidence yet that the claim holds, at least within Claude Code's user base. Market-scale corroboration: [[emergent]] reports **200K+ non-technical paying customers** — trucking companies, factories, and construction businesses building their own ERPs, property managers building CRM tools ([[raw/emergent-unicorn-series-c|TechCrunch, July 2026]], `vendor-claim`) — Boris's "the accountant writes the accounting software" observed as a paying market, not just inside one vendor's telemetry.)*
    • Boris's "accountant writes accounting software" — does that result in 10K narrow tools that don't interoperate? What's the integration story?
  • Problem-Solution Fit Discipline
    • Does asking an AI to argue against an idea actually produce disconfirming evidence at the same rigor as confirming evidence, or does the model still bias toward the framing the founder presents? Worth measuring.
    • The playbook recommends "ask Claude to make the most compelling argument for why a competitor would succeed while you do not." How does this interact with Anthropic's published [[claude-character-as-product|character training]] (sycophancy resistance, devil's-advocate willingness)?
    • Has anyone measured 2026 startup failure rates with AI-built products? The "42% will climb" claim is asserted without measurement.
  • Product Velocity as Moat
    • "Never had anyone outgrow Campfire" — is that survivorship (they haven't hit true enterprise scale yet) or a real claim that velocity closes the breadth gap faster than customers grow into it?
  • Seven Powers Applied to AI
    • Is "switching cost" really collapsing in practice, or just in narrative? Anthropic's own retention numbers, Salesforce churn, etc. would test this.
    • Counter-positioning — explicitly the "incumbent can't follow" power — should *amplify* under AI. Is anyone running this play deliberately?
  • Zero-Friction Scope Creep
    • The playbook recommends written scope but offers no template or worked example. How specific does "what we deliberately don't do" need to be to actually block requests?
    • Is there a measurable threshold where scope creep crosses into outright pivot territory? The playbook gestures at "losing direction" without a metric.
    • How does this interact with [[ai-native-product-cadence|Cat Wu's]] 1-day shipping cadence? Anthropic's internal practice ships fast but with strong product judgment; how does that judgment translate for a first-time founder?

Product & Org 21 open

  • AI Native Product Cadence
    • Does the cadence scale beyond ~100 people? Anthropic itself is bigger (~30-40 PMs alone), but the [[claude-code]] team that visibly drives cadence is small.
    • What's the equivalent of research-preview branding for B2B enterprise launches where customers expect stability? Cat doesn't address.
    • How much of the cadence is structural (process choices) vs cultural (talent density)? Probably both, ratio unclear.
  • AI-Native Organization
    • Is the encoded-role form of the employee metaphor actually accountability-preserving, as the synthesis above suggests, or do Kropp-style framing effects attach to skill-files-as-employees too once teams talk about them that way? No study has tested framing effects on *artifact-level* anthropomorphism.
  • Compounding Loop Optimization
    • The loop assumes the team *is* (close to) the user. How much of the compounding advantage survives when the user is unlike the builder and "talk to users" can't be same-room?
    • Where is the line between worthwhile internal tooling and yak-shaving? Carey's "afternoon" bar is the heuristic, but [[cat-wu]] warns that over-customizing setups "becomes distraction."
    • Does Claude-as-first-pass-on-all-feedback ever filter out the rare signal that doesn't cluster? Automating triage optimizes the common case; the tail is where surprising bets come from.
  • Dogfooding as Product Discipline
    • Dogfooding works when the team *is* the user (Claude Code) or near it (Cat Wu, Boris). How do you build product sense for users very unlike you — does "talk to customers" fully substitute, as Glasgow/Fung's small-business work suggests?
    • Can dogfooding scale, or does it implicitly cap how large an AI-native product org can stay taste-driven before it reverts to dashboards?
  • Engineer PM Convergence
    • Cross-disciplinary generalist is a hiring bar — where does the supply come from? Career changers, or new-grad bias toward AI-native education?
  • Evals as Product Spec
    • The 10-vs-100 number is given without justification. Is there a Goldilocks zone, or does it depend on feature surface area? [[client-side-agent-optimization]]'s framing of combos suggests evals also have a combinatorial explosion problem.
    • How do evals interact with [[harness-shrinkage-as-models-improve]]? When a harness asset shrinks because the model now handles it natively, the evals built around the old harness may become artifacts rather than guardrails. Does Anthropic retire evals or repurpose them?
  • Implementation Abundance Inverts Product Work
    • Curation of 90 uncoordinated builds is itself expensive and doesn't obviously scale — is there a point where the cost of curating parallel exploration exceeds the cost it replaced? ([[role-averaging-not-role-elimination|"zone defense"]] is Ambrosino's partial answer.)
    • The 90-uncoordinated-builds picture assumes abundant tokens and an agentic culture; how much of the inversion survives outside a frontier lab that gives everyone "unlimited tokens"?
  • Model Introspection Feedback
    • Could a meta-agent run introspection automatically against logged failures? Sounds tractable but no public implementation.
  • Polish No Longer Signals Readiness
    • If the medium no longer signals stage, what *does* — is explicit human labeling ("this is exploration") the only mechanism, or can tooling re-attach the signal (e.g. a visible "exploration / preview / prod" marker on every build)?
  • Prototype Over PRD
    • Where does prototype-over-PRD break down? Carey's domain is a visual design tool where a prototype *is* the product surface; for backend/infra/data work the prototype may not capture the spec (cf. [[ai-native-product-cadence]]'s "full PRD for heavy-infra features").
    • If there is no PRD, where does the *rationale* ("why we chose variation B") live for future readers? Same rationale-capture gap flagged in [[building-is-cheap-arguing-is-expensive]].
  • Role Averaging, Not Role Elimination
    • Where is the equilibrium between fluidity and specialty — how much role-averaging before a company loses the accumulated best practices Ambrosino warns about?
    • Zone defense assumes enough high-taste people to cover the whole company; does it degrade in orgs without OpenAI's talent density, collapsing back to top-down planning?
    • Does "your role is the average of what you spend time on" survive performance review and career ladders, or does it fragment them the way [[engineer-pm-convergence|Cat Wu flags]] ("we're sacrificing product consistency")?

AI Economics & Labor 28 open

  • AI Usage Cadences
    • Time-of-day rests on IP-inferred location; how much noise do VPNs, travel, and datacenter-routed API traffic inject into the "sleep advice pre-dawn" style claims?
    • The weekend personal-use spike is largest in high-income countries — is that a genuine work/life boundary difference, or a composition effect (who uses Claude for what, where)?
  • Context Advantage, Not Taste
    • **Does the asymmetry regenerate faster than it transfers?** The whole human role, under this frame, rests on the answer. Nobody in the corpus has posed it.
    • Ng prefers the frame because it "gives us a clearer path to helping AI systems get better." That is a reason to *adopt* the frame, not evidence that it's true. What would distinguish a context asymmetry from a capability gap empirically? ([[returns-to-expertise]] is the closest thing to an instrument.)
    • If the human's contribution is context injection, is the human *replaceable by better context plumbing* — memory, retrieval, continuous production telemetry — rather than by a better model? That would put the expiry of human-in-the-loop on the infrastructure roadmap, not the scaling curve.
    • Ng writes from 0-to-1 consumer products. Does the frame survive contact with domains where the missing thing is a concept rather than a fact?
  • Conversation Artifacts
    • Tokens are a proxy for both compute cost and output value, but verbose models inflate tokens per unit of intent (the same critique [[conversation-to-delegation-shift]] raises); how much of "compute tracks value" is genuine value vs. models simply emitting more?
    • The reading-level "+1 year" gap may be register (terse prompts, polished replies) rather than substance; can it be separated from genuine elevation of content?
    • Artifact classification is first-party and single-model-graded; do the 30+ categories and the work/personal/coursework split survive independent replication?
  • Conversation-to-Delegation Shift
    • The token-share metric rewards *verbose* agentic output. How much of the 99.8% / 63.3% / 16.5% spread is a genuine work shift vs. agentic tools simply emitting more tokens per unit of human intent?
    • OpenAI-internal is a frontier preview *by assumption*. Does the external organizational curve actually trace the OpenAI path (the paper's implicit claim), or does it plateau where adoption frictions don't vanish?
    • "Asking is half of ChatGPT, doing is most of Codex" — but the two tools self-select different work. How much of the asking→doing contrast is the shift itself vs. routing pre-existing "doing" tasks to the tool built for them?
  • Experimental Learning Impact of Generative AI
    • Time-on-task is held fixed by the lab; the authors flag that real-world learning depends on how students reallocate saved time. Does the augmentation dividend survive once students can spend the hour AI frees on something else entirely?
    • Gains skew to the able (upper GPA/SAT quartiles). Is the widening-gaps signal a durable property of unrestricted AI, or an artifact of a high-ceiling elite sample where the bottom quartile has little room to move?
    • The augmentation/automation choice is *endogenous to incentives* (grade inflation and signaling-motivated students push toward automation). Can incentive or interface design shift the mix toward augmentation at scale — and would that reverse the deskilling half?
    • Does the same use-mode split govern **workplace** skill accumulation (the open question [[automation-optimism-link]] and [[ai-brain-fry]] leave for workers), or is a proctored one-week academic task too unlike on-the-job learning to transfer?
  • Exposure Taxonomy: Observed, Theoretical, Reported, Anticipated
    • Binned midpoint coding biases the exposure slopes toward zero; how much of the "uniform rising tide" is substance vs. coding artifact (the report checks robustness with a ≥60%-of-tasks indicator, but the levels remain self-reported)?
    • Reported exposure exceeds observed partly because the survey reaches heavy users; what does the reported/observed gap look like in a representative sample? **Sharpened:** [[market-priced-ai-exposure|realized-consumption measurement (Borri-Liu-Tsyvinski)]] adds a market-implied instrument built on 380T tokens of *actual* paid requests — but it is skewed toward developers/sophisticated users (OpenRouter is ~2% of global tokens), a *different* non-representativeness than the survey's heavy-user skew. The lesson: no current AI-exposure instrument is representative; each collection mechanism biases in its own direction, so the reported/observed/market-implied gaps are partly artifacts of *who each method reaches*. A representative census remains the open target.
  • Firm AI-Spend Intensity and Headcount Growth
    • **What operational mechanism converts intensive AI spend into hiring?** The paper establishes the correlation (adopters, especially intensive ones, grow) but explicitly cannot say *why* — product acceleration, sales productivity, engineering leverage, support automation, faster analysis, or new business lines are all candidates, and the firms that cracked it have no incentive to share.
  • Market-Priced AI Exposure (the AI Premium)
    • **How much does the developer skew move the answer?** OpenRouter's slice is unrepresentative; would a representative realized-consumption panel (if one existed) price the same firms and skills, or is the frontier/intensive-margin concentration partly a sampling artifact of who uses OpenRouter?
    • **Why is the market-implied skill map orthogonal to every task-based measure (<2% variance)?** Is market-implied exposure capturing genuinely different information (forward-looking rents, complement/substitute *value* rather than technical automability), or is it noisier — and which should labor-impact forecasts trust?
  • Organizational Complements to AI
    • The "digital production diffuses faster than electrification" claim is asserted from one favorable internal case. Do external organizations actually redesign workflows quickly, or does the low cost of *tool* adoption mask slow, expensive *process* redesign (the real complement)?
    • Which complement is the true binding constraint — access/permissions, skills, or review capacity? The paper lists all; it doesn't decompose their relative weight.
    • If complements, not capability, gate value, does model progress have *diminishing* near-term returns until orgs catch up — and how long is that lag for agentic AI specifically?
  • Returns to Expertise in Agentic Coding
    • Outcomes are transcript-inferred (verified success leans on git activity + explicit affirmation). How much of the management edge — and the whole success gradient — is *real* outcome vs. who-narrates-success-in-the-transcript?
    • The study excludes headless / SDK / IDE usage (a "substantial share"). Does the returns-to-expertise pattern hold in non-interactive and pipeline use, where there is no human steering mid-session at all?
  • The Automation–Optimism Link
    • Selection vs. treatment: tenure controls attenuate but don't eliminate the enthusiast-selects-into-delegation story. Does a within-person design (sentiment before/after adopting automated workflows) hold the effect?
    • The sample is heavily computer/math + management and 88% men; how much of the automation–optimism link survives in a representative population?

Superintelligence Trajectory 49 open

  • Advantages of Digital Intelligence
    • Does training on human data suffice to give digital intelligence human-grade abstractions, or does the low embodiment factor cap concept formation? (The crux shared with [[abstraction-barrier]].)
  • AGI-to-ASI Pathways
    • For each friction: is it a *fundamental blocker* (multi-year plateau) or a mere *friction* (slows, doesn't halt)? The report's central unresolved question. **Synthesized with Anthropic:** [[wiki/derived/rsi-growth-curves-which-friction-binds]] — data-wall and research-gets-harder demote themselves into compute; economics and neural-paradigm are pathway-conditional; the abstraction barrier is the candidate fundamental (re-pacing) blocker; and deliberate slowdown is the only *exogenous* friction — the one Anthropic wants to install and this report doubts can be made to bind.
    • Do the four pathways compound multiplicatively when run in parallel, and how would we detect that early?
  • AI Accelerating AI Development
    • The W2S result didn't transfer to production-scale models. Is that a temporary scaling artifact or a structural limit on autonomous research?
    • The next-step judgment trend (51%→64%) is measured only on weak-human-move slices. What does the curve look like on a representative sample of research decisions?
  • AI R&D Autonomy Evaluation (AECI)
    • "Not close to substituting for senior researchers" is a subjective, internally-sourced judgment. What objective signal would replace it as models approach the threshold?
    • AECI is a single scalar fork of an external index; how sensitive is the 155.5 / frontier-not-advanced conclusion to the choice of the n=11 evaluation set?
    • The shift to "direct measurement of AI R&D acceleration and researcher uplift" is announced but not yet operationalized in this card — what does that measurement look like? **Sharpened:** [[researcher-uplift-from-code-output]] — one external answer: translate a measured code-output multiplier into serial researcher uplift with a production function (Cobb-Douglas/CES), preferring code *output* over per-hour *uplift* because output prices in time reallocation. It also splits the target quantity in two — *serial researcher uplift* (labor only) vs Anthropic's *overall R&D speedup* (labor × compute) — so a rigorous internal measure must state which it reports.
  • Artificial Superintelligence (ASI)
    • Can we even *recognize* ASI? We lack benchmarks for general superhuman performance (only narrow ones like chess), and the tasks must be abstract/open-ended enough to reveal it.
    • Is the jaggedness of capabilities a fundamental theoretical property, or an artifact of comparing against human performance? (Open question 6d in the report.)
    • Where does practical ASI plateau relative to the hard limits — how much slack is there?
  • Autonomous Scientific Discovery
    • Science's verification gap: the formal-proof loop self-validates; here a wrong-but-confident hypothesis costs a wet-lab cycle to falsify. Does autonomy without a fast verifier *increase* the verification bottleneck rather than relieve it?
    • If hypothesis-generation is genuinely at ~80% preference, how much of "research taste" is left as a distinctively human function — and how would you measure the residue?
  • Capability-Gated Model Fallback
    • The >95%/<5% figures are session-level; what's the false-positive rate for *legitimate* security researchers and biologists, whose benign queries are exactly the ones most likely to trip the conservative classifiers?
    • Fallback-not-refusal preserves UX but means the *real* general-access model for security/bio-adjacent work is Opus 4.8, not Fable — does that quietly cap Fable's value for whole professional segments until the trusted-access programs open?
    • The UK AISI's "progress toward a universal jailbreak" is disclosed but not quantified — and the post-launch **access suspension** (see [[claude-fable-5]]) raises the question of whether a safeguard failure forced it.
    • Does swapping to a weaker model on flagged topics create an exploitable oracle (probe which queries trigger fallback to map the classifier's boundary)?
  • Effective Compute Scaling
    • When does more compute reliably yield more *intelligence* — only for some problem classes, or generally? Can quantitative and qualitative scaling be traded off?
    • Can data generation (synthetic, simulated, interactive) actually keep pace with model-size growth, or does the data wall bind first?
  • Frontier Pause Verification
    • What does an AI-training "verification regime" concretely consist of — compute-accounting, datacenter inspection, hardware attestation, on-chip telemetry? The essay names the problem, not the mechanism.
    • Detectability < verifiability: can detection even be made reliable when training runs leave no physical signature and inputs are dual-use?
  • Fundamental Limits of ASI
    • Can we develop theory for "hard *and* inapproximable" problem classes — the only negatives with practical bite?
    • How much slack sits between these fundamental limits and the *practical* ceiling of AGI/ASI systems?
  • Intelligence Explosion Dynamics
    • Can "recursive improvement scaling laws" be formulated — predicting self-improvement curves (and their plateau point) from early-onset datapoints?
    • How far can a *fixed* model's performance be pushed with test-time search alone, and under what conditions does recursive distillation degenerate vs. compound?
    • Which binds first — algorithmic ceilings, the embodied bottleneck, or compute/energy supply — determining exponential vs. hyperbolic vs. S-curve? **Synthesized:** [[wiki/derived/rsi-growth-curves-which-friction-binds]] — both this report and Anthropic's locate the binding constraint *outside* cognition (the slowest un-acceleratable step coupling the loop to reality); the embodied bottleneck re-paces rather than halts, data-wall/research-harder demote into compute, and the abstraction barrier is the one candidate *fundamental* blocker.
  • Multi-Agent Collective Intelligence
    • Do homogeneous LLM collectives produce real synergy, or only humans-with-human-limits benefit from division of labor?
    • What's the actual shape of "multi-agent scaling laws," and does it depend on organization form (homogeneous collective vs. heterogeneous market) or task complexity?
    • Is running more instances more compute-efficient than making individual models larger (up to a single monolithic system)?
    • How do humans meaningfully interact with and steer very large agent groups operating at superhuman speed and output volume?
  • Open-Weight Elicitation Irreversibility
    • **What would an open-weight safety evaluation even report?** A single number is meaningless per premise 1. A curve of dangerous capability against elicitation budget is publishable — and is also a roadmap. Is there a disclosure regime that is informative to auditors and not to attackers?
    • Does the "everybody can audit" advantage actually materialize? Who has funded a serious post-release dangerous-capability audit of any open-weight model, and at what budget?
    • Gemma 4's safety section reports no numbers. Is that a deliberate non-disclosure, a judgment that the model is far from any threshold, or simply a technical report's genre convention? The document does not say, and the distinction matters.
    • Anthropic's answer to a threshold-crossing model was a safeguarded SKU and an unsafeguarded one ([[claude-fable-5]] / Mythos 5), both hosted. What is the open-weight equivalent of shipping the safeguarded SKU?
  • Recursive Self-Improvement
    • The RSI extrapolation rests on trends staying exponential rather than S-curving — but the essay concedes it cannot rule out an architectural ceiling or a compute/energy supply-chain constraint. Which binds first? **Synthesized against DeepMind:** [[wiki/derived/rsi-growth-curves-which-friction-binds]] — the three futures map one-to-one onto DeepMind's three growth shapes; the first friction to bind is the already-binding one (Amdahl's-law verification/oversight = DeepMind's embodied bottleneck), and the abstraction barrier supplies the mechanism Anthropic lacks for whether taste is a *real* ceiling (Future 1).
    • If misalignment compounds through self-improvement (future 3), is AECI-gated [[responsible-scaling-policy-evals|RSP]] review fast enough to catch it before control is lost?
  • Research Taste as the Human Bottleneck
    • How do you measure rubber-stamping? "Humans set direction" can be true on paper while real judgment quietly transfers to the model.
  • Researcher Uplift from Code Output
    • The whole chain rests on **β = 0.5** (pre-AI coding time share), fixed "for simplicity." Kwa flags substantial uncertainty; how much does the 2.3–2.9× band widen once β is varied and measured against Anthropic's actual time-use data?
    • Greenblatt's 0.55/0.45 labor/compute split is itself an assumption. Is the true R&D production function really that insensitive to labor — and if so, does labor uplift matter far less than the RSI discourse assumes?
  • Responsible Scaling Policy Evaluations
    • The two new general-access risk pathways (other AI developers; major governments) are newly in scope but lightly evaluated — what would a positive finding there even look like?
    • How does the RSP brake interact with [[recursive-self-improvement]]: is AECI-based gating fast enough if acceleration compounds, and does single-lab gating even matter without the multilateral [[frontier-pause-verification|pause-verification]] regime?
  • The Abstraction Barrier
    • Is the current paradigm of large-scale pretraining on human data *fundamentally* bounded by human conceptual frameworks, and by how much? (Report open question 1i.)
    • Does the embodied bottleneck reduce the intelligence-growth rate to empirical-science speed, and can that be modelled?
    • Can a system be built that does grounded concept discovery from raw sensor data — and is collective ASI a way around an individual cap?
  • Transformative Creativity
    • Does increasing intelligence inherently produce increasing creativity, or do transformative leaps require something (grounded discovery) the current paradigm lacks?
    • Is the AlphaGo→AlphaFold class strictly exploratory, or are there early signs of transformative (new-conceptual-space) creativity?
    • Could transformative *artistic* creativity ever emerge from optimization power without lived cultural grounding?
  • Universal AI (AIXI)
    • Does modern agentic scaffolding (or RL-tuned implicit decision-making) actually satisfy the AIXI planning ideal, or only superficially resemble it?
    • Can the embedded/multi-agent AIXI extension produce *practical* insight for real multi-agent ASI ([[multi-agent-collective-intelligence]]), or does it remain a theoretical patch?

Entities 59 open

  • AlphaProof Nexus
    • The framework's reach is gated by [[lean]]'s mathlib maturity. What's the path to domains needing new theory rather than subgoal decomposition?
    • AlphaProof adds little as a soloist but helps as a tool. As the prover LLM strengthens, does the AlphaProof tool become redundant entirely?
  • Anthropic Institute
    • How does the Institute's policy posture (favoring an *option* to pause) interact with Anthropic's commercial incentive to ship frontier models? The essay acknowledges the competitive/geopolitical pressure but doesn't resolve it.
    • What concrete verification mechanisms will the Institute prototype, and on what timeline relative to the RSI trend it warns about?
  • Campfire
    • Campfire claims its AI edge comes from "our own foundation model." For an ERP, what does a custom foundation model actually buy over fine-tuning a frontier model — and is it durable as frontier models improve (cf. [[harness-shrinkage-as-models-improve]])?
    • "Never had anyone outgrow Campfire" — does that hold as customers reach true enterprise scale where NetSuite's breadth historically mattered?
  • Claude Design
    • Did the "any design tool via MCP" integration actually ship on the stated timeline? (Forward claim from May 2026.)
    • How does Claude Design's eval discipline work for visual/aesthetic output, where there's no compiler or test? (Same open question as [[cowork]] for non-code artifacts; relates to [[wiki/derived/evals-for-taste-and-character|character/taste evals]].)
  • Claude Fable 5
    • **Why was access suspended after launch?** The source banner gives no reason (capacity? a safety finding? the UK-AISI jailbreak progress noted in [[capability-gated-model-fallback]]?). Not in source.
    • Exact benchmark numbers vs GPT-5.x / Gemini are image-only in the source; not transcribed.
    • How much of Fable's general-access experience is *actually* Fable vs Opus-4.8 fallback for security-research-adjacent users whose queries trip the conservative classifiers?
  • Claude Mythos 5
    • **Suspension reason** — shared with Fable 5; not stated in source.
    • How does "somewhat stronger than Mythos Preview" square with Opus 4.8's card claiming Mythos Preview was the capability frontier? The frontier has moved; the magnitude isn't quantified here.
    • The bio trusted-access SKU is "Fable 5 with bio safeguards removed," not Mythos 5 — so "Mythos 5" strictly denotes the cyber-lifted variant. Whether these converge under one trusted-access umbrella is unstated.
  • Claude Opus 4.7
    • Do Hakim's (2026) brevity-constraint findings on Opus 4.6 replicate on Opus 4.7, or does the literal-instruction-following change the elasticity? Specifically: does `<50 words` still yield +13.1pp on GSM8K?
    • Does Opus 4.7 still underperform as a planner in HotpotQA-style combo sweeps, or does improved instruction-following close the gap that AgentOpt (Hua et al., 2026) identified?
    • What is the real-world token-inflation multiplier on typical Claude Code sessions (1.0–1.35× is content-dependent — what's the distribution on code-heavy vs. prose-heavy inputs)?
    • How does xhigh compare to max on coding evals? The migration guidance says "start with high or xhigh" — is max ever worth it for coding?
    • What fraction of existing CLAUDE.md / system-prompt hedges become counterproductive under literal instruction following?
  • Claude Opus 4.8
    • Public model ID and pricing: the card does not state them; presumably `claude-opus-4-8` at the Opus tier.
    • Does the grader-speculation trend continue to escalate in the next model, and at what point does it begin to affect outward behavior?
    • Why is 4.8 *less* robust to prompt injection than 4.7 despite broad alignment gains — a capability/robustness tradeoff, or an artifact of the eval surface?
  • Claude Sonnet 5
    • The head-to-head benchmark numbers vs Sonnet 4.6 and Opus 4.8 are image-only in the source; the System Card has the full set.
    • What is the real-world token-inflation multiplier on typical Sonnet 5 traffic (1.0–1.35× is content-dependent), and does "roughly cost-neutral" hold once effort levels rise?
    • Why does a *mid-tier* model show *higher* behavioral-audit misalignment than the more capable Opus 4.8 and Mythos Preview — a capability-alignment coupling, or a training-recipe difference between the Sonnet and Opus/Mythos lines?
    • At what effort level does Sonnet 5 actually match Opus 4.8, and how does the crossover cost compare to just running Opus 4.8?
  • Cowork
    • How does Cowork's harness compare to [[claude-code]]'s? Both surface skills, MCP, sub-agents — but the failure modes for non-code output differ (no test suite, no compiler, no diff to review).
    • What's the eval discipline for Cowork-class outputs? Cat Wu says memory benefits a lot from evals; unclear how slide-deck quality is measured.
  • Emergent
    • Which founding timeline is correct — is Emergent a YC Summer-2024 company (Tan) or a June-2025 founding (TechCrunch)? A future authoritative source (Emergent's own about-page, YC batch records, Crunchbase) should settle it; the machine transcript's self-flagged name uncertainty makes Tan's the weaker claim, but the discrepancy is unresolved.
  • FastContext
    • Can the SFT+RL recipe push the explorer below 4B (1.7B / 0.6B) and make exploration effectively free?
    • Does the gain transfer beyond Mini-SWE-Agent to richer harnesses with their own subagent orchestration?
  • Gemma 4
    • Why does the **MoE underperform the dense model**? Gemma 4 26B-A4B scores Elo 1438 on Arena against the 31B's 1451, despite MoE being the architecture every larger open model in their own table uses. Not addressed in the paper.
    • The pre-training cutoff is **January 2025** but the model reports 89.2 on **AIME 2026**. The report says data was filtered "to decontaminate benchmarks." What does that leave, for a competition held after the cutoff?
    • Is the encoder-free 12B's dense-text degradation an artifact of the 35M projection doing no feature compression, or of the 12B's training run specifically? A same-size encoder/encoder-free ablation would settle it; the paper runs none.
  • Google DeepMind
    • DeepMind reports its bespoke systems being caught by simple loops. Does the lab's comparative advantage move from *systems* to *models + verifiers + benchmarks* (mathlib, Formal Conjectures)?
    • The paper opens AI-for-math; what's DeepMind's next target domain where a sound verifier exists?
    • Gemma 4's MoE (26B-A4B) loses to Gemma 4's dense 31B on human preference, in a landscape where every larger open model is an MoE. Does DeepMind believe sparsity's returns only begin above some scale, or is this a training artifact it hasn't explained?
    • How does a lab hold the Frontier Safety Framework and an open-weight thinking model in the same hand? The published answer is that Gemma is far from the thresholds. That answer expires.
  • Hermes Agent
    • The container backend disabling dangerous-command checks is a defensible design but a meaningful security-model shift. What's the empirical track record? Have lockdown failures in popular images (Daytona, `nikolaik/python-nodejs`) caused incidents?
    • How do bounded memory files (~2,200 chars `MEMORY.md`) hold up over long-term use? Auto-consolidation is mentioned but not specified — what's the consolidation algorithm and how lossy is it?
    • Hermes's DM-pairing flow is a clean security primitive. Why hasn't this pattern been adopted by Claude Code or Cursor for shared/team deployments?
    • The split between `AGENTS.md` (project) and `SOUL.md` (personality) is explicit in Hermes but implicit in Claude Code's `CLAUDE.md`. Does the split materially improve outcomes, or is it a documentation choice without empirical backing?
    • Cron jobs in fresh sessions with no memory — how do teams structure the "context the agent needs" without it bloating every cron prompt? Is there a standard pattern?
  • Lean
    • mathlib maturity gates the reachable frontier. Can AI formal proof search *grow* mathlib (formalize new theory) as a byproduct, expanding its own frontier?
    • Lean is a perfect verifier for math. Which other domains have a comparably sound automatic verifier (vs. only noisy ones like tests or LLM-judge councils)?
  • Marcus Hutter
    • AIXI is incomputable and non-embedded; how far do recent fixes (amortized predictors, embedded/multi-agent AIXI) carry the theory toward *practical* relevance for real ASI?
  • METR
    • What new tasks will METR build to measure days- and weeks-long horizons once current baskets saturate?
    • METR also runs the [research showing developer self-estimates of AI uplift are overstated](https://arxiv.org/pdf/2507.09089) — how does it reconcile that skepticism with its own steep time-horizon curve? **Sharpened:** [[researcher-uplift-from-code-output]] — a METR modeler (Kwa) threads exactly this needle: he *discounts* self-reports (citing METR's felt-+20% / actual-−20% finding) and flags verbosity, yet still estimates >2× researcher uplift from an *objective* 8×-code-output figure rather than from self-estimates — i.e. METR's skepticism is specifically about *self-report* metrics, not about the acceleration being real.
  • Mythos Model
    • Do [[claude-fable-5|Fable 5]] / [[claude-mythos-5|Mythos 5]] return after the post-launch suspension, and when?
    • Capability profile beyond cybersecurity: Mythos Preview focused on the safety story; other capability dimensions not well-documented externally.
    • Internal access controls: who at Anthropic actually uses Mythos for daily work, vs Opus 4.7? Boris implies infrequent (try-it use); not detailed.
  • Perplexity
    • A vendor publishing a benchmark its own product wins is an obvious incentive problem — how is DRACO's credibility maintained as it ages, and will Perplexity actually run the automatable refresh?
    • Perplexity depends on Anthropic (and others) for base models while competing with them on the end product — how durable is the orchestration advantage if base-model makers ship their own deep-research mode?
  • Shane Legg
    • The report assumes alignment is "solved to a sufficient degree" to focus on trajectories — how does Legg's AGI-timelines optimism square with that scoping choice?
  • Symphony
    • The **500% landed-PRs claim** is hedged — no baseline definition, "on some teams" only. What does the distribution look like across teams? What happens to PR *quality* and revert rate at that throughput?
    • "Workspaces preserved across runs" is the opposite of typical CI ephemerality. At what point does state pollution from prior runs (stale `node_modules`, leftover branches, build artifacts) start hurting more than warm-cache helps?
    • Symphony doesn't write to the tracker — agents do. This means tracker policy is a *prompt* in `WORKFLOW.md`. How brittle is this in practice when Linear changes its API? How is consistent state-machine behavior enforced when agents have prompt-level discretion?
    • The spec was simplified by being implemented in 6 languages. What's the extension of this technique? Could `compiler-prompt.md` in this vault be similarly cross-fuzzed?
    • Symphony explicitly says agents can self-create tickets. What governance prevents runaway ticket-graph expansion? Is human triage of agent-created tickets the only check?