Do Survey-Assigned RSI Levels Survive an Independent Reading?

Sources#

The question#

From RSI Autonomy Levels (B0–L5): the survey (arXiv 2609.11873, practitioner-opinion) assigns B0–L5 levels from its own reading of each cited work. It runs no inter-rater check and publishes no rubric beyond the prose definitions. Does an independent reading of the same system land on the same level? The bullet calls this falsifiable inside the vault, and it is.

Method. For each system, I took (a) every placement the survey makes, found by grepping its citation number across tables, prose and appendices, (b) the level this wiki already assigned at compile time, and (c) a fresh assignment from the §3.7 boundary definitions applied to the wiki's independent sources. Those definitions are: persistence for B0→L1, strategy selection for L1→L2, learner-conditioned acquisition for L2→L3, deployment under changing conditions for L3→L4, and an inherited improvement mechanism for L4→L5. Of the five systems the bullet names, three are placed by the survey (Ouroboros, DGM, Absolute Zero). The survey cites neither the Caltech protocol nor Cline, so for those two the comparison is wiki-versus-independent only. I added a fourth survey-placed system with an ingested primary, OpenAI's Harness Engineering.

The table#

System	Survey's placement(s)	Wiki's compile-time placement	Independent reading	Verdict
Absolute Zero	L3, three times in Table 2 (self-play, RLVR, execution verification) and as a learner-conditioned acquisition example in §3.4 (line 697: "solver-dependent learnability rewards guide executable task proposal")	L3 (Rationale Bootstrapping (STaR))	L3. CS329A's independent account (CS329A Self-Improving AI Agents — Part 9: Future Research Areas) gives the reward as `1 − average success`, zero for unsolved tasks, so what the proposer generates next is conditioned on the current solver. That is the L2→L3 criterion exactly. Task formats, the executor and the update rule stay external, so it is not L5	Agree
OpenAI Harness Engineering	L1 (Table 2; §3.2: engineers define intent, architecture, CI and rollback, and Codex implements)	not placed	L1 on the improvement decision. When the agent struggles, "human engineers always stepped into the task and asked: 'what capability is missing'", then "feed it back into the repository, always by having Codex itself write the fix" (Harness engineering: leveraging Codex in an agent-first world). But acceptance is internalized. Review is "pushed almost all … agent-to-agent", and agents "often squash and merge their own pull requests."	Agree on level. The ladder has no slot for the acceptance authority this system has delegated
DGM	Four statements: the L5 column of Table 2; "L5 cand." in Appendix B; "bounded L5 characteristics" in Appendix C; "a transition case; task gains alone do not establish a better improvement procedure" in §3.6.1. Table 7 lists parent selection and the benchmark as external	RSI Autonomy Levels (B0–L5): "the paper declines the L5 attribution"	Structural L5, bounded. DarwinX's related-work section, written by authors with no stake in DGM, puts it in the self-referential regime that "compounds task improvements into better self-modification". The agent's edited source is the code that performs the next round's modification (DarwinX: Evolving Agent Harnesses Through Natural Selection). Archive management and parent selection stay outside. Effective L5 is unestablished, and the wiki holds evidence against it. Under HarnessBank's matched protocol, DGM is credited on one of five sealed tests, picks a 15-task spike on LiveCode (0.733 regressing to 0.533), and ships a harness worse than vanilla on Omni-MATH (−1.1%) (HarnessBank: Semantic Gene-Bank Search with Gated Verification for Agent-Harness Self-Evolution, Agent-Authored Harness Optimization)	Agrees with three of the survey's four statements once the structural/effective split is applied. The fourth (§3.6.1) declines only the effective claim. The wiki's "declines the L5 attribution" over-reads it
Ouroboros	One citation, §1.2 Case 2. It is framed as deployment adaptation, but it also says the update "changes the mechanism through which later failures are diagnosed and repaired." Appendix C's software-engineering frontier says "L4 environment adaptation remains largely absent" and names only SICA and DGM as bounded L5. Ouroboros appears in neither list	L4, "no structural L5 claim however many commits it lands" (Continuous Self-Modification Under Review, echoed on Recursive Self-Improvement and RSI Autonomy Levels (B0–L5))	Mostly L4, plus one narrow structural-L5 instance. In experience-driven core evolution, "Hope decides which proposals identify real problems and which changes to pursue", and the review gate and owner modes keep acceptance external. That is L4. The second worked case goes further. Deep self-review tasks were aborting on review-pack context overflow, and the agent "replaced the assembly path" with a centrality-ranked context atlas that "preserves high-connectivity core files during review." Review logic lives in the mutable repository, and the constitution allows changing review bounds under plan review (Ouroboros: A Self-Developing Frontier Coding Agent with Reviewed Core Evolution §3, P3). So a mechanism that diagnoses later failures was revised by the AI, persisted, and governs later rounds. That meets the survey's structural-L5 criterion for that mechanism. In recursive free evolution, "completion can schedule another evolution cycle", which also puts a continue/stop decision inside, the decision Table 8's meta-recursion row asks evaluators to record. Effective L5 is unmeasured. Every benchmark row ran with `evolution off`	The survey contradicts itself. The independent reading sides with Case 2's prose on structure and with Appendix C on "full L5 … not yet demonstrated". The wiki's "no structural L5" is the outlier
Cline's July 2026 campaign	not cited	L2 (Recursive Self-Improvement)	L2. The agent chooses which defects to fix from traces under a human brief with a pinned end state. The objective is fixed (Terminal-Bench), the merge is external, and persistence comes through the merged PR (Agent-Authored Harness Optimization)	Agree (wiki vs. independent)
Caltech knowledge protocol	not cited	L1, "nothing about who decides what to retain has moved" (Recursive Self-Improvement)	Not L1. The cross-task forum "decides which local observations survive beyond the task that produced them" through mandatory `AGREE`/`DISAGREE`/`SYNTHESIZE` stances, and distillation picks what enters the next bundle, including `next_steps` for the next attempt (Knowledge-Centric Self-Improvement). Agents make the retention decision. The survey's practice puts structurally identical text-memory distillers (ReasoningBank, Dynamic Cheatsheet, ACE) in its L4 column (Table 2, Table 6). Caltech falls short of L4 only on the deployment clause, because its loop runs over a fixed 50-task benchmark pool and not under changing operational conditions. Best reading: L2 with an L4-shaped retention mechanism	Disagree with the wiki's L1

What the check shows#

1. Below L4 the ladder is reliable. The persistence, strategy and learner-conditioning boundaries are each a yes/no fact about the loop that a reader can check against a paper's method section. Every system whose relevant mechanism sits in that range was placed the same way by every reader: Absolute Zero, Harness Engineering and Cline. Two readers working from different sources agree there.

2. At L4–L5 a "survey-assigned level" is often not one level. The survey makes four placements of DGM and two of Ouroboros, and they do not all agree. The cause is visible in its own text. §3.6 states that "classification therefore applies to the particular mechanism examined" and that "self-modifying task code provides insufficient evidence when the process responsible for later revisions remains unchanged." Yet its tables and appendices give one level per system. A system with several mechanisms gets whatever level the sentence happens to be about. Once each placement is read per mechanism and split structural/effective, every survey statement and every independent reading reconciles:

DGM: structural L5 for the self-modifying agent code, bounded by external parent selection. Effective L5 not established.
Ouroboros: L4 for the deployment-driven repair loop. Structural L5 for the review-pack assembly. Effective L5 unmeasured.

So the answer at L4–L5 is a reporting rule, not a correction to any one placement. Quote a level only with the mechanism it is about and whether it is structural or effective. A bare "DGM is L5" or "Ouroboros is L4" is not falsifiable either way.

3. Two axes the ladder cannot express caused the remaining friction.

Acceptance authority has no rung. Harness Engineering delegates review and merge to agents while humans keep the "what's missing" decision. Agentic Self-Modification (Agent-Initiated Weight Updates) already records the mirror case, where an agent exercised acceptance and release decisions its designers never delegated. The ladder assumes decisions transfer in order: execution, strategy, agenda, deployment, mechanism. These two systems show acceptance can transfer independently.
The L3→L4 "deployment" clause decides whether retention counts as L4. The same retention mechanism counts as L4 for ReasoningBank and Dynamic Cheatsheet but not for the Caltech protocol, and the only difference is whether the task stream counts as deployment. The survey never defines that threshold.

4. The inter-rater check cuts both ways. The stated worry was that the survey's placements lack a second reader. On this sample, the survey's placements below L4 hold. The demonstrable errors are in this wiki's own compile-time placements, which were made in one pass from the survey's vocabulary without going back to the primaries:

Caltech at L1 contradicts the Caltech page's own description of agent-decided retention.
Ouroboros "no structural L5" quoted §1.2 Case 2 up to the sentence that makes the mechanism claim and stopped.
DGM "declines the L5 attribution" reads §3.6.1's effective-L5 hedge as a structural verdict.

Limits#

n is small and the second reader is one synthesis. Four survey-placed systems with independent treatment in the vault, plus two wiki-only placements. This is a falsification check of the "levels are rater-dependent" worry, not an agreement coefficient.
DGM's independent reading is secondary. The DGM primary (arXiv 2505.22954) is not in raw/. Its structure comes from DarwinX's related-work and comparison tables and its behavior from HarnessBank's rerun. Both are empirical and neither has a stake in DGM's framing, but neither is the DGM paper.
The Ouroboros structural-L5 instance is one documented case. The paper reports one review-mechanism edit end to end. How many of the 1,085 commits touched improvement machinery and not task machinery is not reported.

Flagged for correction (applied 2026-10-21)#

Each correction below was applied on 2026-10-21 as a struck-through original plus a dated correction note, so the earlier claim stays visible.

RSI Autonomy Levels (B0–L5), §L5: "in the two cases where the untouched thing is the selection rule … the paper declines the L5 attribution". §3.6.1 declines the effective claim for DGM. Table 2 and Appendices B and C place it as an L5 candidate or bounded L5. Applied 2026-10-21 on RSI Autonomy Levels (B0–L5) (§L5, plus the two Ouroboros echoes in its per-mechanism paragraph and its Connections line).
Continuous Self-Modification Under Review, How an external survey classifies it: "The level assignment is L4, not L5 … it cannot support a structural L5 claim however many commits it lands". The context-atlas case is a structural-L5 instance for the self-review mechanism. The commit count was never the evidence. The quoted Case 2 passage also stops before the survey's mechanism sentence. Applied 2026-10-21 on Continuous Self-Modification Under Review: the quote now runs to the mechanism sentence, and the level paragraph and the Connections line to RSI Autonomy Levels (B0–L5) were corrected.
Recursive Self-Improvement, The vocabulary problem: the Caltech knowledge base as "L1 by the survey's criterion" (see the table above), and Ouroboros as "L4" with "None of them reach L5". The latter should read "none establishes effective L5". Applied 2026-10-21 on Recursive Self-Improvement: Caltech is now L2 with an L4-shaped retention mechanism, Ouroboros is L4 plus one structural-L5 instance, and the line reads "none of them establishes effective L5".

Citations#

The Last AI Built by Humans: Toward Genuine Recursive Self-Improvement: §1.2 Case 2 (Ouroboros), §1.3 (DGM autonomy attribution), Table 2 (cross-level technique map), §3.4 and line 697 (AZR), §3.6 and §3.6.1 (structural/effective split, per-mechanism classification, the DGM transition case), Table 7, Appendix B (DGM "L5 cand."), Appendix C software-engineering frontier
Ouroboros: A Self-Developing Frontier Coding Agent with Reviewed Core Evolution: §1 two evolution modes, §3 mutable repository contents, Evolution during deployment (context atlas), constitution P3 and P4
DarwinX: Evolving Agent Harnesses Through Natural Selection: §1 and related work (DGM as self-referential; single-parent mutation; archive/selection)
HarnessBank: Semantic Gene-Bank Search with Gated Verification for Agent-Harness Self-Evolution: §4.3 the DGM rerun under a matched protocol
CS329A Self-Improving AI Agents — Part 9: Future Research Areas: the Absolute Zero learnability reward
Harness engineering: leveraging Codex in an agent-first world: the human "what's missing" loop, agent-to-agent review, self-merge
Knowledge-Centric Self-Improvement via Knowledge-Centric Self-Improvement: the cross-task forum and distillation stages