Co-Installed Skill Conflicts

Sources#

• One Skill Too Many: How Co-Installed Skills Conflict in Coding Agents

Summary#

Every skill measurement elsewhere in this wiki holds the skill catalog fixed and asks whether the skill helps (Skill Lift), loads (Harness Activation and Adherence) or improves under revision (Skill Revision Effects). A real session rarely holds one candidate per job. Project skills come from the repository, personal skills from ~/.claude/skills/, plugins add their own, and teams copy whole collections (Skill Copy Provenance). Nobody chooses the pair. The model chooses between the two skills from their names and descriptions, and a skill's requirements live in its body, which loads only after the choice is made.

Yan, Xu, Li, Xu, Liu, Deng & Ma (UNSW, Griffith, NTU, Wollongong; arXiv 2610.11647, 2026-10-08, empirical) are the first to measure what that costs. Their terms, used throughout this page:

• Installed skill A is the one the project or user installed for a purpose. Similar skill B does the same job. A third skill C is an unrelated control.
• Core functions are A's requirements that a competent model would not meet without the skill, quoted verbatim and checkable in the output. Prohibitions count (for example, "never touch git"). They are extracted from each skill alone, before any run. A core function is exclusive when B does not ask for it. 73% of the applicable core functions in the experimental pairs are exclusive.
• Fidelity is the share of A's first three applicable core functions that a run fulfils. It is reported beside task completion.
• Substitution means B is used instead of A. Interference means A is used but the result still changes.

Five findings carry the page:

1. Same-job pairs are common. About one installed skill in four (23.5%) already shares its installation list with a same-job skill. About two in three (63.7%) have a same-job skill by another author that a user could add. In projects that copy a whole collection the rate is 37%, against 19% elsewhere.
2. Most such pairs are normative skills (conventions and formats): 60% of cross-project and 56% of within-project pairs. Capability skills account for 31% and 35%.
3. The loss is invisible to a pass rate. Co-installing B lowers use of A by 19.9 pp and leaves task completion flat (+1.9 pp, CI [−1.0, +4.9]). Fidelity falls 2.6 pp overall and 5.6 pp on A's exclusive core functions, and shared core functions do not move (+0.1).
4. Install scope decides which skill runs, and the user is not told. Listing order moves use of A by +0.6 pp. A same-name personal skill removes A from every listing. When B replaces A, the final reply names the skill it used in 0.9% of runs.
5. The conflict is decided at the first skill read, and it can be stopped there. In 97% of runs that open B first, no file has changed yet. A pre-tool hook that denies the first read of B redirected the model to A after 96% of denials and recovered all of the loss.

How the population was built#

The corpus is 55,698 distinct public SKILL.md files (105,302 non-fork copies) in 20,947 repositories, snapshotted 2026-07-18. Each repository's full file tree is fetched at a pinned commit, because GitHub code search indexes only default branches and returns at most 1,000 results per query. A skill counts as installed when Claude Code would load it in a session: project, nested, dotfiles-personal or committed-plugin locations, with frontmatter on line 1, valid YAML, no disable-model-invocation, a non-empty name and description, and at least ten instruction lines. That keeps 207,863 of 219,817 files (94.6%). Near-duplicates are grouped into families (MinHash over word 5-grams, Jaccard 0.8), and two skills of one family are never paired.

GPT-6 Astra types the repositories. Real projects (4,354) and dotfiles repositories (285) supply installation lists. Curated collections (908) and single-skill repositories (77) supply only B candidates. That yields 5,106 installation lists.

Candidates are retrieved by embedding the text the model sees when it chooses (name plus description plus when_to_use, Qwen3-Embedding-0.6B), top ten per installed skill: 439,860 cross-project and 382,249 within-project pairs, 822,109 in all. A pair judge (GPT-6 Astra) then reads both bodies and confirms "same job" or "near-duplicate." A stratified sample of 3,754 pairs gives 1,639 confirmed (repeat-judge κ = 0.93).

The calibration result matters on its own. Below a description similarity of 0.9, only 10–37% of cross-project pairs in each band are confirmed. Similar wording does not imply the same job, so a deduplicator keyed on description similarity alone would be wrong in both directions.

The experiment#

There are 312 confirmed pairs in six cells: cross-project or within-project, by normative, capability or script-bearing type. Each pair has one case: a request for A's job that never names a skill or quotes its conventions, plus a starting repository of at most 15 files. Configurations change one factor at a time: only-A, only-B, A+B, A+C, A+B with the listing order swapped, and, for the 81 same-name pairs, B in the personal scope or in a plugin. Each runs once per pair on Sonnet 4.6, Haiku 4.5 and Opus 5, under Claude Code 2.1.283 in fresh offline containers with at most 80 turns and 25 minutes. That makes 5,162 analysed runs, plus a 670-run guard experiment and 1,048 repeats: 6,368 runs, 169,294 tool calls, 542 hours, about $5,550.

Skill use is read from the tool-call record. It counts a Skill-tool invocation or any read or execution of a file in the skill's directory, because models often read a skill's files instead of invoking it. Scripts decide 327 of the 921 primary core-function checks, and the output judge (GPT-6 Astra, with skill names and paths masked and the configuration hidden) decides the rest. Repeated runs agree on whether A was used in 88.9% of comparisons (ICC 0.78), and script-decided core functions have ICC 0.84.

A listing-cap caveat that changes how to read every number. Claude Code caps the whole skill listing at 1% of the context window, measured at 8,000 characters for Sonnet 4.6 and Haiku 4.5 and 6,000 for Opus 5. Under that cap, 95.8% of the A+B listings shown to Opus 5 would have lost at least one of the two descriptions. The authors raised the cap to 20,000 characters for every model. The measured conflicts are therefore the ones that occur when both descriptions are visible. At default settings, a large catalog decides many of these pairs earlier and more silently, by truncation. Tool-Catalog Context Cost covers the catalog-size side of that ceiling.

RQ1: the loss hides behind success#

The controls separate the two measures. Only-A fulfils 57.3% of A's primary core functions and only-B fulfils 46.3%, at equal task completion (57.2% against 58.1%). So task completion cannot tell the two skills apart, and fidelity can, by 11.0 pp. On the exclusive core functions the gap widens to 16.1 pp.

With A and B co-installed:

(All values from §4.2 prose. Table 2's run counts sum to the stated 5,162 and Table 3's to 6,368; Figure 5 checked against the per-model statements below.)

• It is the similarity, not the extra skill. Adding an unrelated C lowers use of A by only 5.9 pp and leaves fidelity unchanged (−0.2). C is used in 3.0% of runs and replaces A in 0.1%.
• Most of the loss is substitution or non-use, not interference. With B present, 12.1% of runs use only B and 38.0% use neither skill, against 30.3% that skip A in the only-A arm. Substituted runs carry 36% of the fidelity loss, runs using neither skill carry 45%, and runs that used A carry 20%. A substituted run loses 8.0 pp against only-A.
• The loss grows with what is counted: 1.7 pp on the first core function, 2.6 on the first three, 4.1 on all applicable ones. It is larger on script-decided checks (4.2) than on judge-decided ones (1.2), so the judge is not producing the effect.
• Within-project pairs, the ones projects actually co-install, show the same drop (−18.3 pp use of A).

The worked case is the paper's Figure 1. A project skill checkpoint forbids any change to git state, and alone it produces a handoff note with no commit. With a same-name checkpoint-2 added, Haiku 4.5 uses the copy, commits the work, and names no skill. Both runs complete the task.

RQ2: scope decides, order barely matters, and nobody says#

• Listing order: +0.6 pp on use of A (CI [−2.0, +3.3]). Renaming B to reverse the order adds +4.4 pp on its own. This runs against the position bias reported for tool selection among functionally equivalent tools (BiasBusters). Skills here behave less like a ranked tool list than like a set the model reads from.
• Personal scope overrides as documented, but not completely. With a same-name B in ~/.claude/skills/, A appears in none of the 243 listings, and use of A drops 35.0 pp. Yet in 29.2% of those runs the model still reads A's files from the project directory, and Opus 5 does so in 53.1%. An override that removes a skill from the listing does not stop the model from opening the skill's files on disk.
• A plugin B rarely wins. Listed with a plugin: prefix beside A, it is used in only 2.9% of runs, and use of A drops 6.2 pp, against 16.9 pp for the same pairs with B in the project.
• Models differ sharply. In A+B, Opus 5 opens both skills in 36.8% of runs and neither in 15.5%. Haiku 4.5 uses neither in 56.4% (Figure 5: even only-A Haiku uses A in just 53% of runs).
• Disclosure is effectively zero. Among the 113 runs where B replaced A, the reply mentions that more than one suitable skill existed in 0%, names the skill used in 0.9%, and asks the user to choose in 0%. Across all A+B runs, the reply names the skill in 7.7% (Opus 5: 21.0%) and never asks.

The documented precedence differs by platform: Gemini CLI and Copilot CLI prefer the project skill, Claude Code prefers the personal one, and Codex lists both. As the authors note, the Agent Skills implementation guide calls project-first "the universal convention." Under Claude Code's rule, a team that ships a project skill cannot guarantee its members run that version, and the override is neither visible nor complete.

Claude Code's skill scopes (paper Table 1, from the Claude Code docs; docling welded the five rows into one, so this is the pdftotext -layout transcription carried in the raw):

RQ3: decided at the first read, guarded at the first read#

When it is decided. In 79 pair-model combinations, the A+B and swapped configurations open different skills first, and until that read their runs look alike (median 6 against 6 tool calls). Opening B first then costs 9.4 pp of A's exclusive core functions (CI [−15.0, −4.0], Holm p = 0.004). By model the cost is 22.1 pp for Haiku 4.5 and 12.1 pp for Sonnet 4.6 (not significant after correction). Opus 5 shows none, because it often reads both. In 97% of B-first runs, no file has changed when B is opened, and edits begin only after the first read (Figure 6a).

What is lost. Of the core functions only-A fulfils, opening B first loses 37% of the exclusive ones against 8% of the shared ones, with the same split for every model. Each exclusive core function adds 1.9 pp of loss.

The guard. A pre-tool hook denies the first read of B while A is unread and tells the model to use A. It names no core function. It also denies B once A is in use. Rerun on the 187 pair-model combinations that had opened B first (670 runs, hook on or off in random order):

• After 96% of 179 denials the model switched to A (Sonnet 4.6 94%, Haiku 4.5 96%, Opus 5 99%).
• Fidelity on A's exclusive core functions rose +9.1 pp over all runs. Where both runs of a combination reached for B first, it rose +17.8 pp, significant for every model (Sonnet 17.4, Haiku 20.7, Opus 12.8).
• Redirected runs matched unhooked A-first runs (0.0 pp, CI [−10.2, +10.3]), so the hook recovered the full 16.5 pp that opening B first costs (Figure 6c: about 55% → 38% → 55%). Task completion did not move (+4.1, CI [−1.3, +9.6]).

This is the cleanest case in the corpus of a deterministic pre-execution gate that raises the quality of finished work rather than blocking an unsafe action. It also bears on that page's block-versus-message question without settling it: the denial message carries a redirection ("use A") and none of A's content, and the model's own reading of A does the rest. No arm denies B without the redirection.

What it implies#

The authors' prescriptions, by owner:

• Benchmarks should score exclusive core functions next to task completion, and should co-install a similar skill beside the one under test. A with-skill/without-skill ablation such as Skill Lift's cannot see this loss, because its with-skill arm has no rival.
• Platforms should warn at session start when a personal skill overrides a project skill (the fix proposed in the seasonedcc/seasoned-skills issue #303 report the paper cites). They should show which copy and scope a running skill came from, guard the first read, pin the choice once per pair "as a lockfile does," and let a project declare which skill must win.
• Authors and users control the choice only through name and description. A distinctive name avoids a platform override, and a description that states specific tasks avoids misrouting. Users should audit personal directories for overriding skills, and projects that copy a collection should remove duplicates of skills they already install.

Reading it against the rest of the corpus#

• Harness Activation and Adherence gets a third failure at the activation gate. Lin et al.'s skill-load rate asks whether a skill entered context. With a rival present, the question becomes which skill entered, and the answer can be the wrong one with the load rate unchanged. Under this paper's definition of use, a B-substituted run counts as activated. This paper is also post-Claude-5 (Opus 5), which that page's evidence is not. Opus 5 resolves the conflict by reading both skills, and Haiku 4.5 by reading neither.
• Skill Copy Provenance supplies the cause of the 37%. Seddik finds catalogs copied in bulk, with the top 1% of repositories behind 94.7% of transmissions. This paper finds projects that copy whole collections twice as likely to hold a same-job pair. Bulk copying produces this kind of co-installation.
• Agent Supply Chain Risk has the adversarial twin. Every pair here is benign. The paper cites Saha et al. (agents prefer an adversarially described copy of a skill) and Wang et al. on compositional risk, but measures neither. The Claude Code precedence rule is still the security-relevant fact. A same-name skill in ~/.claude/skills/ silently replaces the project's version, and the user learns about it 0.9% of the time. That is a substitution channel any write to the personal directory can use. That last reading is mine; the paper treats only benign pairs.
• Harness Configuration Defects could not have found this. Kapner et al.'s surviving rules each read one file. A same-job conflict is a pairwise property, and it is not decidable from bytes, since description similarity below 0.9 confirms only 10–37% of the time. It needs a judge over both bodies or a run.
• Against the tool-selection literature (a contradiction, scoped). BiasBusters reports strong position bias among equivalent tools. Here listing order moves skill choice by +0.6 pp. The two measure different objects (a tool call against a skill whose body is read before work starts), and this paper's renaming effect (+4.4 pp) shows the choice is sensitive to the name, not the slot.
• Against Song & Wei's skill-shadowing result (consistent, different measure). Expanding a helpful library to 202 skills lowered pass rate 21 pp, mostly through wrong or missing selection. Here the pass rate is flat, because both candidates can do the job. Both describe one mechanism. Pass rate sees it when the wrong skill cannot do the job, and only fidelity sees it when it can.

Limits#

• One platform for the full design. A Codex check (gpt-5.5, gpt-5.6-luna, gpt-5.6-sol on 193 pairs) finds use of A −18.2 pp and exclusive fidelity −3.4, against −20.1 and −4.6 on Claude Code for the same pairs, with task completion flat. The paper reports this in prose only, with no table.
• Listing cap raised to 20,000 characters (above). The default-settings conflict rate is not measured.
• LLM-heavy pipeline, human-checked at tentative agreement in places. Krippendorff's α between the human consensus and the LLM is 0.72 for the pair judge, 0.71 for skill types, 0.73–0.74 for core functions and exclusivity, and 0.78–0.84 for the output judge. All are above the 0.667 tentative threshold, and several are below 0.8. The judge is cross-family (GPT-6 Astra judging Claude runs), which removes self-preference.
• Single run per combination. Defended by the repeat agreement above. The decision-point result is observational until the randomized guard experiment, which confirms it.
• Prevalence is a public-repository lower bound. Personal skills and user-enabled plugins are rarely visible in public, and the pair judge sees only the most similar candidate.
• No COI. Academic authors, no evaluated product, replication package with all run logs (github.com/ltroin/conflict).

Connections#

• Harness Activation and Adherence — the two-gate decomposition this page adds a rival to: with a same-job skill co-installed, activation can succeed on the wrong skill, and a load rate cannot tell
• Skill Lift — the with/without-skill ablation whose with-skill arm has no rival. Core-function fidelity separates A from B by 11.0 pp at equal task completion, which is the measurement a lift gate would need in order to see substitution
• Skill Copy Provenance — bulk catalog copying, the distribution pattern behind the 37% same-job rate in projects that copy a whole collection
• Agent Supply Chain Risk — the adversarial version of the same substitution: a same-name personal skill overrides the project's version with no notice to the user
• Harness Configuration Defects — the per-file static census that cannot see a pairwise, judgment-dependent conflict
• Tool-Catalog Context Cost — the skill-listing cap measured (1% of context; 8,000/6,000 characters), which would truncate a description in 95.8% of Opus 5's two-skill listings
• Deterministic Pre-Execution Gates — the guard is a pre-tool hook, and here it raises output fidelity (+17.8 pp) rather than blocking a policy violation
• Agent Skill Format — the skill format, its two-stage loading (description resident, body on demand), and the catalog-ceiling open question this page partly answers
• Human-Governed Skill Maintenance — maintenance at the source cannot help if a copy of the same skill wins selection in the user's session
• Claude Code — the platform whose precedence rule (personal over project) and listing format the experiment measures
• Stale Instruction File Precedence — scope precedence used as a lever rather than suffered: the user-level instruction file is read first beside the project file, so one sentence in ~/.claude/CLAUDE.md overrides a stale project-wide convention (4.2% → 91.4% code-following). Both papers find scope ordering deciding outcomes the platform documentation does not describe

Open Questions#

• What is the substitution rate at default listing caps? With 95.8% of Opus 5's two-skill listings over the 6,000-character cap, truncation may decide many conflicts before the model does. Settler: rerun the released harness's A+B arm at each model's default cap, recording which descriptions survived.
• Does an adversarially worded B (a description written to win routing, as in Saha et al.) take more than one run in five from A, and does the first-read guard still redirect 96%? Settler: the released harness with B's description optimized for routing, scored on A's exclusive core functions.

Sources#

• One Skill Too Many: How Co-Installed Skills Conflict in Coding Agents — Chaoliang Yan, Zihao Xu (co-first), Yuekang Li, Shangzhi Xu, Yi Liu, Gelei Deng & Siqi Ma (UNSW; Griffith; NTU Singapore; Wollongong), One Skill Too Many: How Co-Installed Skills Conflict in Coding Agents, arXiv 2610.11647, 2026-10-08, empirical, 23pp. Cited for the abstract, §2 (skill loading, scopes, definitions), §3.2–3.10 (population, retrieval and calibration, sampling, core-function extraction, configurations, listing-cap measurement, measures, human validation), §4.1–4.4 (all five findings), §5 (implications) and §6. Parse warning: Table 1 (skill scopes) is welded. Docling merged its five scope rows into one row; the table above is the pdftotext -layout transcription (page 3) carried in a note block in the raw. Tables 2 and 3 are intact: their run columns sum to 5,162 and 6,368, matching the prose. Every effect size on this page is quoted from prose. Figures 5 and 6 were viewed and agree with the prose (Figure 5's personal-scope Only-A + Both = 30% against the stated 29.2% reading A's files; Figure 6c ≈55 / 38 / 55%). Figures 1–4 were not opened.