Tool-Catalog Context Cost

Sources#

• Agent Skill Evolution: How Revisions Affect Coding Agents
• Designing MCP Gateway: Uber's MCP Management Platform
• Running a Software Factory Efficiently at Uber Scale

Summary#

An agent's tools have to be described to the model before it can call them, and the standard way to do that is to put every definition (name, description, JSON schema) in the prompt prefix at session start. The cost of that prefix scales with the size of the catalog, not with what the task uses. At a handful of tools the cost is invisible. At an enterprise catalog it dominates the first call and crowds the context window. This page collects the measured size of that tax across the wiki and the three routes that have been shipped to avoid it.

The anchor is Uber's account of its MCP Gateway (Srivastava et al., Uber engineering blog, 2026-10-01, case-study, first-party), which fronts 800+ MCP servers and 5,000+ tools (self-reported) and states the problem directly: wiring an agent to hundreds of servers "doesn't scale, as all this context would eat up the model context limit." Its two fixes, Omni MCP and Code Mode, are two of the three routes below. The gateway itself (registry, AutoCrawler, data plane, authorization) is written up on MCP and Computer Use.

How large the tax is#

Three independent measurements, from different setups. None measures what the tax does to task success.

Two qualifications come with these numbers.

• Caching discounts the dollars but not the window. Anthropic implicitly caches a large tools= array without a cache_control marker, so in the CAPC run most of the catalog was already billed at the cached-read rate (Prompt-Cache Economics). A cached catalog is cheap per call. It still occupies the context window, which is the constraint Uber names, and it still sits in front of the model at selection time.
• Compressing the catalog can beat curating it. In the same CAPC run, query-agnostic compression of all 287 definitions picked tools closer to the uncompressed baseline than query-aware compression (set-IoU 0.700 vs 0.603), because a query-aware compressor discards the definitions the model was about to need. Any discovery route that pre-filters by the query inherits the same risk.

Three routes off the resident prefix#

1. Deferred loading behind a search tool. Definitions stay out of the prompt until the agent fetches them. Claude Code ships this as deferred tools fetched through ToolSearch (Agent Harness Engineering §Context Engineering for Claude 5-Class Models, practitioner-opinion). The catalog lives in the harness, and the model sees names, then schemas on demand.

2. A meta-tool discovery proxy (Uber's Omni MCP). MCP itself has no cross-server search: "an agent has to already know which server to talk to before it can ask what tools are available." (The 2026-07-28 spec revision's mandatory server/discover does not change this. It describes one server's versions and capabilities, not the catalog; see MCP and Computer Use.) Omni MCP is a single proxy server that exposes four tools:

• discover_server: find servers by query intent
• discover_tools: list the tools on one server
• get_tool_schema: fetch the JSON schema for one tool
• invoke_tool: call it, with the gateway's access control applied

The client configures one endpoint, and context is "loaded step by step, never the full catalog at once" (Figure 6). This is route 1 lifted out of the harness into the network. Any MCP client gets incremental discovery without its harness supporting deferred tools, and the gateway applies authorization at invoke_tool. The cost is turns: a call that used to take one step now takes up to four. Uber reports no turn counts, latencies or selection accuracy for it.

3. A CLI that keeps definitions out of context entirely (Uber's Code Mode). Coding agents already have a shell, so Uber's aifx CLI routes MCP calls through the gateway "without requiring any MCP server to be installed": aifx mcp list, aifx mcp search (across all servers) and aifx mcp call. The agent can chain them in one command and write the output to files, which it then greps, so only the lines it needs enter context. This cuts both sides of the bill at once: no resident definitions, and no full tool response in history. That second half is Tool-Output Pruning's problem solved by indirection instead of by a pruning model. Uber says Code Mode is "now the company default for MCP tool use in coding agents." The default is scoped to coding agents. Omni MCP remains the route for MCP clients without a shell.

The earlier Uber post supplies the only numbers for route 3, measured once on five identical SQL queries in one session: 55–71% token reduction on small result sets from removing polling and schema overhead, ~100% on a 50-row result (1,431,594 → 900 tokens). Both posts are first-party and share an author (Medisetty), so the gateway post is the architecture behind those numbers, not independent corroboration of them (Cost-per-Task Over Cost-per-Token §The full stack).

A fourth move trims the response rather than the catalog. Response Projection adds a field to each tool's request schema in which the model lists the nested paths it wants, and the gateway drops everything else from the response at runtime: "a GraphQL-like calling pattern." Uber credits it with scaling "API schema compatibility for MCP at the enterprise level." It reports no token figure.

The same ceiling on the skill catalog#

Agent Context Files carries the open question this page's material bears on. Skill runtimes (Claude Code, Genkit) inject every installed skill's frontmatter description into the system prompt at initialization, so a large enough skills/ directory should hit the same wall a large tool catalog hits. Uber's account is field evidence that the wall is real at the tool layer. A team running hundreds of servers chose not to keep the catalog resident, and its stated reason is the context limit. It supplies no threshold, though: no catalog size at which selection broke, and no adherence or accuracy curve. The question's falsifiable form (N resident descriptions against a selection or compliance metric) is still unrun.

What deferral costs in effect, measured on skill bodies (October 2026). Skill runtimes already defer one layer: the description is resident and the body loads on demand. Wang et al. (arXiv 2610.04832, empirical) price that choice in both currencies on 80 sandbox tasks. Preloading a Skill body costs +37 to +57% tokens per episode over description-only (mini-SWE-agent re-sends it every turn). Deferring it keeps about 51% of a revised rule's action gain on average across four agents — all of it for Claude Sonnet 4.5, about 38% for three open models, whose loss comes partly from a runner that shows only 2,000 characters of each end of a read. That is the trade this page's first open question asks about for tools (context saved against errors made), answered for skills on one scaffold: the saving is real, and whether it is paid back in errors depends on the consumer model. It bears on that question without settling it, since tool selection from a search step is a different failure from not reading a body.

Where this sits#

• Cost side. The tax is a fixed per-call overhead. That makes it decisive on short rollouts and secondary on long ones, where history replay dominates (Harness Tax: Coding-Agent Cost Multiplies Across Harnesses While Success Barely Moves, Cache-Aware Agent-Loop Cost (TurnCost)). A catalog moved off the prefix saves the most where tasks are short and numerous.
• Validity side. Reasoning–Acting Interleaving (ReAct) notes that enumerating the legal action set in the prompt stops working once the action space outgrows the context. Every route above gives up the enumeration on purpose. What replaces it is schema validation at call time, which Uber's data plane runs as its own layer (Figure 5, L4). That catches malformed calls, not wrong-tool choices, and the gap is recorded on Harness Patterns Under Scale and Domain Shift: Context Routing, Other Domains, Large Action Spaces, and the Overseer ("three recovery routes exist; none is measured at scale").
• Governance side. Routes 2 and 3 work only because a gateway already sits between agent and tool, so discovery comes bundled with authorization, redaction and metering. The same chokepoint carries the security controls described on MCP and Computer Use.

Connections#

• Skill Revision Effects: the skill-layer version of deferred loading, priced in effect as well as tokens: a preloaded body costs +37–57% per episode, and loading it on demand keeps ~51% of a rule's action gain (100% for Sonnet 4.5, ~38% for three open models)
• MCP and Computer Use: home of the gateway architecture (AutoCrawler, disabled-by-default registry, six-layer data plane, tool-level authorization) and of the spec's server/discover, which is not cross-server discovery
• Agent Context Files: the skill-catalog version of the same ceiling, and the open question this page partially answers
• Cost-per-Task Over Cost-per-Token: Uber's earlier lever list, with the only measured numbers for the search + CLI route (50–70K → near zero; the code-mode SQL table)
• Prompt-Cache Economics: the 287-tool, ~93K-token catalog; implicit caching of tools= arrays; query-aware compression discarding the tools about to be selected
• Harness Tax: Coding-Agent Cost Multiplies Across Harnesses While Success Barely Moves: the 13.7× first-call context gap between a 23-tool and a 4-tool harness, at success rates within ±2%
• Tool-Output Pruning: the output-side twin; Code Mode's write-to-file-then-grep and Response Projection reduce what enters history without a pruning model
• Agent Harness Engineering: deferred tool loading via ToolSearch as a shipped harness mechanism
• Reasoning–Acting Interleaving (ReAct): why enumerating the action set in context fails at scale, which is the guarantee every discovery route gives up
• Cache-Aware Agent-Loop Cost (TurnCost): why a fixed per-call overhead matters less on long loops
• Harness Patterns Under Scale and Domain Shift: Context Routing, Other Domains, Large Action Spaces, and the Overseer: deferred tool loading as one of three unmeasured recovery routes for large action spaces

Open Questions#

• What does incremental discovery cost in turns and selection accuracy? Omni MCP turns one call into as many as four, and a search step can miss the right tool where a resident catalog would not. No source reports success rate, turn count or wrong-tool rate for a discover-then-invoke proxy against the same agent with the catalog resident. A matched A/B on one task set would settle whether the context saved is paid back in errors.
• Is Code Mode's file-then-grep pattern a coding-agent artifact, or does it carry to agents without a shell? Uber scopes its default to coding agents and keeps Omni MCP for other MCP clients. Is that a capability boundary (the agent must grep well) or only a deployment one (other agents lack a filesystem)?

Sources#

• Designing MCP Gateway: Uber's MCP Management Platform: Alok Srivastava, Deepanshu Mehndiratta, Gaurav Gill, Prashik Sahare, Vaibhav Tayal, Shiven Tripathi, Uday Kiran Medisetty & Abhishek Bhatia, Designing MCP Gateway: Uber's MCP Management Platform, Uber engineering blog, 2026-10-01, case-study (first-party; every scale figure self-reported; no outcome measured). Cited for §Runtime Discovery (Omni MCP's four tools, Response Projection, Code Mode via aifx and its status as the coding-agent default), the 800+/5,000+ scale claim, and Figures 5–6 (transcribed in the raw; Figure 3 viewed)
• Running a Software Factory Efficiently at Uber Scale: Uday Kiran Medisetty, Uber engineering blog, 2026-08-27, case-study. Cited for the 50–70K → near-zero schema figure, the 49-tool/~22K SaaS server, and the code-mode SQL table; full treatment on Cost-per-Task Over Cost-per-Token
• Agent Skill Evolution: How Revisions Affect Coding Agents: Wang et al. (University of Arizona et al.), arXiv 2610.04832, 2026-10-04, empirical. §5.2.4 (on-demand retention) and §6.2.2 (carrying cost of a Skill body). Full treatment on Skill Revision Effects