Deferred Tool Loading Cuts Agent Costs by 62.5%

Deferred tool loading reduces tool schemas in an agent's context window

Deferred tool loading cuts the cost of running a coding agent by 62.5% without changing the model or framework. Baseten added the feature to its hosted open models, including DeepSeek, Kimi and GLM, and measured tasks running 26.2% faster in Claude Code with three MCP servers connected.

Why framework builders are cutting back

Agent frameworks have shifted from adding features to removing them. Edge cases and guardrails that once gave a model context for a task are now more often bloat, and teams building lean frameworks want the model’s own judgment to carry more of the work. Retrieving tools just in time for each task, instead of preloading them, fits that approach.

The cost of preloading is easy to overlook. Every tool schema from every registered MCP, including the unused ones, travels back with each request. That can easily add up to hundreds of thousands of tokens over a session.

What deferred tool loading does

Most agent setups load every tool schema from every registered MCP into the system prompt on each request. With Slack, Notion and Baseten connected, those schemas added about 141,000 tokens to every call. Deferred loading sends only tool names plus a tool_search tool of roughly 1,300 tokens. When the model needs a tool, it calls tool_search, the client returns tool_reference blocks, and the server renders the full schema at the bottom of the transcript.

The idea mirrors lazy loading in web development, where resources wait until they are needed. It is progressive disclosure for tools.

Why open models missed out

OpenAI and Anthropic APIs typically support this. But Codex and Claude Code turn it off when pointed at a third-party open model provider such as Baseten. The provider needs matching server-side gateway and inference support, so gateways serving open models fell behind. Baseten now supports it in both Chat Completions and Messages API formats.

Why it saves time as well as money

Even when a long prefix is cached, each new output token attends to every cached KV entry, so attention cost grows with context length. Eager loading means reading about 141,000 entries on every generated token. The cached context in that example is roughly 100 times larger than with deferred loading.

The deferred approach also keeps the cache valid. The schema is appended near the end of the transcript instead of changing the system prompt, so later requests share the same prefix up to the tool_search call.

The two use cases Baseten tested

The team wanted to know two things. First, what does deferred loading give almost pure coding work, such as handing an agent a plan and asking it to open a pull request? Those tasks need few tool calls, yet the cost of every registered tool still lands on each request. Second, what happens in agentic tasks where the model must search for many tools and loop through discovery? Baseten found deferred loading came out ahead in both cases and rarely had downsides.

The benchmark results

Baseten sampled 55 tasks from eight benchmarks and ran them headlessly in Claude Code using GLM 5.3, with and without deferred loading. Coding benchmarks were Terminal-Bench, SWE-bench Verified, SWE-bench Pro v2 and DeepSWE v1.1, run with three MCPs registered. Agentic benchmarks were MCP-Atlas, AutomationBench, ComplexMCP and Toolathlon-GYM, using only the tools each benchmark supplied.

  • Coding tasks averaged 62.5% cheaper and 26.2% faster.
  • Agentic tasks averaged 51.1% cheaper and 14.9% faster.

Savings grow when many or bulky MCPs are registered and when a task needs many requests. Time improved even where models had to search for tools repeatedly. DeepSWE and MCP-Atlas ran slightly slower, which Baseten considers negligible against the savings. Only Toolathlon-GYM, at 68 tool calls per task, ran better with eager loading.

What to check in your own setup

If you point Claude Code or Codex at an open model gateway, check whether deferred tool loading is enabled and whether your provider supports it. Then count how many MCP servers you register and how often the agent actually uses them. The more unused schemas you carry, the more deferred tool loading is likely to save.

Source: Baseten

Written by

Saksham Bhargava

Saksham Bhargava runs Alteroid and has been writing about consumer technology for over five years, covering iPhone, Android, Windows, AI tools and gaming. With an MCA and six years of hands-on web development experience, he tests the features he writes about on his own devices and focuses on clear, practical guides that solve real problems. Away from the screen, he enjoys travelling to places most tourists skip.