Google's Gemini API holds back tool details until an agent asks
Summary
Google's Gemini API can now send an agent only its tool names and descriptions, loading a tool's full definition when the model asks for it, which cuts input tokens for agents wired to large tool libraries.
Log input tokens per interaction across real tasks before and after switching it on, because Google publishes no savings figure and the break-even depends on your own traffic.
Google launched Tool Retrieval in preview in the Gemini API on 19 November, letting an agent keep its tool library on standby and load a tool’s full parameter schema only when the model asks for it. Deferred tools still show the model their name and description, so it knows what exists, and the schema it requests arrives inside the same API call.
Two changes to the request switch it on: a tool_search tool in the tools array, and defer_loading: true on each tool to hold back. Gemini runs the search on its own servers, so there is no index to build or host, and the sequence appears in interaction.steps as a tool_search_call, a tool_search_result, then the function_call. Deferred schemas also stay out of the start of the prompt, which leaves a cached prefix unchanged from turn to turn.
Google publishes no savings figure, and the billing explains why. Tool Retrieval carries no separate charge, but turning on tool_search adds a fixed block of input tokens, and every schema the model loads is billed as input for the rest of the conversation, the same as a tool declared up front. Google’s post gives the break-even directly: the saving arrives when the schemas you never load outweigh the ones you do, which it calls the usual case for large libraries, and developers should measure it against their own traffic.
One constraint in the Tool Retrieval documentation decides how that math lands on a long job. A loaded tool stays in context and cannot be unloaded. By that rule, an agent that works through a broad task, pulling in one more schema at each step, finishes the conversation paying close to what declaring everything up front would have cost. The better bet is to treat this as a saving on narrow tasks against a wide library rather than on wide tasks.
That pattern fits a lot of SEO agent work. A crawler’s tool server such as Screaming Frog’s, Search Console and a rank tracker each expose long lists of operations, while the question being answered usually touches two or three of them. Google supports deferring at the server level for exactly that case: set defer_loading: true on an mcp_server tool, alongside a server-side tool_search, and every tool that server exposes loads on demand, so a large MCP server can be connected without listing its tools one by one.
Holding back schemas also puts the entire selection decision on tool names and descriptions, since that is all the model reads before it chooses. Google’s post does not say this, but a vague description that a parameter list might once have disambiguated now has nothing behind it, which makes thin tool descriptions more expensive to keep. The tool_search_call step records which names the model asked for, and that is where to look when an agent reaches for the wrong tool.
Teams running their own registry, embedding index or per-user permission filter can declare a tool_search with "execution": "client" and run the lookup themselves, at the cost of one extra round trip per search. Nothing has to be declared up front on that route, so a tool can appear mid-conversation when a user connects an integration. The documentation allows one tool_search per request and does not let the server-side and client-side modes mix.
What to do
Four steps cover most agent stacks:
- Leave the tools your agent calls on most runs declared normally, and defer the long tail. The fixed
tool_searchcost plus per-load billing means deferring a tool you always use costs more than it saves. - Set
defer_loading: trueat themcp_serverlevel only for servers whose operations the agent rarely touches, since that flag defers everything the server exposes. - Compare input tokens per interaction across a week of real tasks rather than one test prompt, because the break-even is a property of your traffic.
- Rewrite any tool description that was leaning on its parameter list to explain itself.
Watch out for
Tool Retrieval needs Gemini 3.6 or later, including every Gemini 4 model. It is in preview, and Google documents it only for the Interactions API, so an agent built on the older generateContent endpoint will most likely have to move before it can use it.