The model does not run your code. That is the thing most explanations skip. When an LLM "calls a tool," it produces a structured piece of text-a JSON object naming a function and its arguments-and then waits. Your application reads that text, executes the actual function, and sends the result back. The model never left its inference loop.
That gap between what it looks like and what is actually happening is worth understanding, because the mechanics shape every real decision you make about tool design, cost, and reliability.
What "tool calling" actually is
Function calling is a contract between an LLM and your application. You declare functions using JSON Schema. The model receives the user prompt and the tool catalog, decides whether a tool is needed, then returns a structured tool call with typed arguments. Your code runs the tool and feeds the result back.
Here is a concrete example. A user types: "What's the weather in Chicago right now?" The model has been given a schema describing a get_weather function that accepts a location string. Rather than saying "I don't have access to real-time data" - the pre-tool response -
the model scans its available tools and, having found a relevant one, requests a call to that function as a structured response.
It is important to emphasize: the LLM itself does not execute the function. Instead, it identifies the appropriate function, gathers all required parameters, and provides the information in a structured JSON format.
Your server code runs get_weather("Chicago"), calls the weather API, and returns something like {"temp": 72, "condition": "cloudy"}.
The LLM then reasons about the function's output and the original query to return a grounded, well-informed response.
The token-level truth about tool schemas
Here is the part that surprises most people. When you send a tool schema to an LLM, you are not "registering" a function. You are injecting a JSON blob into the model's context window, formatted as a system message. The model is then fine-tuned to output a special token sequence that signals a tool call. This is why the schema counts against your token budget-it is literally part of the prompt.
Under the hood, functions are injected into the system message in a syntax the model has been trained on. This means callable function definitions count against the model's context limit and are billed as input tokens.
The numbers get large fast. In a typical Claude Code session with 20-30 registered MCP tools, the tool schema alone occupies 15-30 KB of context window before a single user message is sent. A more extreme case: every time an LLM agent connects to an MCP server, the full tool catalog is injected into the context window. For a 93-tool GitHub MCP server, that is 55,000 tokens-before the agent does anything. Connect three services (GitHub, Slack, Sentry) and you have burned 143,000 of your 200,000 token window. Seventy-two percent gone. On idle.
Your tool schema is literally part of the prompt. Every tool you attach costs tokens on every single request. Forty tools with verbose schemas equals a fat, permanent tax on your context window. This is the mechanical reason why "show the model fewer tools" works-it is not only about confusing the model, it is about what is physically sitting in context.
The practical implication: if you run into token limits, limit the number of functions loaded up front, shorten descriptions where possible, or use tool search so deferred tools are loaded only when needed.
How the model learned to do this at all
The "tool call format" is an output convention the template asks for. The model was fine-tuned on millions of examples following this template. So when a request needs a tool, the highest-probability continuation is text wrapped in <tool_call> tags matching the schema it just read. The delimiters are special tokens.
A tool call is not a special operation. It is the exact same predict-then-append loop that produced the word "the" a moment earlier. It is just a sequence of tokens that happens to match a structured pattern your runtime knows how to recognize.
Tool calling is learned token behavior baked into the weights. Not a module. Not a planner. Not a parser living inside the model. The model proposes (as text); your code disposes (by executing).
This also explains the difference between providers. OpenAI calls it "function calling" while Anthropic calls it "tool use," but the implementation is nearly identical. Both use JSON schemas to define tools and return structured outputs.
Anthropic's Claude uses a different approach at the token level: it outputs a special XML-like <function_calls> tag and then a JSON block.
The surface API looks similar; the token format differs.
What breaks in production, and why
Because vague descriptions go straight into the model's reasoning, they fail in subtle ways. The JSON schema you define is not just documentation-it is the only thing the model sees. If your descriptions are vague, the model will hallucinate arguments.
Vague descriptions lead to incorrect usage. Good descriptions include expected input types, output format, and failure cases. Think of it as a docstring the model reads to decide whether to call the function.
Most tutorials show one function call. In production, models can emit multiple tool calls in a single turn. If your loop does not handle that, you will silently drop requests and corrupt state.
GPT-4o, Claude 3.5, and Gemini support parallel function calling. If you ask "What's the weather in London and New York?" the LLM can return two function calls in one response. This reduces latency-both calls execute in parallel instead of sequentially.
One more thing that surprises teams: LLMs are nondeterministic by design. There is no guarantee that tool calling works flawlessly all the time. The model chooses which tool to call based on probability, not logic. A tool with a slightly better-worded description will get called more reliably than a technically identical tool with a worse one. That is a writing problem masquerading as an engineering problem.
An AI teammate wired into your Slack or Teams-something like Beagle-runs into the same mechanics: every tool it can reach occupies real tokens, which is exactly why limiting the active tool set per task matters more than maximizing it.
The whole loop, then, is simpler than most explanations make it sound: the model reads your tool catalog as text, decides whether any of those tools would help, writes a structured request for the one it picks, and stops. Your code takes over from there, does the actual work, and hands the result back. Then the model reads that result as text too, and continues. Tokens in, tokens out, the whole way down.