A fairly ordinary MCP setup - four servers covering GitHub, Slack, Sentry, and Grafana - can burn through roughly 55,000 tokens in tool definitions before the model has processed a single word of your actual request. That number comes from Anthropic's own published analysis, and it is the best single fact for understanding what LLM tool calling actually is: not magic, not a plugin, but text injected into a prompt and charged at input-token rates.
Here is what that means in practice, and why it matters for anyone building on top of an AI model.
What LLM tool calling actually is
A large language model is a text prediction engine. It can write code, summarize text, or plan actions - but it cannot execute anything. If you tell it to "send an email," it can generate a perfect JSON body for that email, but it will not actually send it. That is where tools come in: a tool is a bridge between language and action, a real function in your code that the model can ask you to execute.
LLM function calling - also called tool use - is how a large language model talks to the outside world. You describe a set of available functions using JSON Schema. The model receives the user prompt and the tool catalog, decides whether to call a tool, then emits a structured tool call with typed arguments. Your application runs the tool and returns the result. The model uses the result to continue the conversation or call another tool.
The biggest misconception here is that you are "registering" a function with the model. You are not. You are injecting a JSON blob into the model's context window, formatted as a system message. The model is then fine-tuned to output a special token sequence that signals a tool call. This is why the schema counts against your token budget - it is literally part of the prompt.
The model itself does not run the functions and interact with external systems. Instead, it generates parameters for potential function calls. Your application then decides how to handle these parameters, maintaining full control over whether to call the suggested function or take another action.
The request-response loop, step by step
The loop looks like this: the application sends messages plus tool schemas; the model responds with a tool call - for example, { name: "search", args: {"query": "..."} }; the application executes the function and gets a result; the application sends that result back.
The model then decides whether it has enough information to answer or whether it needs to call another tool.
Here is a concrete example. A user asks: "What is the CPU temperature on this server right now?"
| Step | Who does it | What happens |
|---|---|---|
| 1 | App → Model | Sends user message + get_system_stats schema |
| 2 | Model | Returns tool_call: { name: "get_system_stats" } |
| 3 | App | Runs vcgencmd measure_temp, packages JSON result |
| 4 | App → Model | Sends tool result back in conversation |
| 5 | Model | Reads result, generates final text answer |
That is three LLM round trips for one user message if the model needs two separate tool calls. A 2B-parameter model calls tools sequentially - one per round - rather than batching them into a single response. Larger models would likely batch them.
Why the schema is the thing you should care about most
The JSON schema you define is not just documentation - it is the only thing the model sees. If your descriptions are vague, the model will hallucinate arguments. One team's weather function started returning 'pineapple' for location because the parameter description was underspecified.
Each tool is characterized by its name, a description of what it does, and a schema detailing the input parameters. These descriptions are included in the context provided to the model during inference. The model cannot look up your function implementation. It can only read what you wrote in the schema. A one-sentence description of a complex tool is usually not enough.
The schema overhead is also non-trivial at scale. A modest tool definition file with about 300 lines adds approximately 1,300 tokens per query, regardless of whether the tool is actually called.
When you enable tool calling, every tool definition - name, description, parameter schema, examples - gets injected into the model's context on every request. If the model has 60 tools available but only needs two for a given query, you have still paid for 58 irrelevant definitions. Multiply that across thousands of daily API calls and it becomes a substantial, invisible line item.
OpenAI's benchmarks with GPT-5.4 found that retrieving only the definitions relevant to each specific query - rather than loading the entire tool library upfront - demonstrated up to a 47% reduction in total prompt token usage, without compromising response quality or task performance. This pattern, called tool search or deferred loading, is where frontier infrastructure is heading.
What "parallel tool calling" actually means in production
Most tutorials show one function call. In production, models can emit multiple tool calls in a single turn. If your loop does not handle that, you will silently drop requests and corrupt state.
Instead of answering a single question, the model can orchestrate multiple function calls to solve multi-step problems. Planning a trip might involve checking flight availability, booking a hotel, and renting a car through different APIs, all in one conversation.
The provider APIs implement this slightly differently.
OpenAI's function calling API uses a tools parameter in the chat completions API; tool_choice controls whether tool use is automatic, forced, or disabled; it returns a tool_calls array in the assistant message. Anthropic uses a tools parameter with JSON schema definitions and returns tool_use content blocks; tool_choice can force a specific tool.
The main structural difference: OpenAI uses the parameters key, Anthropic uses input_schema. The rest of the structure is identical.
On the accuracy side, even the best models are not perfect. On the MCPVerse agentic benchmark, the best Oracle success rate was GPT-5 at 68.1%, with Claude-4-Sonnet leading on Standard and Max-Scale modes. Under Max-Scale mode - the most demanding configuration - the top model reached only 44.2% average accuracy. Tool calling is reliable enough to ship; it is not reliable enough to run unsupervised on consequential actions.
Beagle in action#engineering, 2:47pmA non-obvious trade-off most teams miss
As new functions or APIs become available, the model can use them without retraining the entire model. That is the genuine upside. The hidden cost is that every tool you add to a shared schema is a tax on every request, whether that tool is relevant or not.
Token cost per tool varies by more than 3× across real servers, depending on schema size, parameter counts, and description length. A raw tool count tells you little about what your definitions actually cost. A schema with five densely-described parameters and three nested objects can cost more tokens than four simpler tools.
The practical implication: start with the minimum set of tools your agent actually needs for a given task, write descriptions as if the reader has no access to your codebase, and test with a weaker model before assuming a frontier model will paper over a bad schema.
LLM tool calling: common questions
What is the difference between tool calling and function calling?
While these terms are often used interchangeably, "tool calling" is the more general and modern term. It refers to a broader set of capabilities that LLMs can use to interact with the outside world - including not just custom functions, but also built-in tools like code interpreters and retrieval mechanisms for accessing data from uploaded files or connected databases. Function calling is the original OpenAI term for the narrower, schema-driven variant.
Does the model actually run the function?
No. The model receives the user prompt and the tool catalog, decides whether a tool is needed, then returns a structured tool call with typed arguments. Your code runs the tool and feeds the result back. The model is always a passenger. Your application is always the driver.
Why do tool schemas cost tokens even when I don't use the tool?
When you send a tool schema to an LLM, you are not registering a function - you are injecting a JSON blob into the model's context window. The entire catalog is in the prompt on every request. The model billing does not distinguish between tokens it read to make a decision and tokens that were irrelevant to that decision.
How many tools can a model handle before accuracy degrades?
Between roughly 20 and 50 tools, based on the guidance the two largest vendors publish.
Some APIs cap the number of tools natively - for example, 128 for GPT-4o and GPT-5, and 512 for Gemini. But hitting the cap is not the real problem; accuracy tends to degrade from selection confusion well before you reach the hard limit.
What happens if the model outputs malformed JSON in a tool call?
Under the hood, the model outputs a JSON string inside the arguments field. The API parses this for you - but if the model outputs malformed JSON, the API returns an error. Some models output unescaped newlines inside strings, which breaks the parser. The fix is strict output validation in your application layer, not trust that the model will always produce clean JSON.