AI inference spending just passed training. Agents are why.

In 2026, global spending on AI inference - $23.3 billion - has surpassed spending on training - $19 billion - for the first time. That crossover sounds like a finance footnote. It is not. It is the clearest signal yet that AI has moved from a research problem to an operational one, and that the economics most teams were assuming when they approved their AI budgets are already wrong.

Why agents, not chatbots, drove the crossover

The short answer: a chatbot handles one request and stops. An agent loops.

EY's analysis of enterprise AI costs found that a simple linear AI workflow - the kind enterprises deployed in 2023 - cost approximately $0.04 per interaction. A modern agentic system involving tool calls, reasoning loops, and iterative execution costs approximately $1.20 per interaction - roughly 30 times higher.

The hardware has not degraded. Goldman Sachs estimates semiconductor providers are delivering inference cost reductions of 60 to 70 percent per year per token. The cost per transaction is rising because the number of tokens per transaction is exploding. Gartner's own analysis confirmed that agentic AI models require between five and thirty times more tokens per task than a standard chatbot query.

So the familiar "tokens keep getting cheaper" chart is accurate. It just does not tell you what your bill will be.

Since February 2026, token consumption by AI agents on OpenRouter jumped from 0.51 trillion to 7.3 trillion tokens - a roughly 14x increase. Human usage grew just 2.8x over the same period.

According to OpenRouter analyst Peter Walker, February 6, 2026 may have been the last day humans consumed more tokens than AI agents.

What's actually inside that 14x

Instead of sending a single prompt and waiting for an answer, autonomous systems can repeatedly call models as they plan, use tools, evaluate results, and continue working toward a goal. Each of those loops burns tokens.

OpenRouter platform data shows that a typical agent task consumes roughly 13 to 15 times more tokens than a human conversation. Agent workloads must carry tool definitions, MCP gateway configurations, skill preamble instructions, and multi-turn reasoning context; a single autonomous programming or research task can consume as many tokens as hundreds of ordinary Q&A sessions.

There is also a less obvious cost driver. The proliferation of reasoning models has amplified this effect. OpenRouter data points to an "overthinking" phenomenon among reasoning models: even for requests that require no deep reasoning, the model spends substantial tokens on internal deliberation before responding.

The good news - partially - is that caching absorbs a lot of it. More than 85 percent of agentic token burn originates in cached prompts, meaning agents spend most of their compute re-reading context they have already been given rather than generating new output.

OpenAI's GPT-5.6 Sol costs $4.00 per million input tokens for a standard context window and only $0.40 for cached inputs - a 90% cost reduction that makes high-frequency agentic usage more scalable.

But caching only helps when you design for it. Most teams have not.

The split market most teams are misreading

Here is the non-obvious part: the AI inference market has split - the floor is collapsing while the ceiling is rising, and confusing the two is what causes most enterprise AI budgets to blow up.

Average inference prices - measured per million tokens - ranged between $1.16 and $1.18 from August 6 to 8, the lowest level recorded this year, per Jefferies citing Silicon Data. Average unit costs have plummeted from $2.04 on May 31 and $1.45 in late July. Budget-tier and mid-range models are getting cheaper fast.

But frontier models tell a different story. OpenAI's current flagship, GPT-5.6 Sol, costs $5 per million input tokens and $30 per million output. Frontier pricing has doubled since January 2026 alone.

This creates a real trap. A team that benchmarked "AI is cheap" in Q1 - when they were running prompt-response chatbots against a mid-tier model - will reach a very different number when they deploy an agent that runs a reasoning loop against a frontier model across a 100K-token context window.

98% of FinOps teams now actively manage AI inference spend, up from 31% just two years ago, per the FinOps Foundation's State of FinOps 2026 survey of 1,192 practitioners. The other 2% are about to get a budget conversation they did not expect.

What teams should actually do about it

The rise of agentic AI amplifies compute intensity through multistep, autonomous execution, making inference the dominant consumption model and positioning AI-optimized IaaS as a critical enabler of enterprise AI strategies. That is Gartner's measured language. In plain terms: inference is now where the money goes, and it needs to be managed like any other production workload.

A few things that move the number in the right direction:

• Route by task complexity. A status-check question does not need a frontier reasoning model. A coding agent debugging a multi-service race condition might. Mid-tier models at $0.40-0.80 per million tokens handle most workplace queries adequately.
• Design for cache hits. Structure your system prompts and tool definitions so they are identical across requests. A prompt that changes on every call cannot be cached; one that stays stable gets the 90% discount.
• Count tokens per workflow, not per seat. Per-seat metrics will hide an agentic cost spike until the invoice arrives. Per-workflow measurement shows you which agents are looping expensively before they accumulate spend.
• Be skeptical of reasoning models on simple tasks. Even for requests that require no deep reasoning, reasoning models spend substantial tokens on internal deliberation before responding. For deterministic lookups, a smaller non-reasoning model is usually faster and cheaper.
• Set agent budgets. Hard token caps per agent run - not per user - are the closest thing to a circuit breaker for runaway loops.

KPMG found that 49% of enterprises cut AI agent rollouts when costs outran value

• which suggests the problem is not awareness, it is the gap between the architecture decisions made when an agent was a demo and the bills that arrive when it runs in production.

The Gartner crossover is not a warning that agents are too expensive to run. It is a warning that the default way most teams are running them is not engineered for cost. Training a model happens once. Inference happens every time someone asks a question - and with agents, every time the agent asks itself a question too.

AI inference costs and agentic AI: common questions

Why did inference spending surpass training spending in 2026?

Gartner attributed the shift to enterprises moving from model development into production-scale deployment. Training a model is a one-time event. Running it in production - especially through multi-step agentic workflows that chain many model calls per task - generates continuous, compounding compute spend that ultimately overtakes the upfront training cost.

Why do AI agents use more tokens than humans doing the same task?

Agents loop. A human types one prompt and reads one reply. An agent plans, calls a tool, reads the result, revises its plan, calls another tool, and repeats - each step burning tokens. A typical agent task consumes roughly 13 to 15 times more tokens than a human conversation, per OpenRouter platform data. Reasoning models add further overhead through internal deliberation even on simple requests.

Are falling token prices canceling out the higher usage?

Not for most teams running agents. The most confusing aspect of the 2026 inference situation is the simultaneous reality of falling unit costs and rising total bills. Per-token inference prices have fallen between 9x and 900x per year for various performance milestones - yet the same enterprises watching token prices collapse are seeing their monthly AI bills multiply.

What is the cheapest way to run a production AI agent?

Structure prompts to maximize cache hits, route simple tasks to mid-tier models, and set hard token budgets per agent run. Five strategies reduce inference costs: quantization (2-4x savings), response caching for repeated queries (3-10x savings), prompt optimization (1.5-3x savings), model routing (2-5x savings), and batch processing (1.3-2x savings). These compound when combined, but only if you build for them from the start.

How fast is agentic token consumption growing?

Agents on the OpenRouter network consumed close to five times as many tokens as human users, and their consumption multiplied roughly fourteenfold since February, according to figures published August 21, 2026 in the a16z newsletter Charts of the Week. That growth rate, if it continues, means inference infrastructure costs will keep climbing even as per-token prices fall.