Token costs dropped 280x in two years. The average enterprise AI budget grew 483% over the same period.
Both facts are true at once, and the gap between them is where most teams get surprised. Gartner published a report yesterday - August 17 - naming this the Inference Paradox: falling unit economics encouraging more complex workflows, which consume far more compute, which drives total costs up faster than the savings come down. The number that stops people: inference costs per agentic workflow will increase more than fivefold through 2028, as applications evolve from simple assistants into autonomous, multistep systems powered by reasoning.
That is not a prediction about model prices. It is a prediction about how teams use models. And it is already visible in real bills today.
Why the per-task multiplier is so large
The unit of AI cost has changed. A chatbot call is one model call. An agent is a loop.
A simple chatbot query triggers one inference call, but an agentic workflow - where an agent calls external tools, verifies outputs, and self-corrects - can trigger 10 to 20 model calls for a single user-initiated task. Each of those calls carries input tokens (often including the full system prompt and accumulated context), output tokens, and any retrieval overhead. The sticker price per million tokens tells you nothing about what a completed task actually costs.
EY quantified the gap in concrete terms: in 2023, a simple linear workflow costs $0.04 per interaction; in 2026, a more complex orchestrated system involving tools, reasoning, and iterative loops costs $1.20 per interaction - about 30 times higher.
That 30x multiplier does not come from model price increases. It comes from workflow shape: planning steps, tool calls, context retrieval, validation, retries. Token prices are becoming a less useful proxy for the economics of an AI product. The more consequential question is how much intelligence a system consumes to complete useful work - and that puts architecture, model choice, and workflow design closer to the economics of AI deployment.
The scale problem compounds the per-task problem
Even if you manage the per-task cost tightly, the volume math is brutal.
Goldman Sachs research expects token consumption generated by AI agents to increase 24 times between 2026 and 2030, eventually reaching 120 quadrillion tokens per month. That projection assumes continued model adoption. It does not assume that each individual agent gets more expensive - it just assumes more tasks get routed to agents.
The two pressures stack: more tasks going to agents, each task consuming more tokens than a chatbot call. Semiconductor providers are achieving inference-cost reductions of around 60 to 70 percent annually per token. These trends appear contradictory but help explain the Inference Paradox: even if the cost of processing an individual token falls 60-70 percent per year, total AI expenditure can continue increasing if agentic applications generate vastly more tokens and computational steps.
The market data matches. Worldwide spending on AI-optimized infrastructure-as-a-service will reach $42 billion in 2026 - nearly double the $21.5 billion recorded in 2025 - and of that total, $23.3 billion will flow toward inference workloads.
For the first time in the history of the AI industry, enterprises are spending more money running AI models than building them.
What teams building on agents should actually do
Gartner's framing is useful for boardroom conversations, but what changes at the team level?
There is no reliable, economical one-size-fits-all model on the horizon. Producing competitive AI products will require developing and maintaining complex multimodel ecosystems. That is less a product pitch than an operational description of what high-functioning teams already do: they do not route every task to the frontier model.
A few concrete practices that move the number:
Route by task complexity, not by capability ceiling. Fine-tuned or smaller models cut inference cost per task by 40 to 70 percent for narrow, repetitive workflows. If a step in your agent loop is classifying sentiment or extracting a date, it does not need GPT-5.6.
Use prompt caching on every multi-turn session. Prompt caching is the single most impactful cost reduction technique for agentic workflows, and most teams are not using it - that is just money left on the table.
A 10-turn session where 80% of tokens are cacheable context costs roughly 30-40% of what it would cost without caching.
Count tool calls, not just tokens. Every tool call an agent makes - whether a database query, a web search, or a code execution step - adds latency and token cost on top of the base model call. Complex agent chains with five or more tool calls per task can multiply your effective cost per completed task well beyond the sticker price.
Set hard per-workflow token budgets before production. Enterprises that scaled past the pilot phase discovered the multiplier only after their production bills arrived. The pilot economics bore no relationship to the production economics of multi-step agentic loops running thousands of times per day.
Beagle in action#eng-ops, Thursday afternoonThe routing discipline matters even for teams not running their own models. Gartner believes falling model prices are tempting users to build more complex workflows, whose greater token consumption can outweigh those savings and drive up overall inference costs. A team that upgrades from a budget model to a frontier reasoning model for every step does not save money when the frontier model's context window encourages longer chains. It spends more.
The non-obvious version of this insight: the Inference Paradox is partly a design problem, not just a pricing problem. Agents that reason excessively - asking themselves sub-questions, re-checking already-confirmed facts, looping on ambiguous instructions - burn tokens on work that does not improve the output. Gartner's Will Sommer describes it this way: "Where a simple chatbot must read and interpret a query and quickly respond with a probabilistically reasonable answer, an AI agent must constantly reason, negotiate, and question itself." All of these responsibilities add up. Tightening the reasoning budget in system prompts and adding exit conditions to loops is free. It just requires someone to look at what the agent is actually doing on each call.
The teams that will manage this well are not necessarily the ones with the most compute budget. They are the ones that treat inference as an engineering variable rather than a line item - routing, caching, budgeting per workflow, and reviewing which steps actually require a frontier model call. Tools like Beagle, operating inside Slack with a draft-and-approve model, help contain cost by scoping each action tightly - one question, one retrieval, one post - rather than spinning up long autonomous chains that generate tokens in the background.
Agentic AI inference costs: common questions
What is the Inference Paradox?
The Inference Paradox is Gartner's term for the dynamic where falling token prices encourage more complex AI applications, which consume more tokens and computation per task, causing total inference spending to rise even as the unit cost drops. Better economics make agents financially viable; viable agents generate more inference; total costs climb.
Why do agentic workflows cost so much more than chatbots?
A chatbot makes one model call per user message. An agent planning, calling tools, checking results, retrying on errors, and validating outputs makes 10-20 calls per task. Each call includes input tokens - often repeating system prompts and accumulated context - so total token consumption per completed task is 5-30x higher than a single-turn interaction, per Gartner's analysis.
How much does a single agentic workflow interaction cost in 2026?
EY's 2026 analysis puts a well-orchestrated agentic workflow at approximately $1.20 per interaction, compared to $0.04 for a simple 2023 chatbot interaction. The figure includes tool calls, retrieval, reasoning loops, and retries. Simpler agents with fewer steps cost less; complex multi-agent pipelines cost significantly more.
What is the single most effective way to cut agentic AI inference costs?
Prompt caching. For multi-turn agent sessions where the system prompt and retrieved context are re-sent on every call, caching those tokens at roughly 10% of the standard input price cuts the total session cost by 30-40%. Most teams are not doing it. The second lever is model routing: using smaller or fine-tuned models for narrow steps in a workflow rather than routing every call to the frontier model.
Will inference costs fall as models get more efficient?
Per-token costs are projected to fall 60-70% annually as hardware improves. But Gartner's forecast is that per-workflow costs will still rise more than fivefold by 2028, because teams are building more complex workflows faster than the efficiency gains arrive. Goldman Sachs projects a 24x increase in total token consumption from AI agents between 2026 and 2030. Lower prices plus much higher volume equals higher total bills.