Uber's engineering org burned its entire 2026 AI coding budget in four months. Claude Code adoption jumped from 32% to 84% of its 5,000-person engineering org between December 2025 and March 2026, and by April the annual allocation was gone. Monthly API costs per engineer were running between $500 and $2,000. Per-token prices didn't cause this. The unit of cost did.
The canonical AI pricing conversation has been about dollars per million tokens, and that number has genuinely collapsed. Blended token prices fell roughly 67% year over year - from $18.40 to $6.07 per million tokens between Q1 2025 and Q1 2026. But 73% of enterprises exceeded their original AI cost projections last fiscal year. The contradiction is not a paradox. The workloads changed underneath the prices.
Why agents cost more per task, not per token
A simple chatbot query triggers one inference call, but an agentic workflow - where a model calls external tools, verifies outputs, and self-corrects - can trigger 10 to 20 model calls for a single user-initiated task. That changes the business model: the relevant unit is no longer cost per prompt, but cost per completed task.
The mechanics are worth understanding precisely. A reasoning agent doesn't just send a prompt and receive a completion. It sends the full accumulated context - including system prompt and conversation history - to the model at every step. By step 20 of a multi-step task, the agent is paying for the same context 20 times over.
A 2024-era chatbot exchange might consume 2,000 tokens. A 2026 agentic workflow that plans, calls tools, reflects on results, retries, and self-verifies can consume 50,000 or more for a single business transaction. If unit price falls 10x while token consumption per task rises 25x, your bill goes up.
That arithmetic is what Uber ran into. It is also what most team budgets never modeled.
The budget model that broke
Most enterprise AI budgets were built on per-seat or per-subscription logic. That logic made sense when AI meant a SaaS tool with a fixed monthly price. When API access replaced subscriptions and agents replaced chatbots, the logic broke.
The numbers show just how fast this shifted. Per-token prices have fallen roughly 80% since mid-2023; total enterprise AI spending grew 483% from 2024 to 2026 anyway, because agents consume tokens in fundamentally different ways than chatbots do.
Consider the concrete math. An enterprise running an AI assistant that handles 100,000 agentic tasks a day, each consuming a conservative 5,000 tokens (not 50,000), runs through 500 million tokens daily. At $1 per million tokens - a reasonable mid-tier rate - that is $500 per day, or roughly $182,000 per year for a single workflow. Run the same estimate on GPT-4 class pricing at $10 per million output tokens, and one modestly sized agentic pipeline crosses $1M annually before you add retrieval overhead.
RAG systems - the backbone of most enterprise knowledge agents - add 2,000-10,000 tokens per query depending on retrieval chunk size and top-k settings. Teams that deploy RAG without tuning these parameters routinely discover their retrieval overhead exceeds their reasoning costs by 3:1.
The framing the industry handed teams - "AI is getting cheaper" - was true at the token level and misleading at the task level. A 95% AI inference cost reduction has not lowered most companies' AI bills. It has raised them. This is the Jevons paradox - when a resource gets cheaper, you use far more of it.
What model routing actually changes
The real lever is not switching to a cheaper model across the board. A small model that retries often, calls tools poorly, or writes unusable answers can waste more money than a larger model that completes the task in one pass. The goal is routing the right step to the right tier.
The winning pattern is not "small instead of large." It is LLM-SLM orchestration: route each step to the smallest model that provably clears your accuracy bar, with a calibrated escalation path.
The research on this is encouraging. Production deployments demonstrate significant savings: RouteLLM achieves 85% cost reduction on MT Bench while maintaining 95% GPT-4 performance.
At 100,000 queries per day, routing simple requests to cheaper models while reserving frontier capacity for complex tasks can yield annualized savings exceeding $150,000.
There is an honest caveat here. The RouteLLM results apply to open-ended queries - the quality metric is "which response do humans prefer?" Production agent tasks are fundamentally different: a tool call that returns malformed JSON is not a "lower quality answer" - it is a broken output that causes the downstream agent to fail or retry.
Teams applying RouteLLM to production agent systems should validate routing accuracy on their specific task types, not rely solely on MT-Bench numbers.
A few routing principles that hold in practice:
Classify before you call. Intent classification - even a lightweight BERT classifier - can identify the ~70% of requests that a smaller model handles cleanly. If a routing classifier can correctly identify the 70% of queries adequately served by the cheap model, it captures most of the cheap-model savings while avoiding quality degradation on the complex 30%.
Watch context length, not just model tier. A 128K-token context filled to 80% capacity costs 4-6x more per turn than the same task handled in a 16K context window. Trimming context often saves more than switching models.
Measure cost per completed task, not cost per token. An agent that takes 3 attempts at $0.001/token costs more than one that takes 1 attempt at $0.003/token.
Add an eval gate before you route down. Never route down model tier without eval evidence. Cost saved on paper means nothing if support ticket volume doubles.
A teammate like Beagle, which lives in Slack and drafts before posting, has a natural cost shape here: one well-scoped call per trigger, with a human approval step that catches retries before they multiply.
Beagle in action#engineering, 11:03amAgentic AI inference cost: common questions
What makes agentic AI cost so much more than a chatbot?
Agents re-send their full accumulated context to the model on every tool call. A chatbot exchange might consume 2,000 tokens. An agent handling the same user goal - planning, calling tools, checking outputs, retrying - can reach 50,000 tokens or more for a single task, with input tokens dominating the bill rather than output tokens.
Why are enterprise AI bills rising when per-token prices are falling?
Despite token prices falling, total enterprise AI spend has risen by 320% over the same period. More AI usage at lower unit costs still produces higher total bills when adoption grows faster than prices drop. The shift from chatbots to agents multiplied per-task consumption faster than per-token prices declined.
What is LLM model routing and does it actually save money in production?
Model routing sends each request to the cheapest model that can handle it. In controlled benchmarks the savings are real - up to 85% cost reduction while maintaining 95% of frontier model quality. In production agentic systems, results depend heavily on whether the router is validated against your specific task types, since a routing failure in an agent pipeline causes retries that erase the savings.
How do I estimate the per-task cost of an agentic workflow?
Multiply the average number of model calls per completed task by the average tokens per call (input tokens grow with each step because context accumulates). Add retrieval tokens if you use RAG - typically 2,000-10,000 per query. Then multiply by your model's output token rate, which is typically 4-5x the input rate. Most teams find the honest per-task figure is 10-20x their initial estimate.
Is self-hosting a cheaper option for high-volume agentic workloads?
At one million monthly conversations, hosted frontier LLM costs run $15,000 to $75,000 per month. A fine-tuned small language model deployed inside your own infrastructure costs $150 to $800 per month at that same volume. That is not a rounding error - it is a 20x to 100x cost difference for tasks where the small model performs equally well. The catch is that self-hosting works when your tasks are narrow and repeatable; broad reasoning tasks still require frontier capability.