Token prices dropped more than 90% since 2023, yet spending on large language models has doubled since late last year , according to the Silicon Data Token Expenditure Index. If that sentence describes your team's last two quarters, you are not mismanaging your budget. You are experiencing a well-documented 19th-century economic pattern, applied to GPUs.
The number that frames everything: while token costs were halved from December 2024 to December 2025, tokens consumed grew by 450% in the same period , according to a Bain analysis. Per-unit price and total spend move in opposite directions. This is not going to resolve itself as models get cheaper.
Why cheaper tokens produce bigger bills
This is the Jevons paradox, an economic pattern first observed with coal: when a resource becomes more efficient to use, consumption rises so much that total usage increases even as unit cost drops.
Microsoft CEO Satya Nadella acknowledged the dynamic directly when DeepSeek released its low-cost model: "Jevons Paradox strikes again."
Three forces push total token consumption up at the same time per-token price falls:
- New workloads become viable. Use cases that were too expensive to justify last year are worth running once tokens are cheap, so more workflows move into production.
- Existing workloads run more often. When a workflow works and is cheap, teams use it far more, and usage scales with success.
- Models consume more tokens per task. Reasoning models and agents generate and process far more tokens per request than older chatbots, looping and re-checking to reach an answer.
The frontier tier barely deflates. The newest flagship models tend to launch near the price of the previous flagship, so the most ambitious workloads see little benefit from falling prices.
The pattern compounds because teams keep upgrading. Bain's analysts found that companies feel compelled to upgrade their AI models to take advantage of improved technology, rather than sticking with their current model and pocketing the savings. You bought cheaper tokens, then you bought better tokens.
The agentic multiplier your budget model probably missed
A chat-style AI feature - a user asks a question, the model replies - is easy to cost. One call in, one call out. Multiply expected volume by your per-token rate, done.
Agentic workflows break this math. Agentic workflows trigger 10 to 20 model calls per user task, and retrieval-augmented generation (RAG) expands context windows by 3 to 5 times. An approval workflow that looks like "one task" might produce 15 model calls under the hood: planning, tool selection, tool execution, error retry, summary, and a final answer. Each step burns tokens. Each retry doubles the spend on that request.
One fintech startup's fraud detection agent cost $5K/month with 50 users in Q3 2025. By January 2026, with 500 active users, they were burning $15K/month. The per-token price had fallen in that window. The bill tripled anyway. By 700-1,000 concurrent users, their unit economics inverted. They killed the project.
The failure mode here is architectural, not accidental. Most teams measure inference cost after launch, when the architecture is locked. Measure token consumption on day 1 during development.
Beagle in action#eng-ops, budget review, day 1 of a new agent buildWhat self-hosting actually breaks even at
The self-hosting calculation is frequently cited and frequently wrong in both directions. Here is the math as it actually stands.
At 500M tokens/month with a 70B FP16 model on an 8×H100 pod, compute costs run roughly $1.90/M tokens, or about $950/month. The same volume via OpenAI GPT-4o at $10.00/M output tokens is $5,000/month.
That looks decisive. But the breakeven depends entirely on utilization. Self-hosted breakeven requires 50%+ GPU utilization for 7B models and 10%+ for 13B models. A team running a 70B model at 20% utilization is paying for idle capacity - and the per-effective-token cost quietly rises back above the API rate.
| Scenario | Monthly token volume | Approach | Est. monthly cost |
|---|---|---|---|
| Small team, variable usage | <100M tokens | API (mid-tier model) | ~$300-800 |
| Growing product, steady load | 500M tokens | API (GPT-4o class) | ~$5,000 |
| Growing product, steady load | 500M tokens | Self-hosted 70B, >50% utilization | ~$950-1,200 |
| High-volume, unpredictable spikes | 1B+ tokens | Hybrid routing | Varies by mix |
The hybrid routing approach deserves more attention than it gets. Routing 70% of traffic to a $0.10/million token model and 30% to a $3/million token model yields a blended cost of $0.97/million - dramatically lower than using the expensive model for everything. The catch is that it requires upfront classification logic to decide which calls need the expensive model. Most teams skip that work and route everything to the same endpoint. That is where the budget goes.
Where the macro trend is actually going
Token prices have experienced dramatic declines over the past three years. Using GPT-4-level capabilities as a benchmark, inference prices fell from approximately $60 per million output tokens in early 2023 to less than $1.5 per million output tokens in early 2025 - a more than 40-fold reduction.
That trend continues. LLM inference costs declined 10× annually - faster than PC compute or dotcom bandwidth. GPT-4 equivalent performance now costs $0.40/million tokens versus $20 in late 2022.
The counterintuitive implication: cheaper inference is not primarily a cost-reduction story for most teams. It is a capability-expansion story. When GPT-4 cost $60 per million output tokens, organizations were careful. They deployed AI only where the return was unambiguous. As prices fell, economically impossible use cases became viable, then affordable, then ubiquitous.
Goldman Sachs Research estimates that by 2030, agentic AI will multiply token consumption twelve times on the consumer side alone, projecting to 120 quadrillion tokens processed per month across enterprise and consumer adoption.
That number is almost certainly an underestimate given the Jevons curve so far. What it tells a team building AI features today: every time you hit a cost-efficiency milestone, a new tier of workloads opens up. Budget for the workloads that will become viable at the next price point, not just the ones you are running now. A teammate like Beagle sitting in Slack or Teams already produces this kind of per-workflow token log as a byproduct of operation - which turns out to be genuinely useful when the renewal conversation arrives.
AI inference cost at work: common questions
Why did my AI bill go up if token prices fell?
Token prices fell, but your team ran more tasks, adopted more capable (and token-hungry) models, and expanded into agentic workflows that call the model 10-20 times per user request. The volume growth typically outpaces the unit-price decline. This is the Jevons paradox: efficiency gains expand total consumption more than they reduce per-unit cost.
At what volume does self-hosting a model make sense?
The crossover is roughly 500M tokens per month for a 70B model, assuming above 50% GPU utilization. Below that threshold, idle compute raises your effective per-token cost above what API providers charge. Smaller models (7B-13B) break even at 10%+ utilization, making them viable at much lower volumes if the task quality is acceptable.
What is the cheapest way to cut inference cost without changing models?
Prompt routing: classify each request by complexity and route simple calls to a cheap small model. Routing 70% of traffic to a $0.10/M token model and 30% to a $3/M model yields a blended rate under $1/M. The investment is a classifier and one week of production logging to establish your traffic mix.
How much do reasoning models cost compared to standard models?
Output tokens are already 4-5× more expensive than input tokens to generate. Reasoning models compound this because they produce long internal chains of thought before the final answer, which can multiply output token counts by 5-20×. A task that costs $0.01 on a standard model can cost $0.20 on a heavy reasoning run - which matters a lot in agentic loops that retry.
Should I model AI costs by token or by task?
By task. Per-token pricing is how vendors bill, but it obscures total cost. A task might involve planning, retrieval, tool execution, and summarization - each a separate model call. Track cost per completed task end-to-end, including retries and context reloads. That is the number that determines whether the feature has sustainable unit economics at scale.