Your finance lead asks the Slack AI assistant to pull Q2 churn by segment. The model call costs fractions of a cent. But the agent behind that answer read the Notion doc, queried the data warehouse twice, retried a failed tool call, and re-sent its full context window at each step. By the time the answer posts, you've burned forty times the tokens the question implied.
That gap - between what a token costs and what a task costs - is the most important thing happening in AI economics right now.
Average inference prices hit $1.16-$1.18 per million tokens on August 6-8, the lowest recorded this year according to Jefferies, citing Silicon Data. That's down from $2.04 on May 31 and $1.45 in late July.
GPT-4-class inference has fallen from $30 to under $0.50 per million tokens - roughly 95% in two years. By the headline numbers, AI has never been cheaper.
And yet. Token prices fell roughly 80% between mid-2023 and early 2026. Enterprise AI spending rose roughly 320% in the same period. The savings are real. They're just not landing where teams expect.
Why cheaper tokens don't mean cheaper AI
The cost-per-token metric made sense when teams were running chatbots. One user message in, one completion out - you could model it on a spreadsheet. Agents work differently.
A simple chatbot query triggers one inference call, but an agentic workflow - where an agent calls external tools, verifies outputs, and self-corrects - can trigger 10 to 20 model calls for a single user-initiated task. That changes the relevant unit: it's no longer cost per prompt, but cost per completed task.
An AI agent planning a multi-step task sends its full accumulated context - system prompt, conversation history, and all intermediate results - to the model at every step. By step 20, the agent has paid for the original context 20 times.
Agentic workflows consume 5 to 30 times more tokens per task than a standard chatbot query.
This is why Uber's CTO described being back at the drawing board on AI budget. Claude Code adoption jumped from 32% to 84% of Uber's 5,000-engineer org between December 2025 and March 2026. By April, the entire annual AI budget was gone. Monthly API costs per engineer were running between $500 and $2,000. Uber is not an outlier - it's just unusually candid.
What the cost-per-task number actually looks like
A worked example grounds this better than a multiplier range. A five-step support agent that sends about 3,000 input and 500 output tokens per step spends roughly 15,000 input and 2,500 output tokens per task. At the new $1.16/M index price, that's under two cents. Sounds fine - until you add retries, context growth across turns, and tool-call overhead. Actual task-relevant output is often only 5-15% of total tokens consumed. Everything else is context overhead.
A typical PR from a coding agent involves 3-5 agentic tasks. At API rates that ranges from $0.09 (Gemini 2.5 Pro, 3 tasks) to $0.65 (GPT-5 high reasoning, 5 tasks). Real-world costs are 3-5x higher due to context overhead, putting the realistic range at $0.27-$3.25 per PR depending on model choice and codebase complexity.
The table below shows how the multiplier changes the picture for three common team workflows, using the current $1.16/M index as the input rate and $4/M as a rough output rate:
| Workflow | Chatbot tokens/task | Agentic tokens/task | Rough cost/task |
|---|---|---|---|
| Single-turn Q&A | ~500 in / 200 out | - | ~$0.001 |
| Support triage (5 steps) | - | ~15K in / 2.5K out | ~$0.03-0.05 |
| Coding agent (one PR, 4 tasks) | - | ~60K in / 8K out | ~$0.10-0.30 (listed rate) |
| Coding agent with context overhead | - | ~180K-300K in | ~$0.30-$3.25 real-world |
Token prices fell 67% in 2026, yet 73% of enterprises exceeded AI budgets
- with retry loops and background inference cited as the primary drivers. The gap between listed price and actual bill is not a billing error. It's architecture.
Beagle in action#engineering, 2:47pmThree controls that actually move the number
The teams managing this well are not just shopping for cheaper models - they're changing how they deploy them. Three levers that show up consistently:
Route by task shape, not by preference. Running a cheaper model for planning and repo scanning, and a more capable one only for the final generation step, beats one-model-everywhere on both cost and quality. A small model that fails and retries costs more than a capable model that gets it in one pass.
Budget at the step level, not the session level. Map every agent loop and identify the token multiplier for each workflow. Any agentic pipeline consuming more than 10x tokens per user-initiated task needs architectural review. Session-level caps don't catch runaway loops - step-level budgets do.
Watch context growth, not just volume. AI agent token sprawl is driven by retry loops that re-send growing context on every correction cycle, background agents that run continuously without human-triggered requests, and context snowballing where every agent step re-reads the full interaction history. Truncating or summarizing prior steps before re-injection is one of the highest-leverage interventions, and most frameworks don't do it by default.
The broader point: the next big savings will likely come from better use, more aggressive compression where quality holds, routing by task difficulty, and improved serving designs that lift throughput without lifting error rates. The easy price cuts - model provider competition, hardware efficiency gains - are mostly in the market already.
A teammate like Beagle, operating inside Slack with a draft-and-approve model, keeps each interaction bounded to a single task rather than leaving a session open across the afternoon - which means the token clock stops between queries rather than accumulating in the background.
AI inference costs: common questions
What does AI inference cost per million tokens right now?
Average inference prices hit $1.16-$1.18 per million tokens in early August 2026, the lowest level recorded this year, per Jefferies citing Silicon Data. That's a blended index across API providers and hosted open-weight platforms. Frontier closed models run higher; quantized open-weight models hosted on commodity infrastructure can go lower.
Why is my AI bill going up if token prices are falling?
Organizations didn't just buy more tokens at a lower price; they deployed agents that each consume far more tokens per task, and they deployed many more agents across many more workflows. The agentic multiplier - 5 to 30x tokens per task versus a chatbot - more than offsets the price decline for most teams past the pilot stage.
What is the actual cost per task for an AI agent?
An AI agent resolves a task for about $0.62 versus $7.40 for a human in 2026 , though that figure shifts significantly with model choice, retry rate, and context overhead. Real-world coding agent costs range from $0.27 to $3.25 per PR depending on model and codebase complexity - 3-5x higher than listed API rates suggest.
Should teams self-host to cut inference costs?
Self-hosting breaks even at roughly 50-100 million tokens per month for most 70B-class models when you factor in engineering overhead. Below 20 million tokens per month, managed APIs win on total cost including ops. Above 100 million tokens per month, self-hosting almost always wins on unit economics. For most teams the answer is managed APIs until usage is large enough to justify the operational load.
Which open-weight models are worth considering for cost reduction?
Open-weight models now compete directly with flagship closed systems on quality, while delivering speed and deployment flexibility that proprietary APIs match only at higher cost.
Reasoning-heavy tasks still favor closed models like Claude Opus and GPT-5 Pro by 3-8 percentage points on benchmarks like GPQA Diamond. The coding gap has largely closed: DeepSeek V3.2 and MiniMax M2.7 now sit within striking distance of Opus on real-world coding workloads, with MiniMax M2.7 costing roughly 50x less per million output tokens.