Cisco's CFO has an AI agent that benchmarks the company's financials against competitors' earnings calls, drafts most of the MD&A section in quarterly SEC filings, and routes each of those jobs to whichever model is cheapest for that specific task. Starting this month, every one of the company's roughly 90,000 employees gets that same setup. Cisco will give each employee a personalized AI agent starting in its new fiscal year at the end of July 2026, with the system routing each task to whichever model is most cost-efficient rather than defaulting to frontier models.
That last clause is the part most teams gloss over. The headline is the scale. The substance is the architecture.
Why "don't burn tokens on frontier models" is the whole strategy
In general, AI agents use far more tokens than standard chats because they continuously plan, call tools, and process intermediate steps. While a typical chat may use a few thousand tokens, complex agent tasks can consume hundreds of thousands to millions in a single run. Multiply that across 90,000 people and you have a bill that scales as fast as adoption - which is exactly the trap Cisco's CFO is describing when he says the system is designed not to waste tokens on models that are more powerful than the job requires.
LinkedIn posts from founders and LLM researchers throughout late 2025 consistently report the same pattern: agentic systems cost 5-25x more per task than non-agentic alternatives. The reason is straightforward: reasoning steps multiply token consumption. A simple classification task that costs $0.01 in a chat interface costs $0.10-0.50 as an agentic workflow - tool calls, verification, re-checking, context loading.
So when Cisco says it built its own stack to query different models based on the use case, that is not an engineering preference. It is a financial survival decision at scale.
The inference cost problem no one models early enough
The cost structure of a frontier model API makes sense for a prototype. Frontier models run $2-$15 per million input tokens and $10-$75 per million output tokens. At that price, a small pilot with 20 internal users looks fine. The unit economics collapse when you multiply by 200 users, each triggering a multi-step agentic loop several times a day.
Deloitte's Q4 2025 report found enterprise teams discovering "tens of millions" in monthly inference bills from multi-agent deployments. These aren't pilot costs; these are production systems that shipped with inference cost modeling that didn't account for the real usage pattern. Teams are forced to choose: shut down the agent, or rebuild the architecture to strip cost.
Cisco's answer to this is to build the routing layer in from the start. Patterson said the architecture avoids burning tokens on frontier models when a lighter model will do, and much of the infrastructure runs on-premises, which the company says gives it more control over both cost and data security.
On-premises is a meaningful choice here, and Gartner's framing of where the industry is heading supports it. Expensive inference of frontier-level models must be heavily gated and reserved exclusively for high-margin, complex reasoning tasks.
Routine, high-frequency tasks must be routed to more efficient small and domain-specific language models, which perform better than generic solutions at a fraction of the cost when aligned to specialized workflows.
That is exactly the architecture Cisco described. It is also the architecture that makes enterprise AI agent rollout actually viable at headcount scale.
What this means if you're planning an agent rollout for your team
The Cisco rollout is useful not because most teams have 90,000 seats, but because the core problem is identical whether you're deploying to 90 people or 90,000. The agent architecture question - what runs on which model, for which task, at what trigger - is the one you have to answer before you commit to the stack, not after the bills arrive.
Per-token pricing is falling, which is a positive trend. But total token consumption is rising faster than prices decline, because today's advanced models reason, loop, and chain workflows in ways that burn far more tokens per request than earlier systems.
That dynamic does not fix itself. The falling price per token is real - Stanford's 2025 AI Index shows inference costs dropped from $20 to $0.07 per million tokens for comparable capability over roughly two years - but the token volume from agentic loops rises just as fast, so the net bill can still climb quarter over quarter even while the unit cost drops.
The three decisions that determine whether an enterprise AI agent rollout stays solvent:
- Which tasks go to a frontier model, and which go to a smaller on-premises or self-hosted model
- Whether the agent retries on failure (each retry is more tokens), and how many times
- Whether context windows reload in full on every step, or selectively
None of these are AI decisions. They are product and infrastructure decisions, and they need to be made early.
The model-routing pattern spreading past Cisco
Cisco is not an isolated case. Gartner projects that 40% of enterprise applications will have embedded agents by the end of the year, up from less than 5% in 2025. The teams getting there are not necessarily the ones with the biggest budgets - they're the ones treating inference cost as a first-class architectural concern.
The formal argument for smaller models in agent loops got a spine in June 2025, when NVIDIA researchers published "Small Language Models are the Future of Agentic AI." Those researchers argue that because the task space inside an agent loop is narrow and repeating, a specialist small model frequently matches or beats a generalist giant on the work that actually runs.
That thesis has supply to match it now. Microsoft's Phi-4-mini (3.8B parameters, MIT license, released February 2025) runs in roughly 3 GB of VRAM at Q4 with a 128K context window, and matches Llama 3.1 8B on the full MMLU benchmark using about half the memory.
Alibaba's Qwen3-4B scores 83.7 on MMLU-Redux, beating models up to twice its size, with strong native tool-calling out of the box. These are credible options for the high-frequency, narrow steps inside an agentic loop - status lookups, routing decisions, classification, formatting - without touching a frontier model at all.
The practical read for a team standing up something like a Slack-native agent workflow: map your agent's steps before you write any code. For each step, ask whether it requires open-ended reasoning or whether it is just pattern-matching and tool dispatch. The latter category does not need GPT-class capability. It needs reliability, low latency, and a bill that does not quadruple when the team starts actually using the thing.
Beagle in action#ops-team, 11:02amCisco's CFO framed the architecture choice plainly: "It's not going to burn a whole bunch of tokens with frontier models." That sentence is doing more strategic work than it sounds like. It is the difference between an agent rollout that compounds in value and one that compounds in cost.