Via businesssearch.org
The telecom giant is routing routine AI queries to cheaper open-source models, processing 45 billion tokens daily while barely denting output quality.
AT&T found a way to slash its AI coding costs by more than half, and the trick is almost disappointingly simple: stop using the expensive model when a cheaper one works just as well.
The telecom giant implemented model routing technology through LiteLLM that redirects routine employee queries, particularly coding-related ones, toward lower-cost open-source models. The result was a 56% reduction in AI coding costs with only a 2% decline in performance quality. For a company processing roughly 45 billion tokens daily through its internal “Ask AT&T” platform, those savings add up fast.
The routing playbook #
The concept behind AT&T’s approach is what the industry calls model routing, essentially a traffic cop for AI queries. Simple questions get sent to lightweight, inexpensive models. Complex tasks still go to premium options from OpenAI and Anthropic.
AT&T VP Mark Austin noted that open-source models are narrowing the performance gap with their proprietary counterparts, with a difference of only 6-10 months in capabilities.
Currently, open models handle about 40% of employee AI queries at AT&T. The company is targeting 60-70% in the near term. The models doing the heavy lifting on the open-source side include Nvidia’s Nemotron, Meta’s Llama, and Google’s Gemma, with AT&T actively evaluating additional alternatives.
Telecom-tuned models push savings even further #
AT&T didn’t stop at generic model routing. Between February and July 2026, the company experimented with custom telecom-tuned models designed for industry-specific tasks. Those experiments delivered up to 90% savings in inference costs at scale.
The telecom-specific models are purpose-built for the kinds of queries AT&T employees actually make: network troubleshooting, customer service scripts, internal documentation lookups.
What this means for enterprise AI spending #
Goldman Sachs has flagged this trend as potentially advantageous for Big Tech firms, suggesting AT&T’s task-specific routing approach could serve as a template for cost management in enterprise AI deployment.
For other enterprises considering a similar move, the 45-billion-token-per-day figure is instructive. AT&T isn’t running a small pilot. This is production-scale deployment across a workforce of roughly 150,000 employees. Disclosure: This article was edited by Editorial Team. For more information on how we create and review content, see our