AI agents can handle complex tasks autonomously, but not every job demands the same horsepower. That's where model routing comes in.
The concept is simple: model routing matches AI models to the complexity of the task, saving users and enterprises money by calling on lighter, cheaper models for most jobs and reserving the more costly ones for when they are needed. Snowflake just became the latest company to unveil its own dynamic model routing solution within Cortex AI Gateway and its flagship AI products, Snowflake CoCo and Snowflake CoWork.
“Enterprise AI is moving toward a world where companies can draw on the best available intelligence for each task without having to manage the model landscape themselves," Baris Gultekin, VP of AI at Snowflake, told The Deep View.
"Snowflake is building toward that future by helping customers benefit from rapid innovation across the model ecosystem without adding more operational burden. Over time, that flexibility will make it easier for enterprises to adopt AI more broadly, reduce costs, and turn model advancements into sustained business value."
Another major advantage of model routing, as highlighted in the blog post, is that it lets enterprises skip rebuilding agents or apps for different models. In internal evaluations, Snowflake said agents using dynamic model routing with Cortex AI Gateway to build a dbt pipeline saw up to 3x greater token efficiency than a frontier-model-only path, without compromising quality. In another test, engineers completed the same number of pull requests with 25 percent greater token efficiency.
NVIDIA just released its own routing solution last week: NeMo Switchyard. It is an open-source library for smart routing within popular agent tools, allowing enterprises to build routers to meet their needs and take advantage of the same benefits as above: lower costs that don't compromise performance or require rebuilding apps.
In a conversation with The Deep View, Kari Briski, VP of generative AI software for enterprise at NVIDIA, also highlighted the performance benefits model routing can offer by rerouting tasks to domain- or task-specific models, in addition to cost-efficient ones.
"Simple tasks or even niche tasks need to be domain-specific, and so that results in not just faster responses because you're sending to maybe smaller, more efficient models, but then greater token efficiency, and then even in some cases a higher accuracy based on your domain," said Briski.
Our Deeper View #
Underpinning the development of model routing is a broader industry trend towards efficiency and cost reduction, which is imperative as costs continue to skyrocket. A Gartner report published this week predicts that AI inference costs per agentic workflow will increase more than fivefold through 2028. When asked if these rising costs would soften AI demand, Scott Bickley, Advisory Fellow at Info-Tech Research Group, told The Deep View that efficiency would be the focus: "The initial response will be for a focus on efficiency and more intelligent use and consumption via an array of techniques such as fine-tuned models, small LLMs, model routing, caching, distillation, prompt compression, etc., rather than mass abandonment of AI."