Dynamic model routing will follow the path blazed by software-defined wide-area networks
Stripe Inc.’s planned acquisition of OpenRouter Inc. put a spotlight on dynamic model routing. Information technology organizations need to understand what dynamic model routing is all about, as it will have significant impacts on cost, quality and performance as artificial intelligence permeates the enterprise.
Organizations have quickly realized that they shouldn’t rely on a single model for every task. Different models have different strengths in performance, latency, cost, privacy and domain-specific accuracy. Model routing provides an intelligent decision layer that directs tasks to the optimal model or inference endpoint based on the needs and conditions at that moment.
Dynamic model routing mirrors the shift that took place years ago in software-defined wide-area networking, or SD-WAN. As digital environments become more distributed, dynamic and business-critical, static decision-making must give way to systems that can evaluate context and adapt in real time to deliver the best possible experience.
From reachability to context-awareness
As applications moved out of the data center and into multiple clouds and software-as-a-service environments, the network’s role evolved beyond simply establishing a connection to delivering the experience the business required. As traffic patterns became more distributed, the SD-WAN became more application-aware, policy-driven and adaptive. Modern WAN architectures now use signals such as application type, path performance, security policy, availability and business priority to deliver the right experience over the right path, with the right policy applied.
Model routing introduces a similar decision-making challenge at a different layer of the stack. The question is no longer whether an AI request can be answered but which model, endpoint or environment is best suited to answer it based on the task, performance requirements, cost, privacy constraints and current conditions.
Just as WAN routing became more valuable as it incorporated richer signals, model routing will require a similar foundation. The “best” model can’t be determined by a single metric. The cheapest model may not meet accuracy requirements. The fastest model may not satisfy privacy needs. Without clear policy, visibility and controls, dynamic decision-making can introduce unpredictability instead of agility.
Model routing shouldn’t mimic WAN routing, but it still needs grounding in context, policy, visibility and controls. SD-WAN fabrics continuously collect real-time telemetry on path performance, application identity and security posture, then apply centralized policy engines to steer traffic accordingly.
Application-aware steering evaluates conditions such as latency and packet loss in real time and automatically shifts traffic to a better path. Observability platforms give IT teams a consolidated view across branches, clouds and software-as-a-service so they can continuously validate routing decisions rather than set them once and leave them alone.
AI workflows are different
Traditional applications followed relatively predictable paths between users, data centers, and applications. AI changes all that. A single AI interaction may initiate a workflow spanning a branch location, multiple clouds, SaaS applications, data stores and multiple AI models. AI agents call services, retrieve data, trigger actions and generate new requests dynamically.
Each step can introduce different requirements. Some data may need to stay within a certain environment. Some requests may require low latency. Others may prioritize cost efficiency. Some workflows may need stronger security controls or more detailed auditability.
And many of these dependencies may sit outside environments the enterprise directly owns or controls. The need for intelligent dynamic routing is especially clear at the branch and edge, where many AI-powered experiences reach employees and customers.
Urgency at the edge
Most retail stores, banks, clinics, factories, schools and public-sector locations are already dependent on cloud-based applications. AI will add complexity as customer interactions, clinical workflows, fraud analysis, inventory checks and field service requests increasingly depend on applications and data spread across clouds, SaaS platforms, private environments and AI services. Latency, outages, policy gaps and limited visibility can slow down the business and add operational burden for IT teams.
The branch can’t be treated as a static endpoint in the AI era. It will need networking, security, automation and observability working together so AI services can operate securely and reliably.
Networking has already shown what this evolution looks like. As environments became more distributed, routing became more adaptive, policy-driven and informed by real-time conditions.
AI infrastructure is entering a similar phase. Model routing is the latest example of a pattern networking has been solving for years. It’s a useful reminder that dynamic, context-aware routing is only growing more important.
Peterson is vice president of product management for Secure WAN at Cisco Systems Inc. He wrote this article for SiliconANGLE.
Image: theCUBE Research/Gemini
Support our mission to keep content open and free by engaging with theCUBE community. Join theCUBE’s Alumni Trust Network, where technology leaders connect, share intelligence and create opportunities.
- 15M+ viewers of theCUBE videos , powering conversations across AI, cloud, cybersecurity and more
- 11.4k+ theCUBE alumni — Connect with more than 11,400 tech and business leaders shaping the future through a unique trusted-based network
Are you an AWS customer? Support SiliconANGLE financially by buying your AWS services from our Marketplace portal page and links: https://siliconangle.com/aws-marketplace/
About SiliconANGLE Media
theCUBE AIand theCUBE SuperStudios — with flagship locations in Silicon Valley and the New York Stock Exchange — SiliconANGLE Media operates at the intersection of media, technology and AI.
Founded by tech visionaries John Furrier and Dave Vellante, SiliconANGLE Media has built a dynamic ecosystem of industry-leading digital media brands that reach 15+ million elite tech professionals. Our new proprietary theCUBE AI Video Cloud is breaking ground in audience interaction, leveraging theCUBEai.com neural network to help technology companies make data-driven decisions and stay at the forefront of industry conversations.