Turn Distributed Capacity into AI Infrastructure

Cisco introduced an AI Inference Routing blueprint at NVIDIA GTC Berlin, outlining how service providers can place, route, secure and assure inference across a distributed footprint spanning a single edge site to a sovereign regional hub. The blueprint builds on existing mechanisms — Segment Routing over IPv6 (SRv6), Flexible Algorithm (Flex Algo) and Border Gateway Protocol (BGP) — so providers can translate customer intent such as response time, jurisdiction, resiliency or cost tier into network behavior without a new routing protocol or a separate network per workload. Cisco positions the design as a way to raise utilization through placement and routing rather than adding infrastructure, letting idle capacity in one region serve demand in another.

For service providers, AI creates a familiar strategic question. Will the network carry someone else’s value, or will it become the platform on which that value is created? Inference gives service providers a strong opportunity to choose the second path. They already have assets that the AI economy needs: distributed facilities, power, trusted infrastructure, and reach in the regions where enterprises and governments operate. Those assets become more valuable when they can operate as one system. An inference request may have suitable capacity near by, yet still travel farther than necessary because the network does not have enough context about the workload. The best destination may depend on latency, available throughput, cost, or a jurisdictional requirement. Conventional reachability alone does not express all those needs. At NVIDIA GTC https://www.nvidia.com/gtc/ Berlin, Cisco is introducing a blueprint for AI Inference Routing, https://www.cisco.com/c/en/us/solutions/collateral/design-zone/ai-inference-wan-wp.html one of several areas where Cisco and NVIDIA are working together for service providers. It outlines how service providers can place, route, secure, and assure inference across a distributed footprint, from a single edge site to a sovereign regional hub. The goal is to solve utilization through placement and routing rather than more infrastructure. Capacity sitting idle in one region can be applied to demand in another, and the resulting service can be built on demand rather than pre-provisioned site by site. Why inference changes the role of the network An AI prompt may look like a simple exchange between a user and a model, but the underlying transaction crosses a distributed system. It can pass through the Internet or an enterprise network, policy engines, API gateways, model-routing layers, CPU serving infrastructure, and a GPU fabric before the response returns https://blogs.cisco.com/sp/why-ai-inference-is-becoming-a-networking-issue . That makes placement and connectivity part of the application experience. This matters even more as inference spreads across central AI factories, regional data centers, metro locations, and edge sites. One request may need the lowest-delay path. Another may need to remain within a national boundary. A third may prioritize cost while still meeting a performance objective. Service providers need a way to translate those requirements into network behavior, and to verify with measurement that the service is delivering what was committed. A blueprint built on deployed network standards The AI Inference Routing blueprint https://title:%20https/www-author.cisco.com/c/en/us/solutions/collateral/design-zone/ai-inference-wan-wp.html?wcmmode=disabled%20%C2%A0%20Listing%20Page:%20https://www.cisco.com/c/en/us/solutions/design-zone/service-provider.html%20Landing%20HTML:%20https://www-author.cisco.com/c/en/us/solutions/collateral/design-zone/ai-inference-wan-wp.html?wcmmode=disabled starts with mechanisms that service providers already use, including Segment Routing over IPv6 SRv6 , Flexible Algorithm Flex Algo , and Border Gateway Protocol BGP . A customer states what they need, such as response time, jurisdiction, resiliency, or cost tier, and the blueprint describes how the network can interpret that intent to make inference locations reachable and select paths that satisfy it. It does not require a new routing protocol or a separate network for every workload. Flex Algo can calculate paths using defined metrics and constraints. Segment Routing can steer traffic along those paths, while BGP distributes reachability information across the footprint. Used together, these mechanisms provide a practical foundation for routing an inference request toward an eligible location and across a path that reflects its latency, throughput, cost, and jurisdiction requirements. In this design, paths can be instantiated when a service requires them rather than pre-built between every pair of locations, and network slicing can keep inference traffic isolated from other services on the same converged fabric. For service providers, the business value is straightforward: distributed compute no longer has to be sold as a collection of isolated sites. It can become a service with consistent policy and assurance across a broader footprint. That creates a foundation for differentiated inference connectivity, sovereign AI services, and managed edge AI offerings. Resilience, security, and operations must work together Delivering that service consistently across a complex, distributed footprint means these pieces can’t operate in isolation. The blueprint brings them together as one coordinated system. Resilient infrastructure The blueprint is designed to treat every participating inference site as part of one reachable system. Policies can distinguish among service objectives instead of sending every request along the same default path. Because the design uses established transport mechanisms, service providers can build on their current network architecture and operating knowledge. The blueprint also defines how to measure results by verifying response time, the path taken, and the location that served the request against what the service provider committed. Service providers can then prove that they met each service objective. Embedded security Distributed inference increases the number of network, cloud, and edge domains that a workload may cross. Security and sovereignty need to be considered with path selection. The blueprint places policy within the routing decision, showing how service providers can keep traffic on approved paths and apply protection consistently across access, metro, core, and edge domains. Agentic operations Service providers and large enterprises are balancing capability against control as they bring AI into network operations. Open models deliver on both, giving customers the control and economics that they want without sacrificing the capabilities that they need. Cisco’s portfolio already includes elements that service providers can use to execute this blueprint. For example, Crosswork AI, Cisco’s multi-agentic framework for network operations, integrates NVIDIA Nemotron open models and NVIDIA AI Enterprise software to support agentic tasks, such as deep network troubleshooting across a distributed footprint with the option to run models on premises. Crosswork AI observes network conditions, correlates signals across domains, and proposes a resolution before an issue affects service. Service providers can also build their own agents for their environments. Policy checks and operator approval remain part of the workflow, providing a governed path toward greater autonomy. Turning network reach into new services Service providers have spent decades building resilient networks, regional facilities, and operational expertise. AI inference gives those investments new roles. Following this blueprint, an operator can use itto build toward offering customers: - Inference connectivity services that select paths for defined latency, throughput, cost, or policy requirements - Sovereign AI services that direct traffic through approved infrastructure and within required boundaries - Managed distributed inference that combines regional or edge capacity into one service experience Service providers can shift from selling undifferentiated bandwidth to delivering an assured outcome. Customers do not have to assemble the network, compute locations, policy controls, and operating model themselves. The service provider can bring those elements together and manage them as one service. Extending our work with NVIDIA The AI Inference Routing blueprint is part of a broader effort between Cisco and NVIDIA to help customers build and operate AI from core to edge. Cisco Secure AI Factory with NVIDIA https://www.cisco.com/site/us/en/solutions/artificial-intelligence/secure-ai-factory/index.html provides a modular, pre-validated architecture for the AI workloads that run on the infrastructure. The blueprint extends that work to how requests reach distributed capacity across the provider network. Together, these efforts connect the AI stack with the network that carries its traffic. The infrastructure to compete in distributed AI is already present in many service provider networks. Our goal now is to help service providers make those assets function as a coordinated system, with the resilience, security, and operational control required for carrier-grade services Cisco at NVIDIA GTC Learn more https://www.cisco.com/site/us/en/learn/events/nvidia-gtc-berlin.html Get AI-ready connectivity Explore how Cisco builds networks for the AI era https://www.cisco.com/site/us/en/solutions/service-provider/networking/agile-services/index.html Stay up on Thought Leadership from Cisco Get the latest blogs from Cisco Executives in your email. Subscribe to the Executive Platform https://app.feedpress.com/e/mailverify?feed id=CiscoExecutivePlatform Executive Perspectives Navigate the latest technology trends and get solutions with the help of Cisco executives. Go to Executive Perspectives https://www.cisco.com/c/en/us/solutions/executive-perspectives/index.html