How GKE networking scales to support high densities of concurrent AI agents

Google's GKE Dataplane V2, powered by eBPF, now separates pod labels from annotations to lift the cluster-wide ceiling of 65,536 network identities that high-density AI agent workloads were approaching, Google Cloud reported. The change targets sub-second provisioning for agentic workflows, where reinforcement-learning jobs can churn thousands of isolated sandboxes per minute and research labs demand up to one million concurrent code-execution sandboxes. Google cited a customer, Mistral AI, that used Agent Substrate on GKE to orchestrate hundreds of thousands of secure environments across clusters and handle spikes of over 30,000 sandboxes on a single cluster.

What do you need to orchestrate high-density concurrent AI agents? As we transition from static requests toward autonomous agentic workflows, the need for secure, high-density sandboxes is pushing conventional infrastructure far beyond its architectural breaking point. This transition is driving top-tier research labs to demand infrastructure at unprecedented scale — often up to one million concurrent code-execution sandboxes At this scale, the efficiency of your Kubernetes networking infrastructure directly impacts the economic viability of an agentic initiative. In advanced workflows like reinforcement learning RL , extreme pod churn can mean thousands of isolated sandboxes spinning up and down every minute. To make sure that valuable GPUs and TPUs are spending their time executing models rather than waiting on sandboxes to initialize, you need sub-second provisioning and extreme compute density. By providing secure, lightweight isolation, Agent Sandbox https://docs.cloud.google.com/kubernetes-engine/docs/concepts/machine-learning/agent-sandbox on GKE is uniquely positioned to power these highly dynamic execution environments. Furthermore, for next-generation workloads with millions of concurrent idle sessions, Agent Substrate on GKE builds on these primitives to deliver even greater density and sub-second state rehydration. In this post, you will learn how GKE Dataplane V2 https://docs.cloud.google.com/kubernetes-engine/docs/concepts/dataplane-v2 optimizes network identity management, solves identity scaling limits, and enables sub-second provisioning to power high-density, large-scale AI agent workloads. Operating with massive concurrency introduces fascinating new engineering challenges, particularly in how you manage network identities. Traditionally, custom controllers and workload orchestrators often assign unique, dynamic labels to each pod such as agent-id, timestamps, or claim-uid . GKE Dataplane V2 https://docs.cloud.google.com/kubernetes-engine/docs/concepts/dataplane-v2 , powered by eBPF https://ebpf.io/ technology, helps enforce robust network security policies by mapping pod labels to a numeric identity. Because agentic workflows generate highly unique labels for every pod, the system proactively provisions a distinct identity for each sandbox. In a rapidly changing environment, this serial allocation process can briefly pause pod creation as the system securely updates eBPF maps, occasionally leading to slower-than-desired initialization. Additionally, sophisticated workloads can approach the cluster-wide limit of 65,536 network identities, either through this rapid generation or overall scale of the workload itself. Providing a reliable and highly secure platform is fundamental to everything we do. As such, we needed to find a way to provide the same underlying identity and security controls while providing higher scalability and performance. That's where the architectural innovations in GKE Dataplane V2 come in. A customer has testified to the success of Agent Substrate on GKE https://cloud.google.com/blog/products/containers-kubernetes/accelerate-agentic-rl-with-gke-agent-sandbox?e=48754805 :~:text=%E2%80%9CTo%20push%20the,Engineer%2C%20Mistral%20AI to orchestrate hundreds of thousands of secure environments across clusters and handle spikes of over 30,000 sandboxes on a single cluster. To achieve this, GKE relies on a two-pronged networking approach: Migrating high-cardinality data Google's GKE Dataplane V2 separates "identifying" data from "functional" metadata. Ensure your fields that do not actively participate in service selector https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/ matching or core resource grouping are migrated from the Pod's Labels set to the Annotations set. GKE Dataplane V2 intelligently calculates network identities based on labels rather than annotations, moving this high-cardinality metadata out of the labels completely bypasses the need for per-Pod identity generation. This elegant shift natively resolves identity scaling limits without sacrificing visibility. Advanced Cilium label filtering In addition, applying precise label exclusion filters e.g., using the cilium-config-emergency-override ConfigMap in kube-system by configuring rules like agents.x-k8s.io/sandbox-name-hash or timestamp, these explicitly instruct GKE Dataplane V2 to gracefully ignore these dynamic labels https://docs.cilium.io/en/stable/operations/performance/scalability/identity-relevant-labels/ during security identity calculation. This means that even if a developer framework inadvertently introduces high-cardinality labels, configuring appropriate exclusion filters allows Google’s network dataplane to drop them automatically. As a result, identical agent types continue sharing a single Cilium pod identity, maintaining the ultra-low-latency startup times critical for dynamic AI workloads. By preventing linear identity growth and avoiding Cilium’s 65,536 identity limit, GKE provides leading speed and efficiency for fully isolated agents with 300 sandboxes per second at sub-second latency. For workloads that require extreme performance, scale, and density, Agent Substrate https://docs.cloud.google.com/kubernetes-engine/ai-ml/about-agent-substrate natively integrates these GKE Dataplane V2 recommendations while bypassing Kubernetes scheduling limits, pushing GKE’s performance and scale even further. In benchmark validation running 200,000 agents, Agent Substrate sustained 500 activations per second at less than 500 ms suspend and resume latency. You can deploy it on your existing GKE clusters today to minimize idle compute costs for your most demanding AI workflows. By optimizing network identity calculation and filtering high-cardinality metadata, GKE Dataplane V2 addresses critical infrastructure bottlenecks in large-scale AI workloads. These architectural improvements reduce sandbox provisioning latency to sub-second levels, minimize expensive GPU idle times, and help maintain high compute density. Ultimately, these efficiencies let you scale your agentic AI operations reliably while controlling operational costs. To learn more, explore Agent Sandbox https://docs.cloud.google.com/kubernetes-engine/docs/concepts/machine-learning/agent-sandbox and Agent Substrate https://docs.cloud.google.com/kubernetes-engine/ai-ml/install-overview-substrate on GKE.