Accelerating agentic RL and evaluation research velocity with 45x faster GKE Agent Sandbox

Google announced general availability of GKE Agent Sandbox optimized for reinforcement learning, along with an Agent Sandbox RL orchestration SDK and native integrations for RL gyms and harnesses, claiming 45x faster sandbox startup for agentic RL and evaluation workloads. In a 10-node gVisor sandbox pool with GKE image streaming, the Agent Sandbox controller and an in-cluster SDK driver, Google said the setup supports thousands of images larger than 1.2GB while cutting the time-to-first-code (TTFC) that otherwise leaves GPU clusters idle during CPU sandbox cold-starts. The release targets frontier AI labs running tens of thousands of parallel rollouts, with SandboxWarmPool pre-initialized environments plus snapshot, suspend, resume and sandbox forking for checkpointing and parallel agent branching.

When scaling up agentic reinforcement learning RL and evaluation across massive parallel rollouts, frontier AI labs inevitably hit a bottleneck: Expensive GPU clusters sit idle, waiting minutes for CPU sandbox cold-starts, plus thousands of multi-gigabyte SWE-bench https://www.swebench.com/original.html -style image pulls and scheduling backlogs. It’s a sandbox infrastructure problem that silently slows down your research and burns your training budget. To solve this fundamental infrastructure bottleneck, today we are introducing GKE Agent Sandbox optimized for RL along with the Agent Sandbox RL orchestration SDK https://github.com/kubernetes-sigs/agent-sandbox/tree/main/examples/agent-sandbox-rl , plus native integrations https://github.com/kubernetes-sigs/agent-sandbox/tree/main/clients/integrations for popular RL gyms and harnesses, now generally available. As the operating system for modern AI, Kubernetes has evolved to power massive GPU/TPU training clusters and distributed inference. Now Kubernetes is expanding to drive the next AI compute frontier: agents. But unlike static workloads, agentic workloads evolve rapidly, so infrastructure must evolve just as fast. Rather than guessing at what RL researchers needed, we placed Kubernetes itself on an auto-research and verification loop driven by performance benchmarks and evaluations . We used heavy agentic benchmarks like SWE-bench https://www.swebench.com/verified.html to intentionally stress-test and break our own clusters. Every bottleneck that surfaced — from etcd timeouts to GPU idle spikes — was fed back into our development cycle to refine GKE’s core primitives. This resulted in a purpose-built sandbox layer for agentic RL and eval workloads that features: With this new primitive, AI labs and agent-native startups can now reliably run large scale agentic RL trajectories and evals simultaneously, minimizing accelerator idle time and drastically accelerating their research velocity. Before we talk about the solution, let’s be precise about what makes agentic RL so demanding for infrastructure in the first place. In a standard agentic RL loop, an LLM policy generates actions like code snippets on GPUs and executes them inside isolated CPU sandboxes to observe a reward signal. However, when scaling up this loop to support tens of thousands of parallel rollouts, three critical infrastructure bottlenecks emerge: These are not hypothetical problems. They are the daily reality for frontier AI labs training state-of-the-art agents. To fan-out the code execution sandboxes, we set up a relatively modest cluster — a 10-node gVisor sandbox pool with GKE image streaming enabled, the Agent Sandbox https://docs.cloud.google.com/kubernetes-engine/docs/concepts/machine-learning/agent-sandbox controller, and an in-cluster SDK https://github.com/kubernetes-sigs/agent-sandbox/tree/main/examples/agent-sandbox-rl driver to claim warm pods. We tested various strategies and setups including a large number of images and high cardinality. GKE Agent Sandbox https://docs.cloud.google.com/kubernetes-engine/docs/concepts/machine-learning/agent-sandbox is the open Kubernetes primitive for secure agent execution, featuring built-in SandboxWarmPool capabilities that eliminate cold-start overhead by maintaining pre-initialized, healthy environments. By integrating SandboxWarmPool with GKE Image Streaming https://docs.cloud.google.com/kubernetes-engine/docs/how-to/image-streaming , we effectively support RL and eval workloads that demand high image cardinality — even with thousands of large images 1.2GB , while delivering the exceptionally low TTFC required for responsive agentic training. Its snapshot, suspend and resume https://docs.cloud.google.com/kubernetes-engine/docs/how-to/agent-sandbox-pod-snapshots capabilities support checkpointing for error recovery, and sandbox forking for parallel agent branching logic to explore multiple trails. This GKE primitive is already powering the agentic RL training infrastructure at Mistral AI, a frontier AI lab: “To push the boundaries of reinforcement learning, you need infrastructure that can instantly scale to handle unpredictable demand. By leveraging GKE's high-performance Agent Sandbox for RL, we can seamlessly orchestrate hundreds of thousands of secure environments across clusters and handle spikes of over 30,000 sandboxes on a single cluster. It provides the reliable foundation we need to accelerate our model training and iteration cycles.” - Jean-Malo Delignon, Research Engineer, Mistral AI To help researchers harness this power without writing Kubernetes YAML or embedding custom daemons into container images, we built the Agent Sandbox RL orchestration SDK https://github.com/kubernetes-sigs/agent-sandbox/tree/main/examples/agent-sandbox-rl . It exposes a clean, async Python API with pluggable warm-pooling strategies tailored to your specific evaluation or RL training pattern. We also built native integrations https://github.com/kubernetes-sigs/agent-sandbox/tree/main/clients/integrations for RL tools like Gymnasium https://github.com/kubernetes-sigs/agent-sandbox/tree/main/clients/integrations/gymnasium , NVIDIA NeMo Gym https://github.com/kubernetes-sigs/agent-sandbox/tree/main/clients/integrations/nemo-gym , and OpenHands https://github.com/kubernetes-sigs/agent-sandbox/tree/main/clients/integrations/openhands , with more to come. We ran this on a 10-node gVisor sandbox pool against two workloads: a 500-image SWE-bench https://www.swebench.com/verified.html environment, and a harsher 4,578-image R2E https://huggingface.co/R2E-Gym/datasets corpus that does not fit in local disk. 1.TTFC and tail latency: 10x–45x faster | Core metric | Why it matters | Raw K8s pod baseline | GKE Agent Sandbox SDK | Benchmark gains | |---|---|---|---|---| | TTFC, average | Determines GPU idle time | 44s – 85s average | 1.1s – 8.8s average | 10x faster | | Tail latency — max TTFC worst case | The bottleneck for the batch | 7.5 mins 450 seconds | Strictly under 10 seconds | 45x faster | Concurrency ranged from 500 simultaneous tasks and sandboxes up to 18,312 4,578 R2E images × 4 rollouts . The gain held across every setup. How we got here. We instrumented the controller, the SDK, and the RL fleet, then ran hundreds of comparison runs against that fixed 10-node budget. Two changes produced nearly all of the gain: Image hydration moved off the critical path. A high-cardinality corpus far exceeds local disk, so pulling cold images mid-training creates I/O contention and multi-minute tails. The SDK plans image placement across nodes, and GKE Image Streaming https://docs.cloud.google.com/kubernetes-engine/docs/how-to/image-streaming plus upfront warm-pooling handle the rest — hydration finishes before the rollout ever asks for it. The control plane is paced. High-concurrency rollouts trigger API-server thundering herds. Agent Sandbox Controller v1.0.0 https://github.com/kubernetes-sigs/agent-sandbox/releases/tag/v1.0.0 adds rate controls that bound how fast warm pools refill, keeping etcd and the API server stable through the burst. The tuning knobs, the benchmark harness, and the load tests behind these numbers all ship in the agent-sandbox-rl https://github.com/kubernetes-sigs/agent-sandbox/tree/main/examples/agent-sandbox-rl example, which emits human-readable and JSON reports so you can reproduce the comparison on your own cluster. 2. 3x less control-plane and pod lifecycle churn during multi-trajectory rollouts | Core metric | Why it matters | Raw K8s pod baseline | GKE Agent Sandbox SDK | Benchmark gains | |---|---|---|---|---| | Control-plane churn 4,578 images × 4 rollouts run | Control-plane saturation | 18,312 | 5,869 | 3.1x less churn | Raw Kubernetes recreates a pod for every trajectory step. At 18,312 tasks that inflates scheduling overhead, wastes disk I/O, and — in our large-scale tests — triggered unbounded garbage-collection loops. The SDK's in-place recycle strategy keeps the pod alive instead, running an in-pod git reset and repository checkout between rollout episodes. Pod creations drop 3.1x and the control plane stays flat through the burst. Warming environments up front spends inexpensive CPU and background cluster time so that image streaming and readiness checks never land on the critical path. For an RL fleet that’s a straightforward trade: Accelerator idle time is the expensive resource, and this approach eliminates it. One detail matters at training scale: The SDK counts and surfaces every failed sandbox as retriable rather than silently dropping it. An untracked drop is not just a lost rollout — it is reward bias.