# Untrusted Code, Trusted Cluster Scaling Secure AI Agent Workspaces with GKE Agent Sandbox

> Source: <https://dev.to/gde/untrusted-code-trusted-cluster-scaling-secure-ai-agent-workspaces-with-gke-agent-sandbox-1mk1>
> Published: 2026-05-31 04:03:42+00:00

How gVisor-powered sandbox isolates AI-generated code at the kernel level and why it changes everything for multi-tenant agentic systems.

In this article we are going discuss on below points

The problem with AI agents writing code

What is GKE Agent Sandbox?

How gVisor intercepts the kernel

Architecture deep dive

Setting it up: step by step

Production patterns

Conclusion

There's a moment every engineer running AI agents eventually faces: an LLM generates a perfectly plausible subprocess.run() call, pipes it to bash -c, and realise that one prompt injection away from a full container escape. The code looks reasonable. The agent trusts itself. And cluster's blast radius just became everyone's problem.

This is the defining security problem of the agentic era. Language models don't just generate text anymore they write, execute, and iterate on code in tight feedback loops. The capabilities that make them useful (unrestricted Python, shell access, file I/O) are exactly the capabilities that make them dangerous in a shared cluster.

Google's answer — **GKE Agent Sandbox**

**GKE Agent Sandbox** is built for agentic workloads that require high-level scale, extensibility, and security. Key benefits include:

**Kernel-level isolation**: Provides strong, kernel-level isolation for untrusted, LLM-generated code by using built-in GKE features like GKE Sandbox. Agent Sandbox also supports the open source Kata Containers software.

**Sub-second provisioning**: Offers an out-of-the-box mechanism to provide sandboxes significantly faster than standard Kubernetes Pod scheduling allows (typically <1s).

**Cloud-native extensibility**: Leverages the power of the Kubernetes paradigm and the managed infrastructure of GKE.

By providing a declarative, standardized API, GKE Agent Sandbox offers a single-container experience that provides isolation and persistence characteristics similar to a virtual machine (VM), built entirely on Kubernetes primitives

**The problem with AI agents writing code**

Agentic AI systems whether you're building with LangGraph, AutoGen, Claude's tool-use API, or rolling your own share a common architectural pattern: the model generates code, a runtime executes it, results flow back to the model, and the loop continues. At each iteration, the model has broader context about what worked and what didn't. This is enormously powerful for automating complex tasks.

It also creates an attack surface that traditional Kubernetes security was never designed to handle.

**Container escape**

LLM-generated code exploits known kernel vulnerabilities or misconfigured capabilities to break out of the container boundary.

**Prompt injection via code output**

Malicious content in retrieved data embeds instructions that manipulate the agent into executing attacker-controlled payloads.

**Lateral network movement**

An agent with network access can enumerate internal services, extract credentials, and pivot across your cluster — all through legitimate-looking Python requests.

**Filesystem exfiltration**

Without mount restrictions, agents can read service account tokens, Kubernetes secrets mounted as volumes, and host path data.

Standard container security — **securityContext**, network policies, Pod Security Admission provides defence in depth but doesn't address the fundamental issue: containers share the host kernel. If the kernel has a vulnerability, a sufficiently motivated attacker (or sufficiently capable LLM) can exploit it regardless of namespace isolation.

**What is GKE Agent Sandbox?**

GKE Agent Sandbox is a Google-managed node pool configuration that applies gVisor-based container sandboxing specifically tuned for agentic AI workloads.

At its core, it combines three things:

**gVisor runtime (runsc) as the default OCI runtime**

Every container in the sandbox node pool runs under runsc instead of the standard runc. This intercepts all syscalls through a user-space kernel implementation called Sentry.

**Agent-specific resource isolation profiles**

Pre-configured seccomp and AppArmor profiles optimised for Python/Node.js/container-in-container workloads that AI agents commonly generate. No manual tuning of syscall allowlists required for standard use cases.

**Integrated observability via Cloud Monitoring**

Syscall audit logs, sandbox violation events, and resource consumption metrics flow automatically into Cloud Monitoring — giving you behavioural baselines for agent workloads without custom instrumentation.

**How gVisor intercepts the kernel**

Understanding what gVisor actually does is essential for reasoning about its security guarantees. The mental model most engineers have of containers — "a process with namespaces and cgroups" — breaks down when thinking about gVisor.

In a standard container, your application's open(), read(), execve(), and socket() calls go directly to the host Linux kernel via the system call interface. The kernel has to handle them, which means a kernel vulnerability is reachable from inside the container.

With gVisor, those same syscalls are intercepted by Sentry a Go implementation of the Linux kernel that runs entirely in user space. Sentry implements the Linux ABI from scratch. When your agent code calls execve(), it's Sentry that handles it, not the host kernel. Sentry then makes a much smaller set of calls to the actual host kernel (through a restricted interface called the "platform") to handle things like memory mapping and scheduling.

**End-to-End Architectural Blueprint**

To isolate untrusted code execution while maintaining a highly responsive management plane, the architecture splits the cluster into two distinct, specialized node pools.

**Standard Node Pool (The Brain)**- This pool runs your trusted, long-lived orchestration services. Because this code is written and audited by your team, it runs on the standard Linux host kernel for maximum performance and native access to internal cluster resources.Agent Controller: The core engine managing the life cycle of AI agent tasks, spin-up times, and state tracking.Tool Router: Mediates external API calls and manages what capabilities (e.g., web search, database querying) are exposed to the agent.Result Collector: Aggregates outputs, logs, and state changes from the runtime pods.State & Storage (Postgres/Redis): Highly available data layers tracking session memory and agent state.

**Agent Sandbox Node Pool (The Muscle)** - This pool is dedicated entirely to executing untrusted code generated by AI models. It uses the runtimeClassName: gvisor configuration to enforce strict kernel-level isolation.Code Executor Pods ($N$ Pods): Ephemeral, rapid-churn pods designed to spin up, run a specific snippet of generated code, and terminate.The Sentry (User-Space Kernel): gVisor’s core component. Instead of letting a Python agent talk directly to the host Linux kernel via standard system calls (syscall()), the Sentry intercepts them. It implements a core suite of Linux kernel primitives in user-space, shielding the host bare-metal or VM infrastructure from container escape vulnerabilities.

**Workload Identity & RBAC Separation**

By separating Kubernetes Service Accounts (KSAs) and mapping them to distinct Google Cloud IAM Service Accounts, we eliminate the risk of privilege escalation if an agent is compromised.

**Observability and Behavioral Analysis**

Because sandbox runtimes are naturally adversarial, observability shifts from standard application performance monitoring (APM) to real-time behavioral and security auditing

**Syscall Audit Logs**: gVisor provides structural logs of intercepted system calls via its internal logging mechanisms. Unusual system calls (e.g., attempts to call forbidden network protocols or direct raw socket manipulations) are immediately streamed to Cloud Logging.

**Violation Events**: Any attempt by a sandboxed container to bypass the Sentry or execute an invalid operation triggers an immediate containment event, surfaced directly in Google Cloud Security Command Center.

**Cloud Monitoring**: Aggregates container-level metrics (CPU, Memory, Churn rate). Crucial for detecting malicious infinite loops or resource-exhaustion (DDoS) attempts disguised as AI agent tasks.

**Cloud Trace**: End-to-end distributed tracing maps exactly how long a request spends routing through the Tool Router versus how long it spends executing inside the gVisor sandbox, allowing you to fine-tune the performance overhead introduced by user-space context switching.

**Setting it up: step by step**

Here's a complete walkthrough from a fresh GKE cluster to a running sandboxed agent workload. This assumes you have gcloud, kubectl, and Terraform configured for project.

**Production patterns**

**Pattern 1: Warm pool with pre-forked executors**

Cold-starting a new pod for every code execution adds latency. The standard pattern is to maintain a pool of warm executor pods that listen for work over a task queue (Pub/Sub or Redis Streams). The controller dispatches code snippets to idle executors; completed executors reset their environment and return to the pool. A garbage collection sidecar restarts pods that have been warm too long to prevent state accumulation.

**Pattern 2: Execution budget enforcement**

AI agents can get into infinite loops. Beyond Kubernetes resource limits, apply an application-level timeout using Python's signal.alarm or Go's context cancellation. A 30-second wall-clock timeout with a 10-second CPU-time budget covers almost all legitimate agent code execution patterns while preventing runaway loops from consuming pool capacity.

**Pattern 3: Network egress allow-listing per agent type**

Different agent personas have different legitimate network needs. A data analysis agent needs access to BigQuery and GCS. A web research agent needs HTTP egress to public internet. A code review agent needs neither. Model this with separate NetworkPolicies per agent label, and use PodSpec labels to bind agents to the right policy at scheduling time.

**Conclusion**

The agentic era is here, and it runs on code execution. Whether you're building autonomous research assistants, DevOps automation agents, or data pipeline orchestrators,eventually going to need a principled answer to the question: what happens when the model writes something it shouldn't?

GKE Agent Sandbox doesn't make the threat go away. Prompt injection is still a model-level problem. Lateral movement still requires complementary network controls. Secrets management still requires RBAC discipline. But the sandbox answers a specific, hard question — what if agent-generated code exploits a kernel vulnerability or escalates privileges? — with a credible, production-tested answer: it runs against Sentry, not your host kernel.

For most teams running agentic workloads on GKE, the operational cost is low (a single node pool configuration), the performance cost is acceptable (single-digit percentages for typical agent workload patterns), and the security benefit is significant (kernel-level isolation with full Kubernetes observability).

That's the architectural question GKE Agent Sandbox is designed to answer. Build agentic systems with the assumption that the code will sometimes be wrong, sometimes be manipulated, and occasionally be malicious and design your execution environment accordingly.

**References and Documentation**

[https://docs.cloud.google.com/kubernetes-engine/docs/how-to/agent-sandbox](https://docs.cloud.google.com/kubernetes-engine/docs/how-to/agent-sandbox)

[https://docs.cloud.google.com/kubernetes-engine/docs/concepts/machine-learning/agent-sandbox](https://docs.cloud.google.com/kubernetes-engine/docs/concepts/machine-learning/agent-sandbox)
