Kagent and Agent Substrate: Running AI Agent Sandboxes on Kubernetes

Kagent v1.0 and Agent Substrate combine to run AI agent sandboxes on Kubernetes, using gVisor for software-level isolation and microVMs for hardware-level isolation, according to the kagent.dev documentation. Agent Substrate runs all agents in Workers, which are Pods, facilitated via Actors managed by the ateapi control plane, while kagent serves as the runtime for declarative and bring-your-own agents covering frameworks such as LangGraph and ADK. The default Helm installation of kagent creates one WorkerPool with two Workers (Pods).

Isolation. Security. Scale. Without these three things, Agents cannot be properly implemented in a controlled environment, they will not have policy enforcement to ensure they do what they’re supposed to do, and there’s zero way of ensuring that the Agent will be readily available when needed. Kagent as the runtime for declarative and BYO covering frameworks like langgraph/ADK Agents combined with Agent Substrate for the underlying sandbox environment ensures that you can implement all three production-level needs when running agentic workloads. In this blog post, you’ll learn the key concepts to agent sandboxing, how Substrate is used, and implementing it with kagent. To follow along with this blog post in a hands-on fashion, you will need: If you do not have a k8s cluster, you can follow along from a theoretical perspective for the learnings until you have a cluster readily available. If you spin up Docker or whatever containerization engine you prefer, create a container, and run a workload in it, technically speaking the workload is “sandboxed”. However, that level of sandboxing might as well not even exist. Ingress and egress is wide open, the workload can send packets to whatever it likes, and there’s zero policy enforcement. Of course there are ways to tighten this up network policies, segmentation, routing rules, etc. , but that doesn’t change the fact that sandboxing as a whole is “flat” by nature. Sandboxing an Agent in a secure and effective fashion means: To ensure that the above can be properly implemented, you need the ability to isolate Agents at both the hardware and software level along with policy enforcement at the AI Gateway for when Agent traffic is reaching its destination. Container isolation and software-level isolation to that point stops at your kernel. Everything is running in said kernel in the user space , which can be sufficient for some, but how that isolation is implemented matters. How the client routes to the Agent and what is in the middle facilitating that routing is what counts when it comes to isolation. That’s where Agent Substrate comes into play. With Agent Substrate, all Agents are running in Workers, which are Pods. The Agents are facilitated via Actors, which are managed by the ateapi Agent Substrates Control Plane . Where the isolation comes into play is with software and hardware-level isolation. gVisor is used for software-level isolation and microVM is used for hardware-level isolation. With software-level isolation, the isolated workload runs inside of the user space using a dedicated user-space app kernel called Sentry that intercepts and handles system calls. With microVM hardware isolation the kernel itself where the workload runs is at the CPU chip level. With that chip level isolation, all workloads run on the physical chip, not in the user space. In the previous sections, you learned about the architecture, design, and system choices for why sandboxing an Agent is important. In the upcoming sections, you’ll learn how to do it in a hands-on fashion with kagent and Agent Substrate. The interaction points you’ll see here are: kagent is the runtime, Agent Substrate is the platform sitting underneath to help implement and facilitate sandboxing. Without kagent, the only real entry point to interact with Agent Substrate is via a programmatic call with something like a curl to interact with the Actor. Kagent allows you to interact with Agents running in Actors via Substrate in a way that the industry is comfortable with a UI and in a declarative fashion . There are a few steps to installing kagent v1.0 and Agent Substrate, including: To get a play-by-play, step-by-step instruction set, you can take a look at the guide found here https://kagent.dev/docs/kagent/1.x/setup/installation/ . Once you’re up and running, you’ll see that with the Helm installation of kagent by default, one WorkerPool gets created with two Workers Pods . The WorkerPool gets created with the substrateWorkerPool.create=true setting on the Helm Chart. With kagent + Agent Substrate installed, you can now create an Agent Actor and an Agent Template. The AgentTemplate is like a “golden image” although, “image” feels a bit too heavy of wording that contains the configurations for your Agent and can be reused across Agents that you deploy. The Harness is the Actor Agent itself. This can be something like a Harness Codex, Claude Code, etc. or a BYO Agent using a framework like langgraph. Run the below to declaratively create the Harness and AgentTemplate using the kagent CRDs. kubectl apply -f - <<EOF apiVersion: kagent.dev/v1alpha3 kind: Harness metadata: name: kagent namespace: kagent spec: kagent: {} workload: image: PATH TO YOUR AGENT IMAGE substrate: workerPoolRef: name: kagent-default snapshotPolicy: location: gs://ate-snapshots-PATH TO YOUR SNAPSHOTS allowedAgentTemplates: selector: matchLabels: kagent.dev/harness: kagent --- apiVersion: kagent.dev/v1alpha3 kind: AgentTemplate metadata: name: assistant namespace: kagent labels: kagent.dev/harness: kagent spec: modelConfig: name: default-model-config description: A substrate-backed assistant used to verify this main build. systemPrompt: You are a helpful assistant running on kagent. EOF Once deployed, you’ll see the Agent running in kagent along with the template. Aside from creating an Agent in a declarative fashion, you can also create one in the UI. You can then define the template like how you did in the YAML within the previous section. Within there is where you can define the harness like you did declaratively in the previous section to point to the Namespace that you want the Actor/Agent deployed in, the name, runtime adapter, and the location where the image exists. When you open the Agents tab, you’ll see the assistant Agent. Then, go over to the Substrate tab. Notice how it says 0/8 Actors running? That’s because an Actor is only running when there’s an active session this ties back to the efficiency of hardware resources when using Agent Substrate . Next, open your Agent and prompt it with whatever you’d like. You’ll now see that there are 0/9 Actors. The reason why is because when you start a new session, a new Actor spins up. It then spins down once the session is complete the session being what you’re asking the Agent and when it responds . If you open a new session and prompt the Agent, you’ll see 0/10. If you use an existing session, the Actor count won’t change because it’s still the same session. This is important because it can look like “every time I use an Agent, a new Actor spins up and that’s a lot of resources to manage”, but it’s only when a new session occurs. In this post, you not only learned about why sandboxing is important and the key projects driving sandboxing forward, but how to deploy Actors within kagent using Agent Substrate as the sandbox platform/architecture. Moving forward for engineering teams and enterprises alike, having the ability to isolate, secure, govern, and control Agents will be key to truly understanding what your Agents can and can’t do, which is a key holding point today for many organizations when implementing Agents. If you want to learn more about Agent Substrate, there are key live events going on, which you can watch for free.