{"slug": "your-ai-agent-is-sandboxed-have-you-actually-tested-the-boundaries", "title": "Your AI Agent Is Sandboxed. Have You Actually Tested the Boundaries?", "summary": "A developer stress-tested NVIDIA OpenShell, an agent sandbox for Kubernetes, and found that standard Linux checks like non-root UID, NoNewPrivs, and seccomp filter mode confirm hardening defaults but do not demonstrate the actual security boundary. The writeup maps OpenShell's layered controls — Landlock for filesystem access, seccomp user notification for intercepting network requests, a trusted supervisor for policy mediation, and Kubernetes NetworkPolicy — and argues each must be tested separately because the mechanisms are not interchangeable.", "body_md": "*What I learned examining NVIDIA OpenShell on Kubernetes, from seccomp and Landlock to network mediation, policy enforcement, and runtime security evidence.*\n\nI recently started testing NVIDIA OpenShell because I wanted to understand something that isn't always obvious from agent security documentation: what actually stops an AI agent from doing something it shouldn't?\n\nI'd already spent time working with LLM gateways, guardrails, and observability. Those systems are useful for inspecting prompts, controlling model access, and understanding what applications send to an LLM.\n\nBut once an agent can execute commands, access a filesystem, or call external services, prompt filtering is only part of the story.\n\nA perfectly ordinary user request might lead to several tool calls, process executions, and network connections. The model doesn't necessarily have malicious intent. It might simply choose an unexpected way to complete the task.\n\nSo I wanted to examine the runtime boundary itself.\n\nI used OpenShell in a Kubernetes-based lab and began with some basic Linux security checks.\n\nThe first results looked reassuring.\n\nThey also turned out to be much less informative than I'd expected.\n\nMy initial checks returned:\n\n```\nuid=10001\nNoNewPrivs: 1\nSeccomp: 2\n```\n\nThese are sensible defaults for an execution-capable workload.\n\nThe process was running without root identity. The kernel reported `NoNewPrivs`, and seccomp was in filtering mode.\n\nBut none of these values answers the question I was actually trying to investigate.\n\n**Which operations can this agent perform, and where are those restrictions enforced?**\n\nConsider each result separately.\n\nA non-root process generally has fewer privileges than a root process. That's useful, particularly when the workload may execute code selected by an LLM.\n\nBut UID alone doesn't tell us which resources are accessible.\n\nA non-root process can still read files it has permission to access. It may also have network connectivity, application credentials, or writable directories.\n\nThe security outcome depends on the full permission model, not just the numeric user ID.\n\nThe `no_new_privs` setting prevents a process from gaining additional privileges through certain execution transitions.\n\nIt's a valuable hardening mechanism.\n\nHowever, it doesn't revoke privileges the process already possesses.\n\nAn agent doesn't need to escalate privileges to misuse an API credential that was intentionally made available to it.\n\nThis indicates seccomp filter mode.\n\nIt tells us that syscall filtering is active.\n\nIt does not reveal the complete effective filter, which operations are restricted, or whether the controls correspond to the policy we intended to enforce.\n\nA process may be heavily restricted in one area and still have substantial permissions elsewhere.\n\nAt that point, I'd established three useful environmental properties.\n\nI hadn't yet demonstrated the security boundary.\n\nLooking more closely at OpenShell's architecture changed how I thought about testing it.\n\nOpenShell separates the agent workload from a trusted supervisor.\n\nThe agent runs on the untrusted side of the boundary. The supervisor handles policy decisions and mediates supported operations.\n\nOn Linux, the design uses several mechanisms with different responsibilities:\n\n| Mechanism | Responsibility | \n|---|---|\n| Non-root identity and capability restrictions | Reduce process privileges | \n| Landlock | Restrict filesystem access | \n| Seccomp and user notification | Intercept relevant operations, including network requests | \n| Trusted supervisor | Evaluate and mediate permitted connections | \n| Kubernetes NetworkPolicy | Restrict direct workload network connectivity | \n| Sandbox policy | Define permitted resources and operations | \n\nThe important point is that these controls are not interchangeable.\n\nA filesystem restriction cannot replace an API authorization check.\n\nA NetworkPolicy object cannot tell us whether a user is allowed to modify a business record.\n\nA semantic guardrail cannot guarantee that a process will be unable to reach an unauthorized destination.\n\nDifferent boundaries require different enforcement mechanisms.\n\nIn the documented OpenShell architecture, outbound TCP and DNS operations are mediated through the sandbox and trusted supervisor.\n\nConceptually, the flow looks like this:\n\n```\nAI Agent\n   |\n   | Requests a network connection\n   v\nSandbox Runtime\n   |\n   | Intercepts relevant operation\n   | Identifies requesting executable\n   v\nAuthenticated Supervisor Channel\n   |\n   v\nTrusted Supervisor\n   |\n   | Evaluates effective policy\n   |\n   +---- DENY ---> Connection rejected\n   |\n   +---- ALLOW --> Supervisor connects\n                         |\n                         v\n                   Approved Service\n```\n\nThis is a useful design because the workload doesn't make its own authorization decisions.\n\nThe supervisor evaluates the operation outside the untrusted agent environment.\n\nBut another component is equally important: the outer network boundary.\n\nIf the workload could establish direct network connections without mediation, the supervisor wouldn't necessarily see those requests.\n\nOn Kubernetes, OpenShell relies on NetworkPolicy enforcement to constrain that communication path.\n\nThat means the cluster networking layer becomes part of the security argument.\n\nThis is something worth emphasizing when evaluating sandbox environments on Kubernetes.\n\nYou can inspect NetworkPolicy resources using:\n\n```\nkubectl get networkpolicy -A\n```\n\nYou can also review the policies in the sandbox namespace:\n\n```\nkubectl describe networkpolicy -n <namespace>\n```\n\nThese commands tell us what network restrictions have been declared.\n\nThey do not independently prove that the restrictions are working.\n\nKubernetes NetworkPolicy enforcement depends on a compatible network plugin.\n\nIn fact, the Kubernetes documentation makes an important distinction: the API stores the policy, but enforcement depends on the cluster networking implementation.\n\nThe OpenShell security documentation explicitly requires Kubernetes operators to use a CNI that enforces the relevant ingress and egress policies.\n\nFor me, this creates two separate validation questions.\n\n**First:** Does the policy permit only the intended supervisor communication?\n\n**Second:** Does the cluster actually enforce that restriction?\n\nAn effective assessment would need positive and negative connectivity tests in an isolated lab.\n\nA permitted connection should work.\n\nA prohibited connection should fail.\n\nAnd the results should be correlated with the relevant policy and enforcement logs.\n\nChecking only the Kubernetes resource isn't sufficient.\n\nAnother interesting part of OpenShell is its network policy model.\n\nIt's not simply a destination allowlist.\n\nRules can associate approved destinations with specific executables.\n\nFor example, allowing a command-line HTTP client to reach one service doesn't necessarily mean every executable in the sandbox should have the same permission.\n\nThat distinction matters because agents frequently launch other processes.\n\nA Python-based agent might invoke a helper script, which invokes another program, which eventually establishes a network connection.\n\nThe executable actually opening the connection may not be the same path that appears in the original tool call.\n\nOpenShell accounts for executable identity when evaluating the network policy.\n\nThis introduces an important testing requirement.\n\nIt's not enough to know that a destination is allowed.\n\nYou need to verify which executable is allowed to reach it, and whether broad matching rules unintentionally expand that permission.\n\nFor example, a rule intended to allow one trusted client may become much less restrictive if it matches almost any executable.\n\nThis is the type of configuration issue that can be missed during a quick functional test.\n\nThe agent completes its task successfully, so the system appears healthy.\n\nBut successful task completion says very little about whether least privilege has been maintained.\n\nI also found the distinction between base policy and effective policy important.\n\nA sandbox starts with a defined policy, but attached providers can contribute additional network permissions.\n\nThe effective policy reflects the permissions the runtime actually enforces.\n\nThat's what an assessor should examine.\n\nSuppose an agent initially has access to one internal API.\n\nLater, a provider is attached to support another service.\n\nIf that provider contributes network rules, the agent's reachable destinations may change even though the original base policy remains the same.\n\nNothing necessarily went wrong.\n\nThe change may be completely intentional.\n\nBut an assessment based only on the original policy would now be incomplete.\n\nFor repeatable testing, I would record:\n\nI would also distinguish controls established at startup from those that can be updated dynamically.\n\nThat matters for regression testing.\n\nA security test performed before a policy update may no longer describe the environment after that update.\n\nDuring my experiments, I encountered a few operational issues.\n\nOne sandbox returned:\n\n```\nsandbox 'attt-test-01' is not ready\n(phase: Completed)\n```\n\nWhen I attempted to create another sandbox using the same name, OpenShell reported that the name was already in use.\n\nLater, another operation failed because the underlying pod was in a failed state.\n\nThese errors were frustrating, but I wouldn't classify them as security findings.\n\nThey were sandbox lifecycle issues.\n\nStill, they revealed a practical weakness in my test process: I hadn't established a reliable way to recreate the exact environment for each experiment.\n\nIf a test doesn't run because the sandbox has completed, that isn't a security pass or failure.\n\nIt's an incomplete test.\n\nThat distinction sounds trivial, but it's easy to lose when reviewing a collection of terminal logs.\n\nI also observed a runtime security alert involving the execution of a newly introduced binary in a container.\n\nAgain, this wasn't evidence of a sandbox escape.\n\nAn agent that executes code may legitimately create or invoke new binaries.\n\nThe useful question is whether the execution violated a defined policy.\n\nTo answer that, I'd need more than the alert title.\n\nI'd want the executable path, parent process, sandbox identity, policy context, agent task, and corresponding runtime activity.\n\nWithout that information, I can say the event deserves investigation.\n\nI can't say the isolation boundary failed.\n\nAfter reviewing the architecture and my own incomplete results, I started organizing the assessment around security properties rather than individual commands.\n\nHere's the matrix I'd use for the next controlled test cycle.\n\n| Test | Expected property | Required evidence | \n|---|---|---|\n| Process identity | Agent operates with intended privileges | UID, capabilities and process security state | \n| Filesystem read | Restricted resources cannot be read | Effective policy and observed denial | \n| Filesystem write | Unapproved locations cannot be modified | Effective policy and operation result | \n| Approved network access | Required destination remains reachable | Successful request and matching authorization | \n| Unapproved network access | Connection is denied | Failed request and enforcement evidence | \n| Executable identity | Network access is restricted to permitted executables | Executable identity and policy decision | \n| Supervisor interruption | Agent does not continue unrestricted when mediation is unavailable | Runtime state transitions and logs | \n| Tool authorization | User and workflow permissions are enforced | Application authorization decision | \n| Observability | Agent action can be correlated with runtime activity | Trace ID, process and policy events | \n\nThe purpose isn't to produce the largest possible test suite.\n\nIt's to establish a defensible relationship between policy and observed behavior.\n\nFor each test, I would classify the result as:\n\n**Verified:** The test ran successfully and the observed behavior matched the intended restriction.\n\n**Failed:** The test completed, but behavior differed from the intended policy.\n\n**Not verified:** The environment, evidence, or test execution was insufficient to draw a conclusion.\n\nI particularly like keeping *not verified* separate.\n\nAn incomplete test should not turn into a successful security assessment just because no violation was observed.\n\nThere's another security boundary worth discussing.\n\nImagine an investigation agent operating in a properly configured sandbox.\n\nThe agent is allowed to reach an internal incident management API.\n\nIt can retrieve incident records and submit updates through that service.\n\nNow suppose the agent attempts to modify a record belonging to another department.\n\nThe network connection might be completely legitimate.\n\nThe sandbox might be doing exactly what it was configured to do.\n\nBut the operation could still be unauthorized.\n\nThe sandbox controls access to execution resources and network destinations.\n\nThe application must independently determine whether the requesting identity is allowed to perform a particular operation on a particular resource.\n\nThis is why I think tool authorization needs to be part of the agent architecture rather than being treated as a prompt instruction.\n\nA model can be told not to modify certain records.\n\nThat instruction may help guide its behavior.\n\nBut the API should still reject unauthorized modifications, even if the model attempts them.\n\nThe distinction becomes more significant with multi-agent workflows.\n\nOne agent might collect evidence. Another might summarize it. A third might propose a response action.\n\nEven when each tool is individually permitted, transferring information or authority between them may create new risks.\n\nRuntime isolation helps constrain what the execution environment can do.\n\nIt doesn't replace the application's responsibility to enforce identity, permissions, and data access.\n\nThis is the next area I want to explore more seriously.\n\nAt the LLM layer, platforms such as Langfuse can show model calls, retrieval operations, tool invocations, and workflow context.\n\nAt the runtime layer, process monitoring and Kubernetes telemetry can reveal actual execution events.\n\nOpenShell's supervisor and policy logs add another perspective.\n\nIdeally, those events should be correlated.\n\nConsider a hypothetical event sequence:\n\n```\nAgent task received\n        |\n        v\nDocument retrieved\n        |\n        v\nAgent selects a tool\n        |\n        v\nTool starts a process\n        |\n        v\nProcess requests network access\n        |\n        v\nSupervisor evaluates policy\n        |\n        v\nALLOW / DENY decision\n        |\n        v\nResult returned to agent\n```\n\nIf we only log the model response, we may miss the execution decision.\n\nIf we only capture the kernel event, we may not understand why the process ran.\n\nIf we only collect policy decisions, we may not know which user request triggered them.\n\nFor meaningful security investigations, these views need to be connected.\n\nI would start with a small set of correlation fields:\n\n```\nsession_id\nagent_id\ntool_call_id\nsandbox_id\npolicy_revision\nprocess_identity\noperation\ndecision\ntimestamp\n```\n\nThese are proposed telemetry fields, not a claim that OpenShell and Langfuse automatically produce this exact schema.\n\nSome would need to be propagated by the application or correlation pipeline.\n\nI'd also be careful about logging sensitive data. Agent traces can contain prompts, retrieved documents, tool arguments, and credentials.\n\nBetter visibility shouldn't create a new data exposure problem.\n\nBefore this experiment, I tended to start with the familiar container security checks.\n\nIs the process non-root?\n\nIs privilege escalation disabled?\n\nIs seccomp active?\n\nThose checks are still useful.\n\nI haven't stopped caring about them.\n\nBut for AI agent sandboxes, I'd now begin with a different set of questions.\n\nWhere is the trust boundary?\n\nWhich component makes authorization decisions?\n\nWhich operations are mediated?\n\nWhat prevents the agent from using an unmediated path?\n\nWhat does the effective policy allow?\n\nAnd what evidence would convince me that the boundary works?\n\nThese questions are more demanding than checking three Linux security flags.\n\nThey're also closer to the real security problem.\n\nI haven't finished validating every OpenShell boundary in my lab.\n\nI verified some process-level hardening properties, reviewed how the architecture is intended to enforce restrictions, and encountered several operational issues that interrupted the test cycle.\n\nI also observed runtime activity worth investigating, but nothing I collected was sufficient to establish a sandbox escape.\n\nThat's the honest scope of the experiment.\n\nI still think OpenShell is a useful direction for securing execution-capable agents because it moves important enforcement decisions outside the language model.\n\nBut I wouldn't judge any agent sandbox by its architecture diagram alone.\n\nThe same applies to seccomp, Landlock, Kubernetes NetworkPolicy, and application authorization.\n\n**A declared control is not the same thing as an effective control. And an effective control is not the same thing as a control you've actually verified.**\n\nFor AI agents, that distinction is going to matter more as we give them access to more systems and more consequential actions.\n\nI'm interested in how other engineers are testing these boundaries, particularly around dynamic network permissions, multi-agent tool access, and runtime event correlation.\n\nWhat would you consider sufficient evidence that an AI agent sandbox is enforcing its intended security policy?", "url": "https://wpnews.pro/news/your-ai-agent-is-sandboxed-have-you-actually-tested-the-boundaries", "canonical_source": "https://dev.to/techsavant_lab/your-ai-agent-is-sandboxed-have-you-actually-tested-the-boundaries-29ci", "published_at": "2026-11-21 10:58:48+00:00", "updated_at": "2026-11-21 11:11:58.092564+00:00", "lang": "en", "topics": ["ai-agents", "ai-safety", "ai-tools", "mlops"], "entities": ["NVIDIA", "OpenShell", "Kubernetes", "Landlock", "seccomp"], "also_reported_by": [], "takeaway": "Checking uid, NoNewPrivs and seccomp mode only proves hardening defaults, not the boundary — test each OpenShell layer separately, since Landlock filesystem rules, seccomp-mediated network requests, supervisor policy and NetworkPolicy are not interchangeable.", "recommendation": {"advice": "Test each sandbox layer independently — filesystem, network mediation, supervisor policy and NetworkPolicy — rather than treating non-root UID, NoNewPrivs and seccomp mode as proof the agent boundary holds.", "playbook": "agent-security", "playbook_url": "https://wpnews.pro/playbooks/agent-security"}, "alternates": {"html": "https://wpnews.pro/news/your-ai-agent-is-sandboxed-have-you-actually-tested-the-boundaries", "markdown": "https://wpnews.pro/news/your-ai-agent-is-sandboxed-have-you-actually-tested-the-boundaries.md", "text": "https://wpnews.pro/news/your-ai-agent-is-sandboxed-have-you-actually-tested-the-boundaries.txt", "jsonld": "https://wpnews.pro/news/your-ai-agent-is-sandboxed-have-you-actually-tested-the-boundaries.jsonld"}}