{"slug": "nvidia-ropes-in-100-strong-posse-to-leash-rogue-ai-agents-after-openai-s", "title": "Nvidia ropes in 100-strong posse to leash rogue AI agents after OpenAI's walkabout", "summary": "Nvidia launched Open Agent Safety, an open-source platform backed by a 100-participant list including Anthropic, Cisco, Microsoft, Palantir and SpaceXAI, to provide agent governance and control software, following reported incidents in which OpenAI and Anthropic agents breached external systems. Nvidia CEO Jensen Huang said the effort \"is bigger than a single product. It's the beginning of an open ecosystem to build the trust layer for safe agent systems.\" The platform includes OpenShell, an open-source system-layer runtime enforcing policy-based security, network and privacy guardrails, and a Sentry reference system running on BlueField-4 DPUs that Nvidia says stops agents leaving their software boundary in milliseconds; OpenAI has not signed on.", "body_md": "Nvidia launched yet another open initiative aimed at strengthening AI security, a move that came just days after new broke that an OpenAI agent went \"rogue\" in gaining access to an Australian government website.\n\nThe Nvidia-backed initiative includes a 100-name-strong list of participants ranging from consulting giants to AI labs looking to provide agent developers with open-source software and reference designs covering agent governance and controls.\n\nNvidia CEO Jensen Huang in a [post on X](https://x.com/JensenHuang/status/2104499465055023424) said the agent-induced coming together “is bigger than a single product. It's the beginning of an open ecosystem to build the trust layer for safe agent systems.”\n\n“AI is extraordinary technology that will advance discovery, productivity, security, health, and prosperity for generations to come,” Huang wrote. “But its full promise can only be realized when people have confidence that AI is being built to be safe and deployed with wisdom and responsibility.”\n\nOpenAI made headlines for all the wrong reasons last week after reports suggested one of its agents breached Australia's Medicare Statistics Reporting Service. No sensitive data was accessed, but the ChatGPT maker is believed to have had months to reveal the breach. That breach followed another high-profile incident in which one of its agents escaped a test sandbox and snuck into [systems belonging to Hugging Face](https://www.sdxcentral.com/news/palo-alto-networks-ceo-warns-openai-hugging-face-breach-brings-browser-sase-to-the-fore/) (which [Nvidia is trying to buy](https://www.sdxcentral.com/control-plane/nvidia-wants-to-own-the-github-of-ai-what-could-possibly-go-wrong/)).\n\nIncidents of agents going walkabout aren’t limited to OpenAI. A similar safety event occurred at Anthropic, whose head of its Frontier Red Team said an agent broke out because engineers failed to adequately set up guardrails and mistakenly connected to the open internet.\n\nNvidia's Open Agent Safety [announcement](https://nvidianews.nvidia.com/news/open-agent-safety-platform) references such incidents, though without naming any lab in question. Instead, it simply reads: “Recent security incidents have underscored the need to equip organizations with open, customizable tools that enforce more control over long-running agents. Across these incidents, the pattern is the same – the agent circumvented security controls at the application layer to complete its assigned task.”\n\nNotably, Anthropic has signed onto the platform, but OpenAI has not. Among those joining the Claude developer program are Cisco, Microsoft, Palantir, and SpaceXAI.\n\nNvidia explained that the Open Agent Safety platform provides full-stack governance and control software for running agents, as well as the hardware and compute layers powering their work.\n\nIncluded is [OpenShell](https://www.sdxcentral.com/news/nvidia-details-nemoclaw-security-guardrails-in-wake-of-ai-agent-concerns/) – an open-source runtime at the system layer that enforces policy-based security, network, and privacy guardrails for agents. This was already available via security guardrails for [Nemoclaw](https://www.sdxcentral.com/news/nvidia-goes-all-in-on-agents-at-gtc-with-toolkits-openclaw-models/), Nvidia’s OpenClaw integration offering. The vendor said it now provides boundaries for agents executing tasks across both open and closed models.\n\nThe platform also features a reference system for Sentry, an out-of-band watchdog that runs on its [BlueField-4](https://www.sdxcentral.com/analysis/nvidias-bluefield-4-a-first-look-at-the-dpu-built-to-run-ai-factories/) DPUs to monitor agent behavior. The in-silicon security enforcement stops agents attempting to move out of its software boundary, a process Nvidia claims is performed “in milliseconds.”\n\n## Haven’t we done this already?\n\nThe Open Agent Safety unveiling follows hot on the heels of another Nvidia-led effort relating to AI security.\n\nThe [Open Secure AI Alliance](https://www.sdxcentral.com/news/broadcom-joins-nvidia-ai-security-alliance-we-havent-forgotten-open-source/) was formed in the wake of the Hugging Face incident, amassing many of the same logos signed onto this latest effort. Nvidia has since handed the reins of that group [to the Linux Foundation](https://www.sdxcentral.com/news/nvidia-offloads-its-all-star-ai-security-response-to-hugging-face-horror/), which is acting as a neutral home for open-source tools and defensive practices for securing software and AI agents.\n\nOpenAI is again a notable absentee among the Open Secure AI Alliance’s 120-plus members. Google is another major name absent from both initiatives, though it is represented in some form by its [Wiz](https://www.sdxcentral.com/news/google-cloud-closes-wiz-acquisition-begins-platform-player-brawl/) cloud security subsidiary.\n\nThe Open Agent Safety announcement references its now Linux Foundation-owned cousin, stating the platform supports its mission “as well as the broader AI safety and security community.”\n\nThe platform comes as agents join robotics as Nvidia’s latest fascinations. Huang [lauded OpenClaw](https://www.sdxcentral.com/news/nvidia-goes-all-in-on-agents-at-gtc-with-toolkits-openclaw-models/) at GTC earlier this year, famously labeling it and agents in general as “[the iPhone of tokens](https://www.reddit.com/r/aigossips/comments/1s4aqn0/openclaw_is_the_iphone_of_tokens_nvidia_ceo_on/).”\n\nOpen Agent Safety extends to physical AI, with leading robotics developers including Figure, Gecko Robotics, and Skild AI using OpenShell to embed agent safety controls.", "url": "https://wpnews.pro/news/nvidia-ropes-in-100-strong-posse-to-leash-rogue-ai-agents-after-openai-s", "canonical_source": "https://www.sdxcentral.com/news/nvidia-ropes-in-100-strong-posse-to-leash-rogue-ai-agents-after-openais-walkabout/", "published_at": "2026-09-28 14:37:43+00:00", "updated_at": "2026-09-28 14:47:58.944781+00:00", "lang": "en", "topics": ["ai-agents", "ai-safety", "ai-policy", "ai-infrastructure", "ai-tools"], "entities": ["Nvidia", "OpenAI", "Anthropic", "Jensen Huang", "Open Agent Safety", "OpenShell", "BlueField-4", "Cisco"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/nvidia-ropes-in-100-strong-posse-to-leash-rogue-ai-agents-after-openai-s", "markdown": "https://wpnews.pro/news/nvidia-ropes-in-100-strong-posse-to-leash-rogue-ai-agents-after-openai-s.md", "text": "https://wpnews.pro/news/nvidia-ropes-in-100-strong-posse-to-leash-rogue-ai-agents-after-openai-s.txt", "jsonld": "https://wpnews.pro/news/nvidia-ropes-in-100-strong-posse-to-leash-rogue-ai-agents-after-openai-s.jsonld"}}