Open Secure AI Alliance proposes SAFE guidelines as membership tops 120
The Open Secure AI Alliance today proposed a set of guidelines for reporting cybersecurity incidents involving artificial intelligence agents, one week after the group was formed.
The proposal is called Shared AI Findings Exchange, or SAFE, and was published as a request for comments by the Linux Foundation. Nvidia Corp., Cisco Systems Inc., CrowdStrike Holdings Inc., Hugging Face Inc. and Red Hat Inc. led the drafting. Comments are being taken on GitHub.
SAFE would give organizations a confidential channel for handing over details of AI security incidents, agent misbehavior and operational near misses. The alliance would then analyze what it receives, notify the parties affected and flag control failures that keep recurring across its membership. Recommendations would follow, based on the incident evidence rather than on vendor guidance.
The alliance launched on July 27 with roughly two dozen founding members and now counts more than 120 organizations. Adobe Inc. and Cloudflare Inc. were in at the start. BlackRock Inc., Capital One Financial Corp., Intel Corp. and Visa Inc. are also on the roster. Anthropic PBC, OpenAI Group PBC and Google LLC have not joined.
Its formation followed OpenAI’s disclosure on July 21 that two of its models had escaped a sandbox during an internal cyber capability test and used stolen credentials and zero-day exploits to pull test answers off Hugging Face servers. Anthropic followed on July 31 with a separate account of three models attacking targets during evaluations that a configuration error had left connected to the internet. One of the attacks spread to a real cybersecurity company’s infrastructure.
Members used the Black Hat conference in Las Vegas to detail what code they are handing over. From Okta Inc. come agent identity implementations built on its Cross App Access protocol. Palo Alto Networks Inc. brought two tools, Agent Guard and Agent Watch. The Cedar authorization language and the Strands Agents toolkit are Amazon.com Inc.’s contributions. Microsoft Corp. added PyRIT, its Python risk identification toolkit, plus three other projects. Red Hat is supplying asago, a project that maps written policy to runtime governance.
Uber Technologies Inc. handed over ADR, short for agentic AI detection and response. The production system reconstructs the full causal chain of what an agent did, and Uber runs it across more than 200,000 agent sessions a day.
Nvidia’s contributions include Garak, an open-source vulnerability scanner for large language models, and OpenShell, a runtime that restricts what an agent can see, touch and do. The chipmaker also put its Labs Object-Oriented Agent research harness on GitHub for testing and auditing. Verified agent skills are being released as well. Each is cryptographically signed, scanned for risks including prompt injection and tool poisoning and shipped with a documented skill card.
Frank Dickson, group vice president for the security and trust research practice at International Data Corp., said the alliance carries forward the work of Project Glasswing. The focus has shifted from immediate threats to building secure foundations, he said. On validation, he was less certain.
“The whole model is a little bit fraught, because open source is contributed, open source is openly managed…how do you validate that?” Dickson told IT Brew. Code-signing efforts may help, he said, though open-source projects run largely on volunteers.
Image: Nvidia
Support our mission to keep content open and free by engaging with theCUBE community. Join theCUBE’s Alumni Trust Network, where technology leaders connect, share intelligence and create opportunities.
15M+ viewers of theCUBE videos, powering conversations across AI, cloud, cybersecurity and more** 11.4k+ theCUBE alumni**— Connect with more than 11,400 tech and business leaders shaping the future through a unique trusted-based network.
About SiliconANGLE Media
theCUBE AIand theCUBE SuperStudios — with flagship locations in Silicon Valley and the New York Stock Exchange — SiliconANGLE Media operates at the intersection of media, technology and AI.
Founded by tech visionaries John Furrier and Dave Vellante, SiliconANGLE Media has built a dynamic ecosystem of industry-leading digital media brands that reach 15+ million elite tech professionals. Our new proprietary theCUBE AI Video Cloud is breaking ground in audience interaction, leveraging theCUBEai.com neural network to help technology companies make data-driven decisions and stay at the forefront of industry conversations.