# What policy makers need to know about AI safety and security > Source: > Published: 2026-07-26 00:00:00+00:00 We can all see that AI is rapidly changing the world, and it's bringing with it a whole new set of safety and security issues. Fundamentally these are technical problems, but they raise enormous policy questions about how to manage the risks associated with these new technologies. As with all technology policy questions, making good decisions requires some understanding of the technology, but it's also not practical for policy makers to read and understand what is an enormous and often highly technical literature. Rather, technologists need to find a way to turn that literature into something that is readily digestible while also retaining the key points that you need to understand in order to make good decisions. Over the past half year or so I've been working on a white paper that tries to do just that, which I'm calling "What policy makers need to know about AI safety and security" where I'm prepared to release a [working draft](https://educatedguesswork.org/assets/ai-security-policy.pdf). This isn't a finished artifact (hence "working draft") but has seen some level of external review and so is hopefully good enough that people will find it useful. If you find mistakes, please [let me know](mailto:ekr@rtfm.com). For those of you who don't want to read a 39-page PDF, here is the Executive Summary: Artificial Intelligence (AI) is rapidly changing the world. AI is an incredibly powerful set of technologies that offers a new and remarkable set of capabilities. Like most such technologies, it also comes with serious new risks. Even under non-adversarial conditions, AI systems can misbehave badly, and attackers can construct input to create specific attacker-controlled results. Moreover, AI is a dual use technology which can be used directly by attackers to generate misleading, undesirable, or dangerous content. **Existing models are already capable enough to be useful to attackers.** They can mount sophisticated cyber attacks, create convincing fake content, and make realistic explicit content of children and nonconsensual third parties. **It is not practical to prevent model misuse by sophisticated users.** Model guardrails are modestly effective for hosted models, although there is an ongoing arms race between attack and defense. Sophisticated users can use open models, which have unconstrained behavior and are already powerful enough for many applications. In order to manage the risk of misuse of these models it is necessary to focus on the downstream impacts of that misuse. **It is difficult to prevent the distribution of malicious AI-generated content.** Once malicious content is generated, it can be disseminated through existing channels. Much such content cannot be reliably detected at all, and even in cases, such as AI-generated *child sexual abuse material (CSAM)*, where some detection is possible, it is still imperfect. It may be possible to mitigate the risk of AI-based disinformation by improving mechanisms for demonstrating the provenance of legitimate content. **AI models exhibit unpredictable behavior.** Even in non-adversarial situations, they have the potential to misbehave and cause serious damage. In adversarial situations, the risk is even higher; an attacker who can provide input to the model can deliberately trigger misbehavior. It is not presently known how to eliminate this risk entirely. **Using AI models requires trusting the model provider.** Using hosted models requires sending data to the model provider and trusting that it will not misuse it and that it will provide appropriate answers. Local models are somewhat safer, but may still behave maliciously if they were trained by untrusted third parties. Ensuring the existence of both hosted and open weight models developed by trusted providers is critical for reducing the risk of malicious models. **AI models are a powerful dual-use cybersecurity tool.** AI models are very effective at finding and exploiting vulnerabilities in software. While there has been some progress in the use of AI to secure software systems, the current situation favors attackers and may continue to do so for some time. **It is not known how to completely secure agentic AI systems.** Any time an AI system is exposed to potentially malicious input—which is to say any input from a third party—its behavior becomes untrustworthy. Current best practice is to assume that the model will misbehave and attempt to limit the damage it can do. This applies even in non-adversarial settings because of the unpredictability of current models. AI technologies are far too powerful to abandon and we should not want to. Moreover, even if advancement in AI models proper were to cease today—which there is no evidence will happen—we are nowhere near finished exploring the new capabilities that current models offer, which means that we should expect to see continued deployment of new AI systems, both by legitimate and malicious users. Managing the risks presented by these technologies requires a clear understanding of the behavior of AI systems and the environments in which they are deployed. The full paper can be found [here](https://educatedguesswork.org/assets/ai-security-policy.pdf).