An open-source toolkit for controlling out-of-control AI agents

Microsoft launched an open-source Agent Governance Toolkit (AGT) to enforce policy-based controls on AI agents, preventing excessive API calls and security risks like goal hijacking or rogue behavior. The toolkit evaluates agent actions against human-readable policies in under 0.1 milliseconds before execution, addressing OWASP's top 10 agentic risks. This matters because surging agent-driven API traffic has disrupted services, and AGT provides a governance layer that treats agents as a distinct user class requiring lifecycle management.

The rapid uptake of agentic AI https://www.infoworld.com/article/4120858/agentic-ai-exposes-what-were-doing-wrong.html has exposed a range of issues with our non-deterministic helpers. That’s mainly because AI agents are not people and don’t behave like people, even though they generally use the same APIs as humans. For one thing, they make many more queries than a human would, as they build the necessary context to deliver a response. Anecdotal data from companies that have worked with agents or who have users who access services through agents indicate that this can mean massive increases in API usage https://github.blog/news-insights/company-news/an-update-on-github-availability/ , which have affected availability. This increase is the result of automated requests flooding in and blocking calls and responses from APIs that worked perfectly well a year or so ago but now are struggling to cope with the load. A fundamental redesign of our APIs is necessary, but budgets, resourcing, and capacity make this hard to deliver overnight. What’s needed, then, is a way to manage agent interactions with APIs, treating agents as a new class of user, providing and enforcing the policies that are needed to manage agent life cycles. The use of Model Context Protocol https://www.infoworld.com/article/4029634/what-is-model-context-protocol-how-mcp-bridges-ai-and-external-services.html MCP as a standard wrapper for agent access to APIs helps here, as it gives us a common environment where we can implement the governance layer needed to keep agents under control. Microsoft recently launched a public preview https://opensource.microsoft.com/blog/2026/04/02/introducing-the-agent-governance-toolkit-open-source-runtime-security-for-ai-agents/ of its open-source Agent Governance Toolkit https://github.com/microsoft/agent-governance-toolkit AGT , which is intended to wrap policy-based enforcement around agents, ensuring that calls are evaluated before they’re made. You can think of the toolkit as a way to manage agent actions, rather than controlling the inputs and outputs of the large language models https://www.infoworld.com/article/2335213/large-language-models-the-foundations-of-generative-ai.html LLMs your agents use. Figures from Microsoft suggest that this method of securing agents is far safer than relying on rules in prompts. However, in practice it’s a good idea to run a capability tool like Agent Governance Toolkit alongside traditional filters to trap user errors and prompt-based attacks. AGT is a set of tools https://microsoft.github.io/agent-governance-toolkit/ designed to cover OWASP’s list of agentic risks, building on Microsoft’s experience securing its own agents and AI platforms, with more than 13,000 tests built into the toolkit. It works by evaluating actions before they’re run, checking them against your policies, before allowing or denying the action and logging the results. Microsoft expects policy evaluation to take less than 0.1ms per operation, keeping overheads to a minimum. OWASP’s top 10 agent risks https://genai.owasp.org/resource/owasp-top-10-for-agentic-applications-for-2026/ lists the most significant issues that can disrupt agent operations resulting from user prompts and bad application design. These risks include agent goal hijacking, uncontrolled code execution, insecure output handling, and agents going rogue. Features in the toolkit are designed to protect agentic applications from these and other issues, using isolation and sandboxing, as well as validating outputs using content policies. You can use declarative programming techniques to build the policies that define the limits of your agents’ operations https://devblogs.microsoft.com/dotnet/governing-mcp-tool-calls-in-dotnet-with-the-agent-governance-toolkit/ . As the policies are human-readable, you can share them with colleagues and refine them across projects to produce a common set of AGT rules for all agents running inside your organization. The resulting authorization model is intent-based, with agents declaring their intent, which AGT approves or denies. The policy then determines what operations are allowed, whether they’re permitted with an alert, or blocked. This approach allows your code to check orchestrated operations for drift from the top-level intent. When your code is running a harness for multiple parallel agents, this helps keep the application on track and reduces the risk of unwanted operations and using too many tokens. Agent behavior often results in using more tokens than expected, so keeping spend under control is another role for AGT. As platforms move to new pricing and budgeting models, the ability to control the number of tokens that can be spent on an agent interaction will become very important indeed, and the ability to set a token budget in an AGT policy will allow developers to prevent users from significant unintended spend. AGT’s budget management tools can be used to throttle activities as agents approach preset limits, and to reject actions if they look likely to use excessive tokens. You can even use AGT to manage API calls, something that’s becoming increasingly important as agent context-seeking activities can quickly overwhelm APIs designed for human interactions. By building rules that limit the number of calls in a set amount of time, you can now have your agent framework manage throttling for you rather than implementing complex API management tools. One key feature of AGT is the ability to use its mix of declarative policies to find agents that are drifting from your set baselines, helping spot issues before they cost money or affect operations. AGT also can be used to apply kill switches where necessary. Logging and observability capabilities include a way to provide a root cause analysis of issues, along with what Microsoft calls a “decision bill of materials” that keeps track of governance decisions with an audit chain and the details of the trust levels associated with agents. A toolkit like AGT needs to be vendor neutral, as your agents could be working with models running in any environment. AGT is designed to work with Azure Foundry, Amazon Bedrock, and Google ADK, as well as with most common agent orchestration frameworks. AGT supports five different programming languages: Python, TypeScript, .NET, Rust, and Go. While capabilities vary between the different implementations, the Python version has the full set. Microsoft has made some interesting architectural decisions in AGT. Perhaps the most important is that it treats agents as code running on a secure operating system, using concepts from hypervisors to isolate agents from the underlying platform. You can see this in the way the underlying components and packages are named; the core governance package is called Agent OS. Getting started with AGT https://microsoft.github.io/agent-governance-toolkit/quickstart/ can be as simple as installing the complete toolkit using standard package manager of your language of choice. You don’t need to install the whole thing. You have the option of installing specific packages when you don’t need all the features, or when you only want to use a single function. You could download only AGT’s software reliability engineering tools, for example, when you want to put your agents through chaos testing. You can define policies inside your code or as external policy documents. Policies are focused on key security and governance goals, such as blocking dangerous tools from running, looking for PII in messages, and showing users what action has been taken and why. External policy documents are recommended for use with production AGT implementations, as they let you author policies in YAML. The toolkit provides a single call to load all the policy files in a directory, so it’s a good idea to store only production policies in that location to avoid issues. Adding support for AGT in existing code is a matter of adding evaluations before your agent framework makes a call. This can be handled using refactoring tools in your editor, finding the call and wrapping the evaluator and decision results around it. This way you can develop agents without using AGT, adding its tools when you’re confident you have a working agent. Microsoft provides custom adapters for 19 different frameworks https://microsoft.github.io/agent-governance-toolkit/packages/ framework-integrations-19 , tuned to work with them. These can be loaded alongside the rest of the toolkit. AGT is a sign of growing maturity in our agent landscape, applying enterprise rigor to what has been something of a Wild West free-for-all. It addresses many of the concerns associated with using agents at scale, helping control both agent behavior and costs. Autonomous software needs to be kept on a leash, and AGT’s declarative approach to building and applying policies — along with the rest of its suite of tools — gives you the ability to quickly build and apply the necessary controls. With companies looking to control costs and regulators wanting to ensure that agentic software doesn’t violate compliance requirements, tools like the Agent Governance Toolkit will likely become an essential part of our modern agent development environment.