{"slug": "i-stopped-trusting-ai-agents-with-tools-so-i-built-a-gatekeeper", "title": "I Stopped Trusting AI Agents With Tools. So I Built a Gatekeeper.", "summary": "A developer has released Agent ToolTrust, an open-source contextual risk and permission engine that gates AI agent tool calls through a five-stage pipeline. The project, tested against 83 real agents across 10 frameworks with 2,490 passing tests, aims to replace binary allow/deny permissions with four decision states: allow, audit, escalate, or deny. The developer reports that about 18% of MCP server deployments implement access scoping and 80% of organizations admit agents have acted beyond intended scope.", "body_md": "[github.com/deghosal-2026/agent-tooltrust]·`pip install agent-tooltrust`\n\n·[field test report]·[design decisions]\n\nMy last three projects taught me the same thing. Mock agents lie. Unit tests pass. Demos look clean. Then real agents run and everything breaks.\n\nOn my eval harness, I admitted it: field testing \"got added ad hoc, late in the build, because I started getting nervous that unit tests and mock agents were hiding real integration problems.\" On my observability tool: \"I thought it was a detector problem. I was wrong.\"\n\nSame lesson. Three times. But lessons only matter if you change what you do next.\n\nSo this time I did the opposite. Zero mock agents. 83 real ones across 10 frameworks. A covering design that cut a 12-day test matrix into one afternoon. And a release gate that said: no ship until real agents prove the policy works.\n\nIt worked. 2,490 tests green. 83/83 agents passed. PyPI published. Repo public. And the 7 failures taught me something I couldn't have learned any other way.\n\nEveryone is racing to give AI agents more tools. Almost no one is building the permission system that decides when those tools should fire.\n\nRight now, agent permissions are binary: allowed or denied. That's reachability, not authorization. The same tool is harmless in staging and dangerous in production. The same read is fine on public docs and risky on customer data. A `delete`\n\nin a CI sandbox is not the same as `delete`\n\nin production.\n\nAbout 18% of MCP server deployments implement any access scoping. 80% of orgs admit agents have taken actions beyond intended scope. OWASP classifies agent tool misuse as a first-class risk.\n\nGiving an agent tools is the easy part. The hard part is deciding what it should be allowed to do, where, and under what guardrails. I wrote a [PRD](https://github.com/deghosal-2026/agent-tooltrust/blob/main/docs/design/PRD.md) and [architecture spec](https://github.com/deghosal-2026/agent-tooltrust/blob/main/docs/architecture/architecture-v0.1.0.md) before touching engine code — partly to keep myself honest, partly because I've learned the hard way that skipping design leads to shipping the wrong thing.\n\nAgent ToolTrust is a contextual risk and permission engine. Before an agent's tool call executes, the engine runs a five-stage pipeline — normalize, score, decide, explain, audit — and returns one of four decisions: allow, audit, escalate, or deny.\n\n``` python\nfrom agent_tooltrust.engine.engine import Engine\nfrom agent_tooltrust.policy.models import default_policy\nfrom agent_tooltrust.adapters.raw import RawAdapter\n\nengine = Engine(default_policy(\"balanced\"))\nadapter = RawAdapter(engine)\n\n@adapter.guard(\n    tool_name=\"deploy_service\",\n    action=\"deploy\",\n    environment=\"production\",\n    data_class=\"restricted\",\n)\ndef deploy_service(service: str) -> str:\n    return f\"deployed {service}\"\n\n# Agent calls the tool. Engine evaluates first.\n# production deploy on restricted data → escalate\ndeploy_service(\"payment-api\")\n# ToolTrustDecisionError: escalate — \"Write action (deploy) in production\n# on restricted data requires approval...\"\n```\n\nThe decorator is the integration point. The agent calls the tool. The engine intercepts, evaluates, and either lets it through, audits it, escalates to a human, or denies it. The agent never sees the policy. The LLM never knows the rules exist.\n\nThe engine is deterministic. The LLM proposes, policy disposes. No amount of prompt engineering can override a deny — because the engine is outside the model, not inside the prompt.\n\nFour decisions, not two. `allow`\n\nand `deny`\n\nare obvious. `audit`\n\nmeans \"allow but log everything — this is a read on sensitive data.\" `escalate`\n\nmeans \"stop and get a human.\" Binary allow/deny forces you to choose between over-privileged agents and approval fatigue. Four states give you a middle ground.\n\nEvery decision comes with an explanation — a reason code, a human sentence, and a factor breakdown showing which dimension drove the call. Optional LLM prose, off by default. The LLM cannot change the decision.\n\nEvery decision is audited — JSONL, SQLite, or Postgres, with policy version, timestamp, and session ID.\n\nThree posture presets ship out of the box — strict, balanced, permissive — so no one starts from a blank file. YAML policy backend for humans, OPA/Rego backend for teams that already have Rego policies. Shadow mode so you can deploy, observe what would have been denied, tune, then enforce — without changing agent code.\n\nFail-closed everywhere. Unknown tool → deny. Malformed input → deny. Engine crash → deny. The alternative is fail-open, which means an attacker who can crash the engine gets unrestricted tool access. That's [design decision DD-14](https://github.com/deghosal-2026/agent-tooltrust/blob/main/docs/design/design-decisions.md) — written before the first line of code, not retrofitted after a near-miss.\n\nThat's the architecture. But architecture is the easy part. Does it actually work when real agents try to use it?\n\nOn previous projects, the field test was the thing I skipped and regretted. On EvalForge, I added it late and discovered the pass rate was 9% — not because the tool was bad, but because mock agents had hidden every integration problem. On AgentObservatory, I learned that \"the integration, not the judge, broke me.\"\n\nThis time, I put it in the spec before writing any adapter code. [DD-11](https://github.com/deghosal-2026/agent-tooltrust/blob/main/docs/design/design-decisions.md): \"Field tests must pass before any release. They run real agents, not mocks.\" DD-12: \"8-10 real agents across major platforms.\"\n\nI went further than both. Not 8-10 agents. 83 real agents across 10 frameworks. And the [field test plan](https://github.com/deghosal-2026/agent-tooltrust/blob/main/docs/field-test/field-test-plan.md) was in the WBS from day one.\n\nThis is the difference between learning a lesson and applying one.\n\nI wanted this to work across the real agent ecosystem, not just one framework I happened to know. So I built adapters for 10 frameworks:\n\n**LangGraph, PydanticAI, CrewAI, OpenAI Agents SDK, Google ADK, AutoGen/AG2, LlamaIndex, smolagents, SWE-bench (self-test), ToolTrust MCP** (self-test).\n\nEvery adapter follows the same contract — extract a `CallContext`\n\n, forward it to `Engine.evaluate()`\n\n, surface the decision back:\n\n```\n@dataclass(frozen=True)\nclass CallContext:\n    tool_name: str\n    action: str\n    environment: str\n    data_class: str\n    agent_id: str\n    session_id: str | None = None\n    arguments: dict[str, Any] | None = None\n```\n\nThe contract is clean. Getting there was not.\n\nEach framework has its own opinions about how tools are registered, how they're invoked, and how errors surface. I'd write the adapter, run it against a real agent, watch it fail in some framework-specific way, fix it, and repeat. Every failure taught me something about how that framework actually works — not how the docs describe it, but how it behaves when a real agent is driving it. The full per-framework wiring notes are in [§5 of the field test report](https://github.com/deghosal-2026/agent-tooltrust/blob/main/docs/field-test/FIELD_TEST_REPORT.md) — 12 separate learnings.\n\nLangGraph's `ToolTrustToolNode`\n\nsubclasses `ToolNode`\n\nand overrides `_run_one()`\n\n. But in langgraph v1.x, the node isn't callable — so I fell back to wrapping the tool before it enters the graph:\n\n```\n# LangGraph — wrap the tool, then hand it to the graph\nadapter = RawAdapter(engine)\nguarded_tool = adapter.guard(\n    tool_name=\"query_logs\",\n    action=\"read\",\n    environment=\"staging\",\n    data_class=\"internal\",\n)(query_logs_fn)\n\n# Now hand guarded_tool to create_react_agent(llm, tools=[guarded_tool])\n```\n\nGoogle ADK's LLM registry only knows about Gemini. To use a local model, you pass `LiteLlm(model=f\"openai/{MODEL}\", api_base=ENDPOINT)`\n\n. And `InMemorySessionService.create_session()`\n\nis a coroutine — you have to `await`\n\nit, not call it synchronously. The docs don't mention this. The runtime teaches you.\n\nLlamaIndex's legacy `ReActAgent`\n\nhas no `.query()`\n\nor `.chat()`\n\n. You need the workflow agent from `llama_index.core.agent.workflow`\n\n. And execution is driven by `async for event in handler.stream_events()`\n\n— a separate `await handler`\n\nyields nothing. The `async for`\n\nis what drives the agent forward. Without it, the agent silently does nothing. I spent an hour on that.\n\nAutoGen needs hyphens sanitized from agent IDs (`ag-01`\n\n→ `ag_01`\n\n). The local Qwen model answers textually unless you tell it: \"you MUST call the tool exactly named `scn_<id>`\n\n. Do not skip the tool call.\"\n\nsmolagents requires full docstrings with per-arg descriptions on every `@tool`\n\n— or it throws `DocstringParsingException`\n\n. CrewAI needs `litellm`\n\nas a fallback. OpenAI Agents SDK needs `function_tool(..., strict_mode=False)`\n\nto fix a pydantic conflict.\n\nNone of these show up with mock agents. They only surface when you run real code from real repos. And every one I fixed made the adapter stronger.\n\nBy the end, all 10 frameworks built, recorded decisions, and ran real agents through the engine. Ten frameworks where the interception point is proven, not theoretical.\n\nI sourced 83 real agents from GitHub. Not toy examples — real repos with real dependencies, real packaging, real opinions about how to invoke an LLM.\n\nI wrote 30 scenarios: 20 decision scenarios covering all four decision types across 5 agent classes (ci-bot, engineer, general, analyst, sensitive), plus 10 adversarial scenarios — prompt injection, Unicode obfuscation, replay attempts, blank tool names, malformed inputs, grant-bypass attempts. The full matrix is in the [field test report](https://github.com/deghosal-2026/agent-tooltrust/blob/main/docs/field-test/FIELD_TEST_REPORT.md) — every agent, every scenario, every expected and actual decision.\n\nThe math: 83 agents × 30 scenarios = 2,490 runs. Each run calls a local LLM — `Qwen3.5-4B-4bit`\n\nvia OMLX on Apple Silicon. Each call takes 30-80 seconds. That's roughly 2.7 hours at 10 workers.\n\nBut 2,490 is the theoretical minimum. In practice, you debug. Adapters break. Agents fail to import. The LLM answers textually instead of calling a tool. You fix, re-run, fix again. The actual number of LLM calls was 4-5x higher — over 10,000 calls to a local 4B model.\n\nThis is the cost of zero mock agents. I'd pay it again.\n\nMock agents don't need an LLM. They don't take 80 seconds. They don't bring a C extension with the wrong ABI. They don't hardcode API keys at module scope. They don't write to `/root`\n\nat import time.\n\nReal agents do all of that. And every one of those failures is a bug that would have shipped if I'd used mocks.\n\nHere's where I stopped brute-forcing. 2,490 runs through a local 4B model to re-prove what deterministic tests already cover made no sense. Engine correctness was already validated — 2,490 assertions, zero LLM calls, 100% green. The engine is framework-agnostic. `Engine.evaluate()`\n\ndoesn't care whether the caller is LangGraph or CrewAI. Re-running every cell was redundant.\n\nThe field test's real job was adapter proof — does each framework correctly surface allow, audit, escalate, and deny in a real agent loop? That's a covering problem, not a cross-product problem.\n\nSo I split it into two plans.\n\n**Plan A — one scenario per agent (83 runs).** Each agent gets exactly one scenario. The assignment covers all 30 scenarios, all 10 frameworks, all 5 agent classes. Result: **83/83, 100%.**\n\n**Plan B — per-framework decision-type proof (123 runs).** For each framework, run a tier-1 agent against all 4 decision types plus adversarial scenarios. Result: **116/123, 94%.**\n\nTogether: **206 runs instead of 2,490. Same coverage — 30/30 scenarios, 83/83 agents, 10/10 frameworks, 5/5 classes. ~12x reduction.** The full coverage rationale is in [§8 of the field test report](https://github.com/deghosal-2026/agent-tooltrust/blob/main/docs/field-test/FIELD_TEST_REPORT.md).\n\nThis is the part I'm most proud of. Not the engine — that's straightforward. The covering design. The recognition that expensive LLM calls should be spent on what only real agents can prove, not on re-proving what deterministic tests already cover.\n\nThe full cross product would have taken ~12 days. The covering design took one afternoon. Same confidence. On previous projects, I would have either skipped the field test or brute-forced it and run out of time. This time, I optimized.\n\nThe 7 Plan B failures were the most valuable part of the field test. Not because they broke something — because they revealed something no mock would have caught.\n\nAll 7 shared one pattern: `not-available`\n\n. The guard never fired because the LLM didn't call the tool. The local Qwen model, when given 5 tools at once, sometimes answered textually instead of invoking the guarded tool. The engine never got a chance to decide.\n\nA mock agent always calls the tool. A real 4B model sometimes doesn't.\n\n**Never interpret not-available as a policy failure.** It means the LLM didn't call the tool. That's different from\n\n`unexpected-decision`\n\n— when the guard ran and the engine made the wrong call. Only the latter is a real regression.Every tool call that actually executed in Plan B produced the correct decision. The 7 failures pointed at the LLM, not the engine.\n\nThis matters for CI. If you fail on `not-available`\n\n, your gate is flaky because of model nondeterminism. If you fail only on `unexpected-decision`\n\n, your gate is strict but stable. The [field test report](https://github.com/deghosal-2026/agent-tooltrust/blob/main/docs/field-test/FIELD_TEST_REPORT.md) recommends committing a golden `not-available`\n\nallowance so CI fails on real regressions, not on the LLM having a bad day.\n\nI couldn't have learned this with mocks. It took 83 real agents.\n\nThe result I'm most satisfied with: the deny → replan → allow safety loop.\n\nWhen an agent tries `drop_database`\n\n, the engine denies it. A good agent doesn't just stop — it replans. It picks a different, benign tool. The engine allows it. Both calls are audited.\n\n```\n@adapter.guard(\n    tool_name=\"drop_database\",\n    action=\"delete\",\n    environment=\"production\",\n    data_class=\"restricted\",\n)\ndef drop_database(db: str) -> str:\n    return f\"dropped {db}\"\n\n@adapter.guard(\n    tool_name=\"query_audit_log\",\n    action=\"read\",\n    environment=\"production\",\n    data_class=\"internal\",\n)\ndef query_audit_log(query: str) -> str:\n    return f\"audit rows for {query}\"\n\n# Agent tries drop_database → engine denies (delete in production)\n# Agent replans → calls query_audit_log → engine allows (read in production)\n# Both decisions audited. Agent redirected, not blocked.\n```\n\nTested across all 8 LLM frameworks. **8/8 live, 8/8 scripted.** Every framework denied the destructive call, replanned to a benign read, and got an allow. The full replan results are in [§2.5 of the field test report](https://github.com/deghosal-2026/agent-tooltrust/blob/main/docs/field-test/FIELD_TEST_REPORT.md).\n\nThe agent isn't blocked — it's redirected. And every step is on the audit trail. This is the pattern I'll build on in v0.2: the escalation round-trip, where a human approves or denies, and the agent resumes.\n\nOn previous projects, I learned that field testing should be planned, not improvised. This time I learned something deeper: **it should be optimized, not brute-forced.**\n\nThe covering design — Plan A + Plan B — is the application of that learning. 206 runs instead of 2,490. Same coverage. The expensive resource spent on what only real agents can prove. The progression: skip it → add it late → plan it from the start → optimize it. Four projects, four steps.\n\n**Zero mock agents is the right call.** The integration cost is real — 8 compatibility wrappers, 3 pyproject fixes, 1 quarantined C extension, 12 framework quirks [documented in the report](https://github.com/deghosal-2026/agent-tooltrust/blob/main/docs/field-test/FIELD_TEST_REPORT.md). But every fix caught a real bug that a mock would have hidden. The cost of mocks is invisible until production. The cost of real agents is visible from the first run.\n\n** not-available is not a policy failure — it's an LLM reliability signal.** Distinguishing it from\n\n`unexpected-decision`\n\nis the difference between a flaky CI gate and a strict one.**Fail-closed everywhere is non-negotiable.** [DD-14](https://github.com/deghosal-2026/agent-tooltrust/blob/main/docs/design/design-decisions.md), written before the first line of code.\n\n**The replan loop works at scale.** 8/8 frameworks. The agent isn't blocked — it's redirected.\n\n**10 frameworks is the right number for v0.1.** Enough to prove the adapter contract generalizes. Not so many that integration drowns the engine. The 8 that needed LLM calls all passed. The 2 self-test frameworks ran deterministically in CI.\n\n**Building agents with tools?** `pip install agent-tooltrust`\n\n, run `tooltrust init --posture balanced`\n\n, decorate your tools. Four-state decisions with explanations and audit trails. No infrastructure. The [quickstart](https://github.com/deghosal-2026/agent-tooltrust/blob/main/docs/reference/quickstart.md) walks through it.\n\n```\npip install agent-tooltrust\ntooltrust init --posture balanced\n```\n\n**Have existing OPA/Rego policies?** The dual backend reuses them. Same input, same output. No rewrite. The [API reference](https://github.com/deghosal-2026/agent-tooltrust/blob/main/docs/reference/api.md) covers both.\n\n**Want to observe before enforcing?** Shadow mode (`dry_run=True`\n\n) logs every decision without blocking. Deploy, observe, tune, enforce.\n\n**Using LangGraph, PydanticAI, CrewAI, OpenAI Agents SDK, Google ADK, AutoGen, LlamaIndex, or smolagents?** There's an adapter tested against real agents. The [integration guide](https://github.com/deghosal-2026/agent-tooltrust/blob/main/docs/real-agent-integration/README.md) has per-framework wiring.\n\n**Custom risk functions** — register your own with `@tooltrust.risk_function`\n\n, plug into the weighted sum, don't touch the engine.\n\n**Community policy packs** — map a tool ecosystem (GitHub admin, AWS cost ops, Notion writes) onto the taxonomy. `tooltrust pack validate`\n\n, `tooltrust pack add`\n\n.\n\n**New audit sinks** — the `AuditSink`\n\ninterface is pluggable. Splunk, Datadog, whatever your SIEM is.\n\n**New framework adapters** — `BaseAdapter`\n\nis three methods. Extract the context, forward to the engine, surface the decision. Maybe 50 lines. The pattern is proven across 10 frameworks.\n\n**Custom posture presets** — the three shipped presets are YAML files. Fork one, tune thresholds, ship your org's default.\n\nThe [architecture doc](https://github.com/deghosal-2026/agent-tooltrust/blob/main/docs/architecture/architecture-v0.1.0.md) and [14 design decisions](https://github.com/deghosal-2026/agent-tooltrust/blob/main/docs/design/design-decisions.md) are in the repo if you want the internals.\n\nRepo: [github.com/deghosal-2026/agent-tooltrust](https://github.com/deghosal-2026/agent-tooltrust)\n\nPyPI: `pip install agent-tooltrust`\n\n(v0.1.1)\n\n2,490 deterministic tests. Ruff 0. Mypy strict 0. Docker pass. SWE-bench verified. OWASP 5/10. OpenSSF Silver. 10 frameworks, 83 real agents, 30 scenarios, zero mocks. 83/83 Plan A. 116/123 Plan B. 8/8 replan loop. Branch protected. Repo public.\n\nThis is v0.1.0. The engine is shipped and proven. The platform — escalation round-trip, policy packs, rule composition, HTTP /authorize, replay detection, child-agent delegation — is 32 open issues for v0.2.0. The [WBS](https://github.com/deghosal-2026/agent-tooltrust/blob/main/docs/wbs/README.md) tracks it all. The [field test report](https://github.com/deghosal-2026/agent-tooltrust/blob/main/docs/field-test/FIELD_TEST_REPORT.md) is honest about what's proven and what's not.\n\nWhat happens when your agent tries to call a tool it shouldn't? Does your system know the difference between a read in staging and a write in production? Or are you using an allow-list and hoping?\n\nIf you've field-tested agents across multiple frameworks, what broke first — the policy, the adapter, or the LLM? Did mocks hide problems that surfaced later?\n\nHas anyone hit the `not-available`\n\nproblem — the LLM doesn't call the tool and you can't tell if it's a policy failure or a model issue? How do you handle it in CI?\n\nIs four-state (allow/audit/escalate/deny) the right granularity, or overkill compared to binary? The `audit`\n\nstate was the one I wasn't sure about.\n\nFor OPA/Rego users — does dual-backend (YAML + Rego) make sense, or would you rather have Rego-only?\n\nThe covering design cut the matrix 12x. Has anyone else applied combinatorial testing to LLM-based agent testing? I haven't seen this pattern elsewhere — is it novel or just underdocumented?\n\nThe [repo](https://github.com/deghosal-2026/agent-tooltrust) has the full [PRD](https://github.com/deghosal-2026/agent-tooltrust/blob/main/docs/design/PRD.md), [architecture](https://github.com/deghosal-2026/agent-tooltrust/blob/main/docs/architecture/architecture-v0.1.0.md), [design decisions](https://github.com/deghosal-2026/agent-tooltrust/blob/main/docs/design/design-decisions.md), [field test plan](https://github.com/deghosal-2026/agent-tooltrust/blob/main/docs/field-test/field-test-plan.md), and [field test report](https://github.com/deghosal-2026/agent-tooltrust/blob/main/docs/field-test/FIELD_TEST_REPORT.md). Star it, fork it, break it. I'd rather you break it now than after you ship it to production.", "url": "https://wpnews.pro/news/i-stopped-trusting-ai-agents-with-tools-so-i-built-a-gatekeeper", "canonical_source": "https://dev.to/debashish_ghosal/i-stopped-trusting-ai-agents-with-tools-so-i-built-a-gatekeeper-26fb", "published_at": "2026-08-13 06:43:24+00:00", "updated_at": "2026-08-13 07:16:02.208014+00:00", "lang": "en", "topics": ["ai-agents", "ai-safety", "ai-tools", "ai-infrastructure", "developer-tools"], "entities": ["Agent ToolTrust", "MCP", "OWASP", "PyPI", "GitHub"], "alternates": {"html": "https://wpnews.pro/news/i-stopped-trusting-ai-agents-with-tools-so-i-built-a-gatekeeper", "markdown": "https://wpnews.pro/news/i-stopped-trusting-ai-agents-with-tools-so-i-built-a-gatekeeper.md", "text": "https://wpnews.pro/news/i-stopped-trusting-ai-agents-with-tools-so-i-built-a-gatekeeper.txt", "jsonld": "https://wpnews.pro/news/i-stopped-trusting-ai-agents-with-tools-so-i-built-a-gatekeeper.jsonld"}}