{"slug": "show-hn-an-open-source-safety-layer-for-ai-agent-actions", "title": "Show HN: An open source safety layer for AI agent actions", "summary": "An open source Python library called CTRLRun, released under the Apache-2.0 license, adds a safety layer that checks every AI agent action against user-defined rules before it executes, allowing permitted actions, holding sensitive ones for human approval, and blocking forbidden ones. CTRLRun's demo shows five failure modes and refusals, including a €500 refund that becomes a €5,000 request (€4,500 too much) and a lost-response retry that is marked AMBIGUOUS rather than FAILED and refused until a human or a reconcile hook resolves it. The library runs on a single file in production or on Postgres across hosts and is installed via pip install ctrlrun.", "body_md": "**CTRLRun stops AI agents from taking wrong, restricted, or malicious actions in your workflows.**\n\n  Every action is checked against your rules before it runs. Allowed actions go through.\n\n  Sensitive ones wait for a person. Forbidden ones are blocked.\n\n  Execution safety for AI agents. A Python library that sits between the decision to act and the call that acts.\n\n  A consequential action happens at most once, exactly as approved, and leaves a receipt.\n\n  When the outcome is unknown, CTRLRun says so instead of guessing.\n\n  Runs in production on a single file, or on Postgres across hosts. Apache-2.0.\n\n```\npip install ctrlrun && ctrlrun demo\n```\n\n**The model guesses. CTRLRun does not.** The ticket says refund €500. The agent asks for\n€5,000, one extra zero. The tool is in its list, the arguments are well formed, and the model\nis completely confident. Nothing above the call disagrees, because nothing above the call is a\ncheck: a tool being callable is not permission to call it with those arguments.\n\n| Without CTRLRun | With CTRLRun | \n|---|---|\n| Nothing checks the amount. The call goes through. **€4,500 too much.** | Your rule checks the amount. The call never leaves. **€0 wrongly paid.** | \n\nCTRLRun is that check. It reads the arguments about to leave your process and answers what may happen to them: let it run, ask a human, or stop it cold. Four rules do the work, and each one is a test in this repository before it is a sentence here.\n\n| **Exact means exact** | Changed arguments need a new approval. | \n| **Once stays once** | Same effect key, shared store, no repeat. | \n| **Unknown means wait** | Confirm the outcome before retrying. | \n| **Every answer is kept** | Requests, decisions and results, refusals included. | \n\nThe third one is the half people forget. A correct €500 refund commits at the provider and the\nreply is lost coming back, so the agent retries. Retry libraries, agent frameworks and tool\nloops collapse *this failed* into *I do not know what happened*. CTRLRun keeps them apart: a\nlost reply is `AMBIGUOUS`, never `FAILED`, and a retry against an `AMBIGUOUS` effect is refused\nuntil a human, or a `reconcile` hook, says what happened.\n\n## What `ctrlrun demo` shows: five failures and five refusals, byte for byte\n\n``` bash\n$ ctrlrun demo\nCTRLRun demo — five ways an agent action goes wrong, and what stops it.\nPolicy: refunds up to €1,000 are autonomous, up to €10,000 need a human, above that are denied.\n\n1. Duplicate effect after a lost response\n\n   refund €500  →  remote commits  →  response lost  →  effect: AMBIGUOUS\n   agent retries the same refund\n   ✗ BLOCKED — effect may already have committed; blind retry refused\n   remote refund calls: 1\n   only a human moves it on:  ctrlrun resolve refund:txn_1 --committed|--failed\n\n2. Approval mutation\n\n   agent proposes refund €2,000  →  human approves apr_0aa78e0380ba55d77a601dc782f57095 (bound to the action hash)\n   agent executes refund €5,000  →\n   ✗ BLOCKED — approved action ≠ requested action (mismatch)\n\n3. Concurrent agents, same effect\n\n   Agent A  reserve refund:txn_123  →  ACQUIRED  →  executes\n   Agent B  reserve refund:txn_123  →\n   ✗ BLOCKED — already reserved (in_progress)\n\n4. Approval replay\n\n   approval apr_dbc8bc6f06690cdf2e2c55a4e591ef3b used once  →  consumed\n   same approval presented again                            →\n   ✗ BLOCKED — single-use approval already consumed\n\n5. Authority escalation\n\n   human €100,000 delegable  →  finance agent €25,000  →  support agent €2,000\n   support agent's grant: dlg_5f8d41938a3f29972d5489d676cd9edb\n   support agent requests €50,000  →\n   ✗ BLOCKED — outside the delegated grant (authority_constraint)\n   remote refund calls: 0\n   finance agent tries to delegate €50,000 under its own €25,000  →  refused (containment: constraints)\n   support agent requests €1,500  →  authority permits it, and the policy asks a human (apr_f86eca24dd80206ab5189ccb1b62aa55)\n   two axes, and an action needs both: the stricter of the pair wins\n\nReceipts (8): .ctrlrun/demo/receipts.jsonl\nEvents:       .ctrlrun/demo/events.jsonl\n\nRead them:    CTRLRUN_STATE=.ctrlrun/demo/state.db ctrlrun receipts\n```\n\nApproval and delegation ids are generated per run; everything else is exactly what the demo\nprints, and a test fails if the two drift apart. No network, no external service, under a\nsecond. `pip install ctrlrun && ctrlrun demo` runs it locally in about the same time.\n\n**Where it stops.** It does not detect prompt injection: it contains the consequence rather\nthan reading the cause. It cannot promise exactly-once against a remote it does not control, it\nrefuses to *knowingly* act twice, and it rolls nothing back. Receipts are chained, so an alteration\nis detected; a truncation at the end and a forged append are not, because the head that would catch\nthem is a row in the same database, and closing that is what `ctrlrun anchor` is for. They are not\nsigned: alteration is not authorship. The badge above means the\n**declared guarantees pass** in the setup they ran against, and it does not mean secure, safe,\ncompliant, certified or audited:\n[what the badge means](https://docs.ctrlrun.dev/verify#what-the-badge-means)\n· [`OWASP-AGENTIC-TOP10.md`](https://docs.ctrlrun.dev/OWASP-AGENTIC-TOP10)\nnames the four entries this does not address.\n\nIf an agent only reads and answers, you do not need CTRLRun. The moment it can **send, pay,\nrefund, delete, deploy, grant, revoke, approve, submit, purchase or cancel**, you do.\n\nThe animation above is this section, recorded against the real library: one policy file, two short programs, four commands, nothing staged.\n\n**1. Install it.**\n\n```\npip install ctrlrun\n```\n\n**2. Write down what the agent may do.** One file, `ctrlrun.yaml`. Amounts are integer minor\nunits, so `50000` is €500. Both ends of every band are bound, because an upper bound alone lets\na negative amount through, and a refund of a negative amount is a charge. Anything not listed is\ndenied; there is no default-allow.\n\n```\nschema: ctrlrun.policy/v2\n\nactions:\n  stripe.refund:\n    effect: \"refund:{payment_id}\"\n    rules:\n      - when: { amount_gte: 0, amount_lte: 50000 }\n        decision: allow      # up to €500: the agent acts alone\n      - when: { amount_gte: 0, amount_lte: 1000000 }\n        decision: approve    # up to €10,000: a human decides\n      - decision: deny       # above that: never\n```\n\n**3. Wrap the call that has the consequence.** The decorator names the action, the effect key\nnames the consequence it has in the world, and the context names who is acting. `stripe` here\nis a stand-in that records calls instead of making them.\n\n``` python\nimport sys\n\nimport ctrlrun\n\nclass FakeStripe:\n    \"\"\"Stands in for the provider: it records calls instead of making them.\"\"\"\n\n    def __init__(self) -> None:\n        self.calls: list[tuple[str, int]] = []\n\n    def refund(self, payment_id: str, amount: int) -> dict:\n        self.calls.append((payment_id, amount))\n        return {\"id\": f\"re_{payment_id}\", \"amount\": amount, \"status\": \"succeeded\"}\n\nstripe = FakeStripe()\n\n@ctrlrun.protect(\"stripe.refund\", effect=\"refund:{payment_id}\")\ndef refund(payment_id: str, amount: int) -> dict:\n    return stripe.refund(payment_id, amount)\n\nif __name__ == \"__main__\":\n    with ctrlrun.context(agent=\"support-agent\"):\n        print(\"€500   ->\", refund(payment_id=\"txn_1\", amount=50_000)[\"status\"])\n        try:\n            refund(payment_id=\"txn_2\", amount=500_000)\n        except ctrlrun.ApprovalRequired as pending:\n            print(\"€5,000 -> a human decides:\", pending.request_id)\n            with open(\"request_id.txt\", \"w\") as handle:\n                handle.write(pending.request_id)\n        else:\n            sys.exit(\"the €5,000 refund ran without a human; the policy is not in force\")\n    print(\"calls that reached the provider:\", len(stripe.calls))\n```\n\nThe €500 refund runs on its own. The €5,000 one stops and names the request a human answers:\n\n``` php\n€500   -> succeeded\n€5,000 -> a human decides: apr_63e80076f2cccfee52b17491a4b2e125\ncalls that reached the provider: 1\n```\n\n**A human answers from the shell.** The grant names the hash of the exact action the human\nread, and when it lapses. Ids, hashes and dates are generated per run; yours differ.\n\n```\nctrlrun approve \"$(cat request_id.txt)\"\ngranted apr_63e80076f2cccfee52b17491a4b2e125 for sha256:22ec1c398e4b93d080b6cba61e5e11b0e21879552ac5dbf63c192d2b2e6af752\nexpires 2026-09-13T20:10:11.367Z\n```\n\n**The agent presents it, then tries to spend it on something else.** The first call is exactly\nwhat the human approved, and it runs. The second is the same approval with one digit changed,\nand it matches nothing:\n\n``` python\nimport sys\n\nimport ctrlrun\n\nfrom agent import refund, stripe\n\nwith open(\"request_id.txt\") as handle:\n    request_id = handle.read().strip()\n\nwith ctrlrun.context(agent=\"support-agent\"), ctrlrun.with_approval(request_id):\n    # Exactly what the human read: €5,000 on txn_2.\n    print(\"€5,000 with the approval ->\", refund(payment_id=\"txn_2\", amount=500_000)[\"status\"])\n\n    # The same approval, one digit changed.\n    try:\n        refund(payment_id=\"txn_2\", amount=900_000)\n    except ctrlrun.ApprovalMismatch:\n        print(\"€9,000 on that same approval -> refused\")\n    else:\n        sys.exit(\"a mutated action ran on a human's approval; that is the bug this exists to stop\")\n\nprint(\"calls that reached the provider:\", len(stripe.calls), \"(the €9,000 never left)\")\nphp\n€5,000 with the approval -> succeeded\n€9,000 on that same approval -> refused\ncalls that reached the provider: 1 (the €9,000 never left)\n```\n\nEvery attempt, refusals included, left a receipt, and `ctrlrun receipts` lists them. That is the\nwhole integration: a policy file, a decorator, a context, and `with_approval` to present a grant.\nMoney is the example, not the scope. A condition is `<argument>_<op>`, so the same policy\nlanguage reads `role_in: [reader, viewer]` or `replicas_lte: 10` as easily as `amount_lte`, and\n[nine domains](#the-same-shape-in-nine-domains) below have one policy each.\n[Protect your first action](https://docs.ctrlrun.dev/get-started/quickstart) walks the same path\nwith every output explained ·\n[Policy YAML reference](https://docs.ctrlrun.dev/reference/policy-yaml) ·\n[Cookbook](https://docs.ctrlrun.dev/cookbook/index): refunds, deploys, IAM, deletions, email, MCP.\n\n**You probably do not need an adapter.** `@protect` covers anything running in this process: a\nraw model call, a LangChain tool, a hand-rolled loop, a cron job. The gateway covers anything\nthat reaches its tools over MCP, in any language.\n\n| You have | Use | Needs | \n|---|---|---|\n| Python in this process | the `@protect` decorator, shown above | nothing beyond `pip install ctrlrun` | \n| Tools behind an MCP server, in any language | the gateway: `pip install \"ctrlrun[gateway]\"` | one command, no change to agent or server code | \n| A framework with its own approval interrupt | an adapter | the framework to have a human-in-the-loop primitive | \n\n**It works with agents you can and can't modify.** WhatsApp, Slack and Teams bots, ChatGPT,\nCursor, Codex, OpenAI Agents: any AI agent you have. CTRLRun checks the action, not the\nagent, so if the agent acts through a tool server or an API you run, the action is checked, and\nthe agent is not rebuilt, redeployed or told.\n[Agents you can't modify](https://docs.ctrlrun.dev/agents-you-cant-modify) says where the\nboundary goes for each kind.\n\nAn adapter exists for one reason: to route an `approve` decision through the framework's own\ninterrupt, so a human answers where they already answer. There is never a second place to say\nyes. [`ctrlrun-langgraph`](https://github.com/CTRLRun/ctrlrun/blob/main/adapters/langgraph/README.md)\ngives **prevention**, because the resumption carries the arguments and core re-checks them\nagainst the hash.\n[`ctrlrun-openai-agents`](https://github.com/CTRLRun/ctrlrun/blob/main/adapters/openai-agents/README.md)\ngives **attribution**, because that SDK records *that* a call was approved and not what its\narguments were. None of the three is only for agents: a worker, a webhook handler and a\nscheduled job cannot tell a first attempt from a retry either.\n\nEvery protected call, whichever way it arrives, goes through the same seven steps. Only then does it reach your systems.\n\n```\n  normalize  →  decide  →  approve  →  reserve  →  execute  →  resolve  →  record\n```\n\n1. **Normalize: one action, one id.** The call becomes an`Action` : a name, canonical arguments\n(sorted keys, no floats), a resource, the principal. Its SHA-256 is the action hash.\n2. **Decide: allow, ask or block.** Authority first (may*this principal* propose this at all,\nand within what bounds?), then policy (how much autonomy does*this action* get?). Unknown\naction, missing policy or missing principal is`deny` . Silence is never permission.\n3. **Approve: bound to this action.** A human answers against the action hash. The approval is\nsingle-use, expires, and matches nothing but that exact action, so arguments changed after\nthe answer void it and a person answers again. Name a`preconditions=` provider and the\napproval is also bound to the resource state it was granted against, rechecked strictly\nbefore the reservation: that**narrows** the window between the answer and the execution,\nfrom minutes of deliberation to milliseconds. It does not close it, because the recheck is a\nnetwork call and cannot run inside the atomic write.\n4. **Reserve: claimed once.** The effect key,`refund:txn_1` or`namespace:prod-eu:checkout` , is\ntaken in one atomic write. A second caller, in another process or on another host, is refused.\n5. **Execute: your code runs.** Only`NotExecuted` , raised by you, means`FAILED` ; every other\nexception and every timeout means`AMBIGUOUS` . Deciding which one you are looking at is the\nhard part, so`ctrlrun.transport` does it for you:`urlopen` ,`HTTPConnection` and`HTTPSConnection` from stdlib`urllib` and`http.client` , which raise`NotExecuted` only where\nthe connection they opened was handed no request byte. After one byte, every failure stays the\nexception it was, and the outcome is`AMBIGUOUS` . No setting widens that.\n6. **Resolve: unknown is not failed.** An`AMBIGUOUS` effect keeps its key and refuses a retry\nuntil`ctrlrun resolve` , or a`reconcile` hook that asked the remote, says what happened.\nNothing runs twice on a guess.\n7. **Record: a receipt either way.** A portable JSON receipt: who, what, decision, approval,\neffect key, outcome, and the hash of the policy that decided it, chained to the receipt\nbefore it. Refusals get one too.\n\n**Who may ask, and how much.** The policy decides the action and cannot see who is asking. Who\nmay ask at all is a second axis, authority: every principal needs a grant, a delegation cannot\nwiden one, and an action needs both axes, the stricter of the pair. Since 0.9 a grant can also\ncarry a **budget**, a metric with a limit over a rolling window, consumed on reserve inside the\nsame write, so a thousand refunds that each pass `amount_lte` cannot add up to more than the\ngrant allows. An `AMBIGUOUS` effect holds its budget until it is resolved, because otherwise an\nagent that can manufacture ambiguity could manufacture authority. A budget bounds what the next\nreservation may do; it cannot recall an action already in flight.\n[Authority](https://docs.ctrlrun.dev/authority) has the whole model.\n\nState lives in SQLite by default, a file with no server and no ops, and the reservation holds\nacross processes rather than merely across threads. Point it at Postgres when more than one\nhost writes: `pip install \"ctrlrun[postgres]\"`, one URL, the same guarantees graded by the same\nsuite. Prove it in your own setup with `ctrlrun verify`, which runs the kernel's own failure\nscenarios against *your* policy in a scratch store. It reaches no network: the only sockets it\nopens are to the store you named and to loopback listeners it bound itself, which is how it\ngrades the transport classifier.\n\n| Guarantee | `@protect` | Gateway | Adapter | \n|---|---|---|---|\n| **Approval binding** — An approval is bound to the exact action; a mutated or replayed one is refused. | yes | yes | prevention or attribution, per adapter | \n| **One effect, once** — One logical effect happens at most once, across threads, processes and hosts. | yes | yes | yes | \n| **Unknown is not failed** — An unknown outcome is AMBIGUOUS, never FAILED, and blocks a blind retry. | yes | yes | yes | \n| **Fail closed** — An unknown action, a missing policy or a missing principal is denied. | yes | yes | yes | \n| **Authority and delegation** — Every principal needs a grant, delegation cannot widen one, and a grant bounds the total. | yes | yes | yes | \n| **Receipts** — Every executed action leaves a portable JSON receipt of who, what and outcome. | yes | yes | yes | \n\nNothing in CTRLRun knows what a refund is. An action is a **name**, **canonical arguments**, an\n**effect key** and a **resource**, and the three questions asked of it are the same whichever\ndomain it came from: how much autonomy does *this action* get, did a human approve *this exact*\naction, and has this effect already happened. Two things carry your domain, and you write both.\n\n- **The effect key is the only domain knowledge in the system.** It is the string that says two\ncalls are the same real-world consequence:`refund:{payment_id}` ,`namespace:{cluster}:{name}` ,`grant:{user_id}:{role}` ,`prescription:{patient_id}:{drug}` .\nName it well and a retry cannot act twice; leave it out and there is nothing for*at most\nonce* to be about.\n- **Conditions are arguments, not amounts.** The language is`<argument>_<op>` , so the same\noperators read`replicas_lte: 10` ,`role_in: [reader, viewer]` and`to_domain_eq: acme.com` as easily as`amount_lte` . A band is available to a domain that has never issued an invoice.\n\n| Domain | Autonomous | A human decides | Never | \n|---|---|---|---|\n| [DevOps](https://github.com/CTRLRun/ctrlrun/blob/main/examples/policies/devops.yaml) | `k8s.scale_deployment` to 10 replicas | `terraform.apply` | `k8s.delete_namespace` | \n| [Security operations](https://github.com/CTRLRun/ctrlrun/blob/main/examples/policies/security.yaml) | `firewall.add_deny_rule` | `firewall.add_allow_rule` | `edr.disable_protection` | \n| [Healthcare](https://github.com/CTRLRun/ctrlrun/blob/main/examples/policies/healthcare.yaml) | `appointment.reschedule` | `patient.export_record` | `prescription.change_dose` | \n| [Legal](https://github.com/CTRLRun/ctrlrun/blob/main/examples/policies/legal.yaml) | `document.draft_internal` | `document.file_with_court` | `contract.execute` | \n| [HR](https://github.com/CTRLRun/ctrlrun/blob/main/examples/policies/hr.yaml) | `pto.approve` within a band | `payroll.run` | `employee.delete_record` | \n| [Insurance](https://github.com/CTRLRun/ctrlrun/blob/main/examples/policies/insurance.yaml) | `claim.request_documents` | `claim.approve_payout` above a band | `policyholder.delete` | \n| [E-commerce](https://github.com/CTRLRun/ctrlrun/blob/main/examples/policies/e-commerce.yaml) | `inventory.adjust` within a band | `price.update` | `customer.delete` | \n| [Public services](https://github.com/CTRLRun/ctrlrun/blob/main/examples/policies/government.yaml) | `eligibility.precheck` | `benefit.terminate` | `record.delete` | \n| [Payments](https://github.com/CTRLRun/ctrlrun/blob/main/examples/policies/payments.yaml) | `stripe.refund` under €500 | `stripe.refund` above it | `stripe.delete_customer` | \n\nRead any row left to right and it is one rule wearing different nouns. The security row is the\none to read twice: adding a **deny** rule to a firewall is autonomous and adding an **allow**\nrule is not, which no amount threshold would have told you. The policy is where your judgement\nabout your domain gets written down; CTRLRun is what makes it hold.\n\n**[docs.ctrlrun.dev](https://docs.ctrlrun.dev/)** is the documentation: concepts, guides, a cookbook, the\nfull reference.\n\n| Start here | [Why](https://docs.ctrlrun.dev/why) ·[Protect your first action](https://docs.ctrlrun.dev/get-started/quickstart) | \n| The ideas, and doing something with them | [Concepts](https://docs.ctrlrun.dev/concepts/outcomes-and-ambiguous) ·[Guides](https://docs.ctrlrun.dev/guides/protect-a-function) ·[Cookbook](https://docs.ctrlrun.dev/cookbook/index) | \n| Agents and MCP | [Agents you can't modify](https://docs.ctrlrun.dev/agents-you-cant-modify) ·[MCP overview](https://docs.ctrlrun.dev/mcp/overview) ·[The gateway in five minutes](https://docs.ctrlrun.dev/mcp/gateway-in-5-minutes) | \n| Running it for real | [Production](https://docs.ctrlrun.dev/production/index) ·[Postgres](https://docs.ctrlrun.dev/production/postgres) ·[Recovery](https://docs.ctrlrun.dev/production/recovery) ·[Operations](https://docs.ctrlrun.dev/production/operations) | \n| Every key, flag and error | [Reference](https://docs.ctrlrun.dev/reference/policy-yaml) ·[FAQ](https://docs.ctrlrun.dev/faq) | \n| Compared with | [Idempotency keys](https://docs.ctrlrun.dev/compare/idempotency-keys) ·[Framework human-in-the-loop](https://docs.ctrlrun.dev/compare/framework-hitl) ·[Guardrail libraries](https://docs.ctrlrun.dev/compare/guardrail-libraries) ·[Durable workflows](https://docs.ctrlrun.dev/compare/durable-workflows) ·[Governance toolkits](https://docs.ctrlrun.dev/compare/governance-toolkits) | \n| What holds, and what does not | [Threat model](https://docs.ctrlrun.dev/THREAT_MODEL) ·[What `verify` proves](https://docs.ctrlrun.dev/verify) ·[`CLAIMS.md`](https://docs.ctrlrun.dev/CLAIMS) , every sentence mapped to its test ·[How this is built](https://docs.ctrlrun.dev/how-this-is-built) | \n\nIssues and pull requests are welcome:\n[`CONTRIBUTING.md`](https://github.com/CTRLRun/ctrlrun/blob/main/CONTRIBUTING.md) and\n[`CODE_OF_CONDUCT.md`](https://github.com/CTRLRun/ctrlrun/blob/main/CODE_OF_CONDUCT.md) have the\nworking agreement, and\n[`SECURITY.md`](https://github.com/CTRLRun/ctrlrun/blob/main/SECURITY.md) is how to report a\nvulnerability. Every claim in this file has a test behind it, so a change to the prose usually\nmeans a change to the suite.\n[`CHANGELOG.md`](https://github.com/CTRLRun/ctrlrun/blob/main/CHANGELOG.md) and\n[`https://docs.ctrlrun.dev/ROADMAP`](https://docs.ctrlrun.dev/ROADMAP) say\nwhere it is going. Releases carry PyPI provenance attestations from GitHub Actions.\n\nApache-2.0. The enforcement kernel is and will remain fully open source.", "url": "https://wpnews.pro/news/show-hn-an-open-source-safety-layer-for-ai-agent-actions", "canonical_source": "https://github.com/CTRLRun/ctrlrun", "published_at": "2026-09-15 16:03:31+00:00", "updated_at": "2026-09-15 16:21:10.867743+00:00", "lang": "en", "topics": ["ai-agents", "ai-safety", "ai-tools", "developer-tools"], "entities": ["CTRLRun", "Python", "Postgres", "Apache-2.0"], "alternates": {"html": "https://wpnews.pro/news/show-hn-an-open-source-safety-layer-for-ai-agent-actions", "markdown": "https://wpnews.pro/news/show-hn-an-open-source-safety-layer-for-ai-agent-actions.md", "text": "https://wpnews.pro/news/show-hn-an-open-source-safety-layer-for-ai-agent-actions.txt", "jsonld": "https://wpnews.pro/news/show-hn-an-open-source-safety-layer-for-ai-agent-actions.jsonld"}}