{"slug": "from-demo-to-production-the-guardrails-that-make-an-ai-agent-safe-to-ship", "title": "From Demo to Production: The Guardrails That Make an AI Agent Safe to Ship", "summary": "A developer has released a small, open-source harness designed to make AI agents safe for production, addressing the gap between demos and deployable systems. The harness enforces quality gates, human approval for high-risk actions, and vendor-agnostic model abstraction, with an audit trail for compliance. The developer, who built and ran a ~25-agent platform in production, emphasizes that the hard part of agent development is not calling the model but preventing harmful actions.", "body_md": "Hook:Most \"AI agents\" you see on the internet are demos. Here's the single most common\n\nreason they never reach production — and a small, open-source harness that gets past it.\n\nWe are past the phase where the hard part of building an *AI agent* was calling the model.\n\nThe hard part now is the 10% nobody talks about: **what stops the agent from doing something\nharmful?** I've seen this from both sides — I built and ran a ~25-agent platform in production\n\nThe uncomfortable truth: a chatbox that can call 5 tools is *not* a product. The difference\n\nbetween a weekend project and a system you can put in front of customers is three things —\n\nand they're all boring, non-glamorous engineering:\n\nSo I wrote a tiny harness that keeps these front and center. It's intentionally small — small\n\nenough to read in an hour — because the value isn't in a framework, it's in the *pattern*.\n\nAn agent's output is a prediction, not a promise. Before it ships, you need a **check** that\n\nit passes your bar. In the harness this is a pluggable `QualityGate`\n\n— a rule of thumb you swap\n\nwith an LLM judge or a test suite:\n\n```\n# agent_harness/eval.py\n@dataclass\nclass EvalReport:\n    passed: bool\n    score: float\n    checks: List[str]\n\nclass QualityGate:\n    def grade(self, proposal: str, context: str = \"\") -> EvalReport:\n        return self.grader(proposal, context)\n```\n\nThe loop refuses to execute if the gate fails:\n\n```\nresult.report = self.quality.grade(proposal, f\"state={state}\")\nif not result.report.passed:\n    self.approval.log(\"quality-gate\", \"blocked\", result.report.__str__())\n    return result\n```\n\nNotice it **logs the block**. In production you'd want every blocked attempt in your\n\nobservability stack. \"We rejected 12% of agent proposals this week\" is a real KPI — it means\n\nthe gate is doing its job.\n\nThis is the one that actually gets enterprises to say **yes**. When an agent wants to expedite\n\nan order, cancel a subscription, or move money, it should *stop* and ask a human. Silence is\n\nnot consent.\n\n``` python\n# agent_harness/approval.py\nclass ApprovalGate:\n    def request(self, action: str, detail: str) -> bool:\n        # In production: push a notification to Teams / Slack / email and wait.\n        decision = input(f\"Approve {action}? [y/N] \").strip().lower()\n        self.audit.append(AuditEntry(time.time(), action, \"human-reviewer\", decision, detail))\n        return decision.startswith(\"y\")\n```\n\nIn the scaffold, marking a tool `needs_approval=True`\n\nis enough to route it through the gate:\n\n```\n@tool(\"expedite_order\", \"Mark an order as expedited.\", needs_approval=True)\ndef expedite_order(order_id: str) -> str:\n    return f\"PO {order_id}: marked expedited\"\n```\n\nAnd because there's an **audit trail**, you can always answer \"who changed this and why?\" —\n\nwhich is usually the *first* question a compliance team asks.\n\nModels change every few weeks, and so do prices. Your agent loop should never know which\n\nvendor it's talking to:\n\n``` python\n# agent_harness/providers.py\nclass ModelProvider(Protocol):\n    def complete(self, messages: List[Message], tools: Dict[str, Any]) -> str: ...\n\nagent = Agent(provider=default_provider(\"openai\"))   # or \"deepseek\", \"qwen\", \"mock\"\n```\n\nA few lines of abstraction mean you can run the same agent on OpenAI, Azure OpenAI, DeepSeek,\n\nor Qwen with a config change — and test the whole thing offline with a `MockProvider`\n\nthat\n\nneeds no API key. That's not just engineering hygiene, it's a cost lever and a hedge.\n\nThe other huge pattern: don't let an \"agent\" freewheel over your critical process. Model the\n\nworkflow explicitly. Here's a purchase-order exception state machine — the kind of thing that\n\nshows up in every supply chain / ERP copilot:\n\n```\n# agent_harness/state_machine.py\nclass State(str, Enum):\n    OPEN = \"open\"; EXPEDITED = \"expedited\"; CANCELLED = \"cancelled\"\n    EXECUTED = \"executed\"; CLOSED = \"closed\"\n\nclass POStateMachine:\n    def transition(self, target: State, reason: str = \"\") -> State:\n        if target not in TRANSITIONS[self.state]:\n            raise InvalidTransition(f\"{self.state} -> {target}\")\n        self.state = target\n        return self.state\nopen ──► expedited ──► executed ──► closed\n  │          ▲              ▲\n  └──exception──┘  (human approval gate in between)\n```\n\nThe agent proposes a transition; the state machine *enforces* what's legal; a human approves\n\nanything consequential. That's how you get **autonomy with control** — the exact phrase\n\nenterprises want to hear.\n\n```\nagent = Agent(provider=provider, registry=registry,\n              approval=ApprovalGate(), quality=QualityGate())\nsm = POStateMachine(po_id=\"PO-1234\")\nresult = agent.run(\"PO-1234 is stuck; expedite it.\", state=sm)\nprint(result.audit)\n```\n\nPull the repo and run it with **no API key**:\n\n```\npython examples/po_workflow.py --yes\n```\n\nYou'll see the quality gate pass, the human approve, the order get expedited, the state move to\n\n`expedited`\n\n, and the whole thing recorded in the audit trail. That's a real production-shaped\n\nflow in ~300 lines.\n\nIf you're building an agent, build the **guardrails first**. It's the difference between a\n\ndemo you screenshot and a system your business actually trusts. And if you don't want to build\n\nit yourself — or want someone who's shipped this at Microsoft scale — [get in touch](https://zhasun0818.github.io/zhaowei-portfolio/).\n\n*Open-sourced under MIT. Fork it, build on it, and say hi.*\n\n*About the author: I'm Zhaowei Sun, an AI agent / Copilot engineer who's built and run\nproduction agent platforms at Microsoft and high-scale systems at Hulu. I consult on agent\narchitecture, and I keep this repo as the scaffolding I wish I'd had.*\n\n**Tags:** AI Agents, LLM, RAG, Microsoft Copilot, Software Architecture, System Design\n\n**Suggested platforms:** Dev.to | Medium | LinkedIn | Hacker News (Show HN) | 掘金 (translated)", "url": "https://wpnews.pro/news/from-demo-to-production-the-guardrails-that-make-an-ai-agent-safe-to-ship", "canonical_source": "https://dev.to/sunny_1024k/from-demo-to-production-the-guardrails-that-make-an-ai-agent-safe-to-ship-d2o", "published_at": "2026-08-23 15:41:29+00:00", "updated_at": "2026-08-23 16:13:31.282947+00:00", "lang": "en", "topics": ["ai-agents", "ai-safety", "developer-tools", "ai-products"], "entities": ["OpenAI", "Azure OpenAI", "DeepSeek", "Qwen"], "alternates": {"html": "https://wpnews.pro/news/from-demo-to-production-the-guardrails-that-make-an-ai-agent-safe-to-ship", "markdown": "https://wpnews.pro/news/from-demo-to-production-the-guardrails-that-make-an-ai-agent-safe-to-ship.md", "text": "https://wpnews.pro/news/from-demo-to-production-the-guardrails-that-make-an-ai-agent-safe-to-ship.txt", "jsonld": "https://wpnews.pro/news/from-demo-to-production-the-guardrails-that-make-an-ai-agent-safe-to-ship.jsonld"}}