{"slug": "how-i-made-ai-agents-safe-to-run-on-real-infrastructure", "title": "How I made AI agents safe to run on real infrastructure", "summary": "A developer built Cmdop, an agent-to-infrastructure platform that runs LLM agents on live systems, and solved the reliability problem by implementing a verification loop that scores every tool call before, during, and after execution. The system treats agent output as something to verify, score, and constrain rather than execute directly, using structured-output contracts, full tracing, and side-effect scoring to catch actions that complete tasks while also deleting critical data. This approach enabled the developer to safely widen agent autonomy from human-in-the-loop to autonomous operation by measuring reliability through evaluation data rather than relying on model confidence.", "body_md": "*Draft flagship post — publish on your blog (canonical), then cross-post to dev.to / Hashnode and as a LinkedIn article. ~1,100 words. This is your single strongest, most differentiated story — it's what makes a hiring manager think \"this person actually gets agent reliability.\"*\n\nEveryone can get an LLM agent to do something impressive in a demo. Far fewer can get one to act on live infrastructure — your machines, terminals, files, deployments — without occasionally doing something catastrophic.\n\nThat gap is the whole problem. And it's not a model problem. The model is the easy part now. The hard part is making an agent's *actions* trustworthy enough that you'd let it run unattended against systems that matter.\n\nI built [Cmdop](https://cmdop.com) around exactly this problem. Here's the architecture and, more importantly, the reliability loop that turned it from \"a demo that works once\" into a runtime I actually trust.\n\nCmdop is an **agent → gRPC → server → SDK** platform. Agents act on remote machines through a multi-agent runtime: they hand off to each other, call tools, and operate under human-in-the-loop control where it matters. Developers integrate the primitives directly through Node.js, Python, and React SDKs, and the whole thing runs thousands of concurrent agent sessions over persistent gRPC/WebSocket streams.\n\nNone of that is the interesting part. Plenty of systems can route an LLM's output to a shell. The interesting part is what happens around every single tool call.\n\nA plausible-looking action is the most dangerous thing an agent produces. `rm -rf ./logs`\n\nand `rm -rf /logs`\n\nlook almost identical and differ by a catastrophe. An agent that's \"usually right\" is, on real infrastructure, a system that will eventually take down production confidently.\n\nSo I stopped treating agent output as something to execute and started treating it as something to **verify, score, and constrain** — before, during, and after execution.\n\nEvery tool call in Cmdop runs through the same loop:\n\n**Structured-output contract, validated before execution.** The agent doesn't emit free text that I parse hopefully. It emits a structured contract — intended action, parameters, expected effect — and that contract is validated against a schema *before* anything runs. Malformed or out-of-policy calls never reach the system.\n\n**Full trace, logged.** Prompt, tool call, result, latency, and which retry/failover path it took — all captured. You cannot improve what you cannot see, and agent failures are subtle: the run \"succeeded\" but did the wrong thing.\n\n**Scored on the axes that matter.** Not \"did it return 200.\" Each run is scored on **tool-call validity**, **task success**, and — the one everyone forgets — **unintended side-effects**. The side-effect score is what catches the agent that completed the task *and* deleted something it shouldn't have.\n\n**Guardrails + automatic retry/failover where evals expose brittleness.** When the eval data showed a step was fragile, I didn't just log it — I added a structured guardrail and an automatic retry/failover path. Brittle steps became contained steps.\n\nThe payoff wasn't a dashboard. It was **autonomy I could widen safely.**\n\nBefore the loop, every meaningfully risky action needed a human in front of it, because I had no principled way to know which actions were safe to let run. After the loop, I had data: which tool calls were reliably valid, which tasks completed cleanly, which steps produced side-effects. That let me move actions from \"human-in-the-loop required\" to \"autonomous\" *one measured step at a time*, instead of guessing.\n\nThat's the real lesson, and it generalizes far beyond Cmdop: **agent autonomy is earned through evaluation, not granted by confidence.** The teams shipping agents into production aren't the ones with the best prompts — they're the ones who instrumented and scored agent behavior until they knew, with data, where autonomy was safe.\n\nThe interesting frontier in agentic AI right now isn't bigger models — it's the **reliability layer**: evals, guardrails, observability, the harness around the agent. That's the part that decides whether agents stay demos or become infrastructure. It's also, conveniently, the part I find most interesting to build.\n\nIf you're working on agent reliability, agent platforms, or making AI safe to run against real systems — I'd love to compare notes.\n\n*I'm Mark K. (Igor Korotin), a Principal Product Architect / Technical CPO building applied-AI platforms. More at cmdop.com and djangocfg.com, code at github.com/markolofsen.*", "url": "https://wpnews.pro/news/how-i-made-ai-agents-safe-to-run-on-real-infrastructure", "canonical_source": "https://dev.to/markin/how-i-made-ai-agents-safe-to-run-on-real-infrastructure-cj5", "published_at": "2026-06-04 07:07:33+00:00", "updated_at": "2026-06-04 07:12:11.032789+00:00", "lang": "en", "topics": ["ai-agents", "ai-safety", "ai-infrastructure", "ai-tools", "ai-products"], "entities": ["Cmdop", "Node.js", "Python", "React", "gRPC", "WebSocket"], "alternates": {"html": "https://wpnews.pro/news/how-i-made-ai-agents-safe-to-run-on-real-infrastructure", "markdown": "https://wpnews.pro/news/how-i-made-ai-agents-safe-to-run-on-real-infrastructure.md", "text": "https://wpnews.pro/news/how-i-made-ai-agents-safe-to-run-on-real-infrastructure.txt", "jsonld": "https://wpnews.pro/news/how-i-made-ai-agents-safe-to-run-on-real-infrastructure.jsonld"}}