cd /news/ai-agents/we-started-with-four-open-agent-cli-… · home › topics › ai-agents › article
[ARTICLE · art-172392] src=dev.to ↗ pub= topic=ai-agents verified=true sentiment=↑ positive

We Started With Four Open Agent CLI Questions. We Ended With Two Real Wins.

The agentpolicy project released v0.1.0, a local-first, Cedar-based policy decision point for AI agents that returns allow, deny, ask, rate-limit or redact decisions for every agent action, now available on PyPI. In a Tier-1 field test, the team got live harnesses working for Claude Code and Codex — capturing request/response traffic and audit trails, blocking destructive commands and escalating production writes — while documenting honest noes for Cursor and Gemini, whose CLI could not run locally due to a classifier-only gemma route and an OAuth tier gate.

key takeaway

agentpolicy v0.1.0 ships a local Cedar-based policy decision point that returns allow, deny, ask, rate-limit or redact per agent action, with working Claude Code and Codex harnesses that captured six request/response pairs and blocked destructive commands.

by read4 min views6 publishedNov 19, 2026

From the agentpolicy v0.1.0 release: a local-first, Cedar-based policy decision point for AI agents (decide allow, deny, ask, rate-limit, or redact for every agent action, with shipped rule packs). Now on PyPI. This story is the Tier-1 part of the v0.1.0 field test report.

AI disclosure: this article was drafted and edited with AI assistance. The field test, the defects, the fixes, and every number are real and linked below.

At one point, the entire Tier-1 CLI section of the field test was still unresolved.

That was the honest state. Claude Code had not run live yet. Codex had not run live yet. Cursor and Gemini had not been closed out either. If I had stopped there, the report would have been accurate but incomplete.

So I kept going case by case, with one rule: do not claim a harness until it runs with evidence on this box.

That persistence changed the release in a meaningful way. Claude Code and Codex both went live. Cursor and Gemini both got real investigations and honest noes. We shipped the two we could prove and left the other two out.

The Tier-1 set in the plan was four CLIs: Claude Code, Cursor, Codex, and Gemini. Early in the sweep, none of them belonged in a green release note. The report had to say that plainly.

This is where the field-test discipline mattered. A case was only real if it ran through the actual harness and left evidence behind: verdict, assertions, transcripts, audit trail, and captured LLM traffic. If it did not, it stayed unproven.

That kept me from turning "probably works" into "supported."

Claude Code was the first real win, and it took more than a config toggle. The working harness is in scripts/fieldtest/t1_claude.sh, with the hook entrypoint in t1_evaluate.py and the shared policy in t1_policy.yaml.

Three things had to be figured out:

/v1/messages with a real msg_* body, so the CLI could talk to it directly.--settings explicitly because cwd settings files do not load headless the way I assumed.t1_proxy.py Once that wiring was real, the case passed live: a read executed, a prod write escalated without creating the side-effect file, a destructive command was denied, and the audit trail recorded all three decisions. Six request/response pairs were captured.

Codex needed a different investigation. Its harness lives in scripts/fieldtest/t1_codex.sh.

The key findings were different from Claude's:

/v1/responses natively.codex exec reads stdin. Left alone in a harness, it quietly waits forever. Closing stdin with </dev/null> fixed the stall.base_url needed the /v1 suffix or every call 404ed. Codex also forced a more honest test shape. Headless Codex does not expose a pausable per-call approval hook like Claude. So the case proves what the surface actually supports: allow executes, deny holds and replans, plus pre-execution parity against the same in-process policy engine. That is a real pass, just not the same kind of pass as Claude.

The result was still strong: live run, captured traffic, parity audit trail, destructive command blocked, target directory intact.

Gemini was not a hand-wave. The local install worked. gemini gemma setup brought up a local server. But the CLI's main loop still could not run locally because the local gemma route is classifier-only and the CLI hits an OAuth tier gate before any model routing. That is all documented in the report and backed by the bundle-source read plus the parity stand-in that did run.

Cursor was a different kind of limitation. There is no supported headless surface on this box for the permissions.json flow. Cursor is a GUI IDE, and without a reproducible, evidence-capturing path, there was no honest live case to claim.

Those are not failures to hide. They are useful boundaries. They tell the release exactly what it supports today and what would need to change for the other two to re-enter scope.

The encouraging part of this story is not "we only got two." It is that we started with four question marks and finished with two live integrations, two investigated boundaries, and a release note I can defend line by line.

t1_claude.sh`` t1_codex.sh``t1_evaluate.py`` t1_policy.yaml``pip install agentsec-agentpolicy, Have you had a release where the strongest part of the story was not the things that passed, but the things you proved carefully enough to leave out?

── more in #ai-agents 4 stories · sorted by recency
── more on @agentpolicy 3 stories trending now
sponsored brought to you by zahid.host 4,200+ EU-deployed projects
reading about agents? ship yours in a single git push.

Run your AI side-project on zahid.host

EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.

$git push zahid main
→ Live at https://your-agent.zahid.host ✓
Get free account → Pricing
from €0/mo · no card required
LIVE [news/we-started-with-four…] indexed:0 read:4min 2026-11-19 · —