Most developers still use AI like a chatbot.
They open a model, type a prompt, copy the output, fix mistakes, and repeat.
That works for small tasks. It does not scale.
If you want AI to operate like a real engineering teammate, you need more than prompts. You need an operating system around the model: rules, routing, verification, memory, budgets, trust, and rollback paths.
This guide shows how to build an Agentic OS: a lightweight automation layer that can inspect your repo, decide what work matters, assign tasks to models, verify the output, enforce budgets, and gradually earn autonomy.
This is not tied to one vendor. You can adapt it to Claude, GPT, Gemini, local models, OpenRouter, or any coding agent CLI.
By the end, your repo will look like this:
your-repo/
AGENTS.md
Makefile
agent-os/
cycle.sh
policy.md
router.md
intake.md
workers/
execute.md
audit.md
gates/
verify.sh
scripts/
trust.sh
cost-log.sh
cost-check.sh
init.sh
assertions/
example.md
memory/
STATE.md
trust.tsv
assertion-ledger.tsv
usage.tsv
queue.md
incidents.md
The system has five layers:
Signals
↓
Router
↓
Executor
↓
Auditor
↓
Deterministic Gate
The core principle:
AI can suggest. AI can execute. AI can review. But a deterministic command decides whether work is done.
A single AI agent doing everything is fragile.
Common failure modes:
An Agentic OS fixes this by separating responsibilities:
Before writing files, define the laws.
The executor should not approve its own output. A fresh auditor must review the diff independently.
Bad: "Improve the login flow."
Better: "Update login error copy and verify npm test -- tests/login
passes."
Best: "Change only login error messaging. Done when npm test -- tests/login
and npm run lint
both pass."
Use high-capability models where branching decisions matter. Use cheaper models for repetitive, scoped implementation.
Do not give global autonomy. A model may be reliable at docs but unsafe for migrations.
If something matters, keep checking it. A bug fixed once can return later.
You need: bash, git, jq, make, gh, npm.
Optional: llm
, an OpenRouter key, Claude Code, the OpenAI CLI, a local coding agent CLI, cron.
This guide uses placeholder commands like ai-router
, ai-exec
, ai-audit
. Replace them with your preferred CLI, for example:
claude -p "..."
llm -m openrouter/model-name "..."
openai responses create ...
aider ...
cursor-agent ...
The architecture matters more than the specific provider.
mkdir -p agent-os/{workers,gates,scripts,assertions,memory}
touch agent-os/memory/STATE.md
touch agent-os/memory/queue.md
touch agent-os/memory/incidents.md
touch agent-os/memory/trust.tsv
touch agent-os/memory/assertion-ledger.tsv
touch agent-os/memory/usage.tsv
Create AGENTS.md
:
## Non-Negotiable Rules
- Never change more than 200 lines without human approval.
- Never edit authentication, payments, secrets, migrations, or production configuration unattended.
- Never add dependencies without approval.
- Never delete, weaken, skip, or rewrite tests to make a change pass.
- Never claim work is complete unless the validation gate passes.
- Never invent secrets, APIs, file paths, or conventions.
- Never perform destructive actions without approval.
- Never continue after two verifier failures on the same task.
## Definition of Done
1. The requested change is implemented.
2. The diff stays within scope.
3. A fresh auditor approves the diff.
4. agent-os/gates/verify.sh exits successfully.
5. The result is logged in the trust ledger.
## Routing Rules
- Planning and prioritization use the strongest reasoning model available.
- Low-risk implementation uses cheaper execution models.
- Large reading tasks use cheap long-context models.
- Sensitive work is queued for human review.
- Shell commands decide final status.
## Protected Areas
- authentication
- authorization
- billing
- database migrations
- deployment scripts
- environment variables
- secrets
- production configuration
- destructive filesystem operations
This file is the constitution. Make it enforceable, not poetic.
Create agent-os/policy.md
:
## Can Act Alone
- documentation fixes
- typo fixes
- formatting
- lint cleanup
- small test fixes
- small refactors under 100 changed lines
- issue labeling
- draft pull requests
## Must Queue
- authentication
- authorization
- billing
- payments
- database migrations
- production configuration
- secrets
- dependency installation
- diffs over 200 changed lines
- unclear requirements
## Must Alert
- validation fails twice
- budget is exceeded
- protected file touched
- standing assertion violated
- model refuses the task
- tool output contradicts agent claim
- executor and auditor disagree twice
Create agent-os/gates/verify.sh
:
#!/usr/bin/env bash
set -euo pipefail
echo "Running validation gate..."
if [ -f package.json ]; then
npm run typecheck --if-present
npm test --if-present
npm run lint --if-present
fi
if [ -f pyproject.toml ] || [ -f requirements.txt ]; then
python -m pytest || true
fi
echo "Validation gate passed."
Make it executable with chmod +x agent-os/gates/verify.sh
, then customize it for your stack (go test ./...
, cargo test
, pytest
, mvn test
, gradle test
, npm run build
, etc). The gate must be deterministic — it should never depend on AI judgment.
Create agent-os/intake.md
:
You are the intake layer for an autonomous engineering system.
You receive recent repository signals:
- git commits
- open issues
- pull requests
- CI runs
- local validation output
Your job is only to decide whether anything requires action.
Output one of these:
status: quiet
or
status: actionable
finding: <one-line summary>
evidence: <commit, issue, PR, run, or command output>
risk: low | medium | high
sensitive: yes | no
Rules:
- Do not propose fixes.
- Do not write code.
- Do not speculate.
- If authentication, billing, secrets, migrations, or production config are involved, set sensitive: yes.
- If evidence is weak, output status: quiet.
The intake layer should be cheap. Most cycles should end here.
Create agent-os/router.md
:
You are the router for an autonomous engineering system.
You do not write code.
You do not edit files.
You only decide what should happen next.
Read:
- current state
- policy
- trust ledger
- latest intake finding
Return ONLY valid JSON:
{
"action": "execute|queue|stop",
"skill": "stable-kebab-case-name",
"summary": "one-line task summary",
"scope": ["paths or areas allowed"],
"blocked_paths": ["paths or areas not allowed"],
"instructions": "specific implementation instructions",
"done_when": [
"machine-checkable condition 1",
"machine-checkable condition 2"
],
"risk": "low|medium|high"
}
Decision rules:
- If risk is high, queue.
- If the task touches protected areas, queue.
- If requirements are unclear, queue.
- If the diff is likely above 200 lines, queue.
- If there is no valuable task, stop.
- Otherwise execute.
The router is the only agent that chooses work. It should be smart, brief, and conservative.
Create agent-os/workers/execute.md
:
You are the executor.
You receive one work order JSON.
Your job:
- implement exactly the requested task
- make the smallest useful change
- stay inside scope
- avoid unrelated refactors
- avoid new dependencies
- stop if blocked by missing information
- never touch blocked paths
Do not:
- add features
- redesign architecture
- change unrelated files
- delete tests
- weaken tests
- invent secrets or config
When finished, write a short IMPLEMENTATION.md containing:
- what changed
- why it changed
- how to verify it
Keep IMPLEMENTATION.md under 10 lines.
The executor should be constrained. The less freedom it has, the easier it is to verify.
Create agent-os/workers/audit.md
:
You are the auditor.
You receive:
- the work order
- the git diff
You do not know the executor's reasoning. You judge only the diff against the work order.
Check:
1. Does the diff satisfy every done_when condition?
2. Is the diff inside scope?
3. Are blocked paths untouched?
4. Were tests deleted, skipped, weakened, or bypassed?
5. Was unnecessary functionality added?
6. Is the change small enough to review?
Output exactly one line:
PASS: <short reason>
or
FAIL: <short reason>
The auditor should be fresh-context. Do not include executor notes unless necessary.
Create agent-os/scripts/cost-log.sh
:
#!/usr/bin/env bash
set -euo pipefail
STAGE="${1:-unknown}"
COST="${2:-0}"
mkdir -p "$(dirname "$0")/../memory"
echo -e "$(date -Is)\t$STAGE\t$COST" >> "$(dirname "$0")/../memory/usage.tsv"
Make executable: chmod +x agent-os/scripts/cost-log.sh
Now create agent-os/scripts/cost-check.sh
:
#!/usr/bin/env bash
set -euo pipefail
BUDGET="5"
USAGE_FILE="$(dirname "$0")/../memory/usage.tsv"
touch "$USAGE_FILE"
while [[ $# -gt 0 ]]; do
case "$1" in
--budget) BUDGET="$2"; shift 2 ;;
--report)
awk -F'\t' '{cost[$2]+=$3; total+=$3} END {for (s in cost) printf "%-12s $%.2f\n", s, cost[s]; printf "%-12s $%.2f\n", "TOTAL", total}' "$USAGE_FILE"
exit 0 ;;
*) shift ;;
esac
done
TODAY="$(date +%F)"
SPENT="$(awk -F'\t' -v today="$TODAY" '$1 ~ today {sum+=$3} END {printf "%.2f", sum}' "$USAGE_FILE")"
awk -v spent="$SPENT" -v budget="$BUDGET" 'BEGIN {if (spent >= budget) exit 1; exit 0}' || {
echo "Budget exceeded: spent \$$SPENT of \$$BUDGET" >&2
exit 1
}
Make executable: chmod +x agent-os/scripts/cost-check.sh
. This lets your loop fail closed once spending crosses the limit.
Create agent-os/scripts/trust.sh
:
#!/usr/bin/env bash
set -euo pipefail
FILE="$(dirname "$0")/../memory/trust.tsv"
touch "$FILE"
tier_of() {
local runs="$1"
local passes="$2"
awk -v r="$runs" -v p="$passes" '
BEGIN {
rate = (r > 0) ? p / r : 0
if (r >= 20 && rate >= 0.95) { print "auto" }
else if (r >= 10 && rate >= 0.90) { print "review" }
else { print "watch" }
}
'
}
case "${1:-}" in
--render)
printf "%-24s %6s %6s %8s %s\n" "skill" "runs" "pass" "rate" "tier"
while IFS=$'\t' read -r skill runs passes; do
[ -z "${skill:-}" ] && continue
rate="$(awk -v r="$runs" -v p="$passes" 'BEGIN { printf "%.0f%%", (r > 0) ? (p / r) * 100 : 0 }')"
printf "%-24s %6s %6s %8s %s\n" "$skill" "$runs" "$passes" "$rate" "$(tier_of "$runs" "$passes")"
done < "$FILE"
;;
--tier)
skill="$2"
line="$(grep -P "^${skill}\t" "$FILE" || true)"
if [ -z "$line" ]; then echo "watch"; exit 0; fi
runs="$(echo "$line" | cut -f2)"
passes="$(echo "$line" | cut -f3)"
tier_of "$runs" "$passes"
;;
record)
skill="$2"
result="$3"
awk -v skill="$skill" -v result="$result" -F'\t' '
BEGIN { OFS = "\t"; found = 0 }
$1 == skill { found = 1; runs = $2 + 1; passes = $3 + (result == "pass"); print skill, runs, passes; next }
{ print }
END { if (!found) print skill, 1, (result == "pass") ? 1 : 0 }
' "$FILE" > "$FILE.tmp"
mv "$FILE.tmp" "$FILE"
;;
*)
echo "Usage: trust.sh --render | trust.sh --tier <skill> | trust.sh record <skill> pass|fail"
exit 1
;;
esac
Make executable: chmod +x agent-os/scripts/trust.sh
Trust tiers:
watch
= new or unreliablereview
= can draft, human reviewsauto
= can ship after validationThis makes autonomy measurable.
Create agent-os/assertions/example.md
:
name: example
status: active
born: 2026-07-07
predicate: test -f AGENTS.md
on_fail: queue for review
retire_when: only by human decision
Then create agent-os/verify-assertions.sh
:
#!/usr/bin/env bash
set -uo pipefail
BASE="$(dirname "$0")"
LEDGER="$BASE/memory/assertion-ledger.tsv"
VIOLATIONS=0
touch "$LEDGER"
for file in "$BASE"/assertions/*.md; do
[ -e "$file" ] || continue
if grep -q '^status: retired' "$file"; then continue; fi
name="$(basename "$file" .md)"
predicate="$(grep '^predicate:' "$file" | cut -d':' -f2- | sed 's/^ //')"
start="$(date +%s%3N)"
if timeout 60 bash -c "$predicate" >/dev/null 2>&1; then
result="pass"
else
result="FAIL"
VIOLATIONS=$((VIOLATIONS + 1))
fi
duration="$(( $(date +%s%3N) - start ))"
echo -e "$(date -Is)\t$name\t$result\t${duration}ms" >> "$LEDGER"
if [ "$result" = "FAIL" ]; then echo "Assertion failed: $name"; fi
done
if [ "$VIOLATIONS" -gt 0 ]; then exit 1; fi
echo "All assertions passed."
Make executable: chmod +x agent-os/verify-assertions.sh
Examples of useful assertions:
predicate: npm test -- tests/auth
predicate: grep -R "sk_test_" . --exclude-dir=node_modules | wc -l | grep -q '^0$'
predicate: npm run lint
predicate: test -f docs/api.md
The idea: important completed work becomes a permanent check.
Create agent-os/cycle.sh
— the main loop that ties every layer together. Each iteration it gathers recent repo signals (commits, issues, CI runs), runs the intake prompt to decide if anything is actionable, runs the router prompt to produce a work order, creates an isolated git worktree, runs the executor prompt inside it, diffs the result, runs the auditor prompt against that diff, runs the deterministic validation gate, and records a pass or fail in the trust ledger — shipping a pull request automatically only once a skill has earned the auto
tier, otherwise leaving the change queued for human review. The loop respects a maximum iteration count (MAX_ITERS
) and a daily budget (DAILY_BUDGET_USD
), calling cost-check.sh
before and after each iteration and exiting early if the budget is exceeded. Make the script executable, and replace the placeholder commands ai-router
, ai-exec
, and ai-audit
with your actual model CLIs.
If you want a simple adapter layer, create thin wrapper scripts named ai-router
, ai-exec
, and ai-audit
inside agent-os/scripts/
. Each should parse --model
, --system
, and --input
flags and forward them to whatever CLI you use, for example a generic llm -m "$MODEL" -s "$SYSTEM" "$INPUT"
call. Make them executable with chmod +x agent-os/scripts/ai-*
, then add export PATH="$(pwd)/scripts:$PATH"
near the top of cycle.sh
so the main loop can call generic command names no matter which provider is behind them.
Add targets: tick
(run one cycle), queue
(show queued work), state
(tail recent state), trust
(render the trust table), audit
(show the cost report), assert
(run assertions), verify
(run the validation gate), and clean-worktrees
(remove leftover agent worktrees). Now you can run make tick
, make queue
, make trust
, make audit
, make assert
.
Run make verify
first — if it fails, fix your repo before building anything agentic on top of it. Then run make tick
. Expected outcomes: no actionable work, a queued skill, or a change ready for review. Do not automate yet; run manually for the first several cycles.
Create agent-os/assertions/login-tests.md
with a predicate that re-runs your login test suite, retiring only once the login module itself is removed. Run make assert
— if it passes, that fix is now continuously protected; if it fails, either the predicate is wrong or there's a real regression.
Daily cost ≈ (quiet cycles × intake cost) + (active cycles × (router cost + executor cost + auditor cost)). Example: 20 quiet cycles at $0.01 = $0.20, plus 4 active cycles at ($0.25 + $0.10 + $0.15) = $2.00, for a total of about $2.20/day. Keep the quiet cycle cheap — only use your strongest model once intake finds something actionable.
Only after manual testing, schedule the agent loop on weekdays and assertions daily, logging to a file. The loop drafts work; assertions catch regressions — keep those jobs separate.
Do not start with full autonomy.
Run make tick
, make trust
, make audit
, make assert
yourself and review every decision. Goal: three correct routing decisions in a row.
Enable cron, but disallow automatic PR creation. Goal: at least 10 successful runs for one skill.
Let trusted skills open PRs; humans still merge. Goal: one skill reaches 20 runs at a 95% pass rate.
Allow one safe skill (docs cleanup, lint fixes, typo fixes) to auto-PR after validation. Goal: one full week with no failed auto-created PRs.
Quorum Mode — when intake wakes the router too often, have several cheap models vote and only escalate if a majority agree.
Ratchet Mode — for a metric that should only ever improve (lint warnings, failing tests, bundle size, type errors): it may go down, never up.
Sparring Mode — one agent writes a failing test, another fixes the code; the tester can't touch implementation, the fixer can't weaken tests, disputes go to a human.
Compost Mode — run weekly: review failed tasks, repeated auditor failures, broken assertions, rejected PRs, and cost spikes, then output at most three proposals (a new policy rule, a better prompt, a new assertion, a skill demotion, or dead code removal).
| Signal | Meaning | Action |
|---|---|---|
| Budget exceeded | The loop spent too much | Run make audit , reduce active cycles or use cheaper models |
| Assertion failed | Previously working behavior broke | Check commits since the last pass |
| Skill demoted | Reliability dropped | Inspect the last three failures |
| Executor produced no diff | Task unclear or impossible | Improve router instructions |
| Auditor failed repeatedly | Scope or done_when is weak | Rewrite the task spec |
| Protected path changed | Boundary violation | Reject the diff and queue human review |
| Worktree left behind | Task needs review or cleanup | Inspect the worktree, then clean up |
| Validation gate failed | Code is not done | Fix manually or queue another task |
Using the best model for every step — expensive and unnecessary; save strong models for decisions, cheap models for execution.
Letting the executor decide scope — scope belongs to the router; the executor follows it.
Vague success conditions — "make dashboard better" isn't testable; "update dashboard empty state copy, done when npm test
and npm run lint
pass" is.
No persistent assertions — a fix verified once can silently regress later.
Global trust — a model can be great at docs and unsafe at infrastructure; track trust per skill.
Automating before observing — manual first, draft second, autonomy last.
GitHub Actions — run the validation gate on every agent PR in CI.
Docker — run agents in a container for isolation, installing git, jq, gh, make, defaulting to make tick
.
Branch Protection — require CI pass, code owner review, no direct pushes to main, and signed commits if needed.
Secret Isolation — never expose production secrets to agent jobs; use read-only tokens.
Logging — persist work orders, diffs, verdicts, validation output, cost reports, and trust changes.
Add to your policy: never print environment variables, never read .env
files unless explicitly allowed, never send secrets to model prompts, never modify IAM/auth/billing/deployment policy unattended, never run destructive commands without approval, never install packages without approval, never execute downloaded scripts, never modify CI secrets. Also block paths like .env
, .env.*
, secrets/
, infra/
, terraform/
, migrations/
, .github/workflows/
.
AGENTS.md
↓
policy.md
↓
intake.md
↓
router.md
↓
work-order.json
↓
execute.md
↓
git diff
↓
audit.md
↓
verify.sh
↓
trust.tsv + assertion-ledger.tsv
Every layer is limited: intake doesn't fix, the router doesn't code, the executor doesn't decide success, the auditor doesn't run the repo, the shell gate has the final vote, and the ledger decides future autonomy.
The future of AI engineering is not one giant agent with unlimited authority. It is a controlled system of small agents, each with a narrow role, measurable output, and strict validation. An Agentic OS turns AI from a chat window into an engineering process.
Start small: create AGENTS.md
, add verify.sh
, run one manual cycle, track trust, add assertions, and automate only after the system proves itself. That is how you move from prompting an assistant to operating an autonomous engineering workflow.