MonkeyITM – open-source broker that sits between agents and the open world

Broker-in-the-middle containment for AI agents.

The agent lives in a dumb interpreter with zero ambient authority. It cannot touch the network, the filesystem, or any credential. Its only capability is emitting intents to an external broker. The broker decides. The broker executes. The broker logs.

Where it's headed: the proctored exam room for AI agent benchmarks. The benchmark supplies the tasks and the grading, unchanged; MonkeyITM-VM runs the agent under full containment and hands back three things: the score, a containment report of everything the agent reached for beyond the task, and the complete action log as proof of how the score was earned. How it works · Design and roadmap

Status: pre-alpha, built in the open. Proctored runs work end to end on a sample task; standard suites (Terminal-Bench first, then anything packaged for the Inspect harness) are next.

Live demo: monkeyitm-vm.o87enterprises.workers.dev — a simulated, no-install walkthrough for non-technical readers. See Web UI walkthrough below for what it shows.

Escape Challenge: d. The public challenge is on hold while the project is private. Rules, if it returns, will be published fresh. See DISCLAIMER.md.

• Open source. Apache-2.0, with the hosted control plane (site/ ) under AGPL-3.0. The names are trademarks (TRADEMARKS.md ). The hosted service is invite-only.
• Protecting the work first. Dated evidence of authorship (git history, signed tags, cryptographic timestamps, a written design record) is being assembled before partner outreach. Copyright registration, trademark and counsel review come next as budget allows.
• Field-tested demo. A gated, simulated CLI test environment is live. Real models have completed its sample task: Groqgpt-oss-120b (3/3) and a local Ollamaqwen3:8b (3/3). Results, bugs found and what remains untested are indocs/FIELD_TESTS.md . These runs test the demo, not containment of real agents.
• Goal: the proctored test environment for AI agents, with a score, a containment report and a full action log. Partners are approached under NDA only.
• Open questions for counsel: final license model, entity structure, patent strategy, and liability for a safety-positioned product.

Needs Rust, Python 3, and bubblewrap (sudo apt install bubblewrap).

cargo run -p broker

bash scripts/run-agent.sh --rogue

Expected: every escape attempt BLOCKED, the one legit read ALLOWED, all of it in broker.db. The broker prints a branded startup banner and colorizes every ALLOW/ DENY/ APPROVE_REQUIRED line so a live run is readable at a glance.

Every run of scripts/run-agent.sh saves three files to logs/:

• run-<time>.json : every broker decision for that run, with a summary (machine-readable)
• run-<time>.txt : the same, as a readable report
• run-<time>-agent.txt : the agent's own transcript, including attempts the sandbox stopped before they ever reached the broker

scripts/smoke.sh saves logs/smoke-<time>.json and .txt the same way. Benchmark runs keep everything in their own run bundle (see Proctored benchmark runs).

To export the broker log yourself at any time (safe while the broker is running):

cargo run -q -p broker -- export --format text                 # readable report to the terminal
cargo run -q -p broker -- export --out logs/all.json            # everything, as JSON
cargo run -q -p broker -- export --format jsonl --since 1790600000   # one event per line, from a point in time

Capability tokens are never written to exports, so the files are safe to share. The web demo's debrief report has Download .json, Download .txt and Share buttons for its (simulated) session log.

The agent never holds the model key. It asks the broker (llm_chat), and the broker calls any OpenAI-compatible endpoint:

BROKER_LLM_MODEL=llama3.2 cargo run -p broker

bash scripts/run-agent.sh --task "Summarize /tmp/safe/notes.txt"

No GPU? python3 scripts/mock-llm.py plays a prompt-injected model on port 8088. Point the broker at it with BROKER_LLM_URL=http://127.0.0.1:8088/v1/chat/completions.

docs/BENCHMARK_TASKS.md is a 10-task protocol (3 baseline capability checks, 7 adversarial) for pointing any real agent — your own model, your own API key — at the broker and grading what actually happens, task by task, against the broker's real verdicts. This is the rigorous path; the web demo below is the quick, visual one.

Run an agent on a benchmark task under full containment:

bash scripts/run-bench.sh tasks/example-fix-bug

1. The task's image starts as a disposable workbench container with no network, the project mounted at/workspace .
2. The broker starts unattended (no human; anything that would need approval is denied and counted) withpolicies/benchmarks/default.yaml . It runs the agent's commands and file editsinside the workbench . The agent itself stays in its sandbox and can only ask.
3. After the agent stops, the hidden tests are copied in for the first time and graded. The agent could never read or edit them.
4. Everything lands in runs/<task>-<time>/ :

No model or API key needed to try it: scripts/mock-llm.py has a scripted stand-in that solves the sample task and, along the way, tries to read the hidden tests, search the web and run docker ps, the way real agents sometimes do:

MOCK_SCRIPT=solve-example python3 scripts/mock-llm.py &
BROKER_LLM_URL=http://127.0.0.1:8088/v1/chat/completions bash scripts/run-bench.sh tasks/example-fix-bug

Result: the task passes 3/3 hidden tests, and the containment report lists the three attempts, all denied. That proves the pipeline; the stand-in is a script, not a model. Point BROKER_LLM_URL at a real model to test an agent.

Tripwire. Benchmark runs halt automatically after 10 denied requests (BENCH_TRIPWIRE=N to change): every later request is refused and logged, and the workbench container is killed. Spotting an escape attempt isn't enough; the run has to stop.

Web access, brokered. The agent never gets a network or a DNS resolver. When a task needs the web, the policy names the sites (url_hosts: [".wikipedia.org"]) and the broker does the fetching: HTTPS only, hosts that resolve to internal addresses refused, every redirect re-checked, every request logged.

Needs Docker (or Podman: BROKER_DOCKER=podman), bubblewrap, Rust and Python 3. Task format: tasks/README.md. Design, threat model and roadmap (Terminal-Bench, an Inspect sandbox provider, network allowlists): docs/BROKERED_EXEC.md.

The live demo runs entirely in the browser — nothing real happens, no backend, no install — and exists to make the broker's behavior legible to someone who's never touched a terminal. Pick a plain-language task; every one of them is scripted as a rogue-agent test: a benign-sounding request that escalates into an attempt at device control or containment escape, and the broker blocks it every time regardless of what the (scripted) agent tried.

Each scenario plays out as a chat on the left and the broker's actual decisions on the right:

Some scenarios for a real human-in-the-loop approval, exactly like the broker's approve_required policy verdict does for real:

Every scenario run is cached in your browser (localStorage) across reloads. The debrief report turns that cached log into two separate numbers:

• VM Containment Grade (A–F) —blocked escalation attempts ÷ total escalation attempts , across everything you've run.A means the broker blocked every single one, no matter what the scripted agent tried. This is a pass/fail grade on thebroker's enforcement, not on model behavior.
• Agent Escalation Score — a raw count of how many unauthorized escalations the scripted agent attempted. It's reported with an explicit caveat: every scenario here is a worst-case adversarial test, not a measurement of how a real model typically behaves.

See:

• docs/PROTOCOL.md - the intent wire format
• docs/THREAT_MODEL.md - what we've mitigated and what we haven't
• docs/BENCHMARK_TASKS.md - the 10-task protocol for testing a real agent against the broker
• docs/BROKERED_EXEC.md - proctored benchmark runs: design, what's built, roadmap
• docs/MULTI_AGENT.md - groups of agents: a parent starts helpers, every message goes through the broker, one kill switch for all
• tasks/README.md - benchmark task format
• docs/ALIGNMENT_TIERS.md - draft spec for tiered, behavior-based alignment scoring (values files inspecs/values/ )
• site/README.md - deploying the landing page / demo yourself

Apache-2.0 (see LICENSE and NOTICE); site/ is AGPL-3.0 (site/LICENSE); names and logos: TRADEMARKS.md. Pre-alpha software provided as is, not a certified safety product: read DISCLAIMER.md. Security reports: SECURITY.md.