{"slug": "run-ai-proposed-shell-commands-in-a-systemd-probe-before-you-run-them-for-real", "title": "Run AI-Proposed Shell Commands in a systemd Probe Before You Run Them for Real", "summary": "MonkeyCode's product outreach presents a systemd-based probe that runs AI-proposed shell commands in a sandboxed environment before execution on a real server. The probe uses systemd-run with strict protections, records stdout, stderr, and exit code, and compares results against declared expectations to catch unsafe commands.", "body_md": "Disclosure: This article was prepared as part of MonkeyCode's product outreach. A free model can produce shell commands that are syntactically valid but operationally unsafe, and the free server option gives you somewhere to stage them. The problem is not whether the command parses; it is whether the command's observable effect matches the intent.\n\n`bash -n`\n\nand ShellCheck catch syntax and common mistakes, but they do not know the runtime context of your server. A command such as `docker compose up -d --remove-orphans`\n\nmay be valid on a workstation but may pull new images, expose ports, or use the wrong Compose file on the server. Generated commands are often written from generic knowledge, without the local usernames, mount points, and firewall rules.\n\nThe gate in this article runs the proposed command in a short-lived systemd service with a read-only system tree, no new privileges, restricted devices, and bounded memory and runtime. It records stdout, stderr, and exit code in a JSON artifact, then compares the result to a declared expectation before the command is allowed anywhere near a real shell.\n\nDefine four values for each generated command:\n\n`COMMAND`\n\n— the exact string from the model.`EXPECT_RC`\n\n— the exit code you want, usually `0`\n\n.`EXPECT_MARKER`\n\n— a literal substring that must appear in stdout, or empty for none.`ALLOW_WRITE_DIR`\n\n— optional path if the command needs write access, added as `ReadWritePaths=`\n\n.Separating the command from the expected observable effect prevents 'it exited zero' from hiding a command that wrote no output but did something else.\n\nSave the following as `probe_cmd.sh`\n\nand make it executable. It uses `systemd-run`\n\nunder `sudo`\n\n; change `RuntimeMaxSec=12`\n\nand `MemoryMax=64M`\n\nfor longer checks.\n\n``` bash\n#!/usr/bin/env bash\nset -uo pipefail\n\nPROPOSED_CMD=\"${1:?usage: probe_cmd.sh 'command'}\"\nEXPECT_RC=\"${EXPECT_RC:-0}\"\nEXPECT_MARKER=\"${EXPECT_MARKER:-}\"\nALLOW_WRITE_DIR=\"${ALLOW_WRITE_DIR:-}\"\nEVIDENCE_DIR=\"${EVIDENCE_DIR:-/var/tmp/ai-probes}\"\nPROBE_ID=\"ai-probe-$(date +%s)-$$\"\n\nmkdir -p \"$EVIDENCE_DIR\"\nout_file=\"$EVIDENCE_DIR/$PROBE_ID.out\"\nerr_file=\"$EVIDENCE_DIR/$PROBE_ID.err\"\nlog_file=\"$EVIDENCE_DIR/$PROBE_ID.json\"\n\nextra_props=()\nif [[ -n \"$ALLOW_WRITE_DIR\" ]]; then\n  extra_props+=(--property=\"ReadWritePaths=$ALLOW_WRITE_DIR\")\nfi\n\nsudo systemd-run --wait --pipe --quiet --property=User=nobody --property=NoNewPrivileges=yes --property=ProtectSystem=strict --property=ProtectHome=read-only --property=PrivateTmp=yes --property=PrivateDevices=yes --property=MemoryMax=64M --property=RuntimeMaxSec=12 \"${extra_props[@]}\" bash -lc \"$PROPOSED_CMD\" > \"$out_file\" 2> \"$err_file\"\nrc=$?\n\nmarker_ok=0\nif [[ -z \"$EXPECT_MARKER\" ]]; then\n  marker_ok=1\nelif grep -qF -- \"$EXPECT_MARKER\" \"$out_file\"; then\n  marker_ok=1\nfi\n\njq -n --arg id \"$PROBE_ID\" --arg cmd \"$PROPOSED_CMD\" --arg expected_rc \"$EXPECT_RC\" --arg rc \"$rc\" --arg expected_marker \"$EXPECT_MARKER\" --arg marker_ok \"$marker_ok\" --arg stdout \"$(cat \"$out_file\")\" --arg stderr \"$(cat \"$err_file\")\" '{probe_id:$id, command:$cmd, expected_exit:($expected_rc|tonumber), observed_exit:($rc|tonumber), expected_marker:$expected_marker, marker_satisfied:($marker_ok|tonumber), stdout:$stdout, stderr:$stderr}' > \"$log_file\"\n\npass=1\nif [[ \"$rc\" -ne \"$EXPECT_RC\" ]]; then\n  pass=0\nfi\nif [[ -n \"$EXPECT_MARKER\" && \"$marker_ok\" -ne 1 ]]; then\n  pass=0\nfi\n\nif [[ \"$pass\" -ne 1 ]]; then\n  echo \"probe failed: $log_file\" >&2\n  exit 1\nfi\n\necho \"probe passed: $log_file\"\n```\n\nHere's a read-only smoke test:\n\n```\nEXPECT_MARKER=\"PRETTY_NAME\" ./probe_cmd.sh 'cat /etc/os-release'\n```\n\nIf the marker is missing or the exit code differs, the script prints `probe failed`\n\nand stores the JSON file under `/var/tmp/ai-probes/`\n\n. Inspect it with:\n\n```\njq '.probe_id, .observed_exit, .marker_satisfied' /var/tmp/ai-probes/*.json\n```\n\n| Risk | Systemd property | Effect |\n|---|---|---|\n| Writes outside the intended tree | `ProtectSystem=strict` |\n`/etc` , `/usr` , `/boot` , and `/efi` become read-only |\n| Home directory tampering | `ProtectHome=read-only` |\n`/home` , `/root` , and `/run/user` become read-only |\n| Raw device access | `PrivateDevices=yes` |\nMost device nodes are hidden |\n| Privilege escalation | `NoNewPrivileges=yes` |\nsetuid bits and new capabilities are blocked |\n| Resource runaway |\n`MemoryMax=64M` , `RuntimeMaxSec=12`\n|\nBounds memory and execution time |\n\nThe baseline properties do not block network access. If the command should not reach the network, add one property to the `systemd-run`\n\nline:\n\n```\n--property=IPAddressDeny=any\n```\n\nIf the command is supposed to talk only over a local socket, use:\n\n```\n--property=RestrictAddressFamilies=AF_UNIX\n```\n\nThe probe proves observed behavior in one constrained runtime. It does not prove correctness, idempotency, remote API effects, or safety under a different kernel and systemd version. Commands that need root-owned writes should be reviewed manually, not waved through with `ReadWritePaths=/`\n\n. If you depend on exact network egress, add restrictive address-family and IP-address properties because the sandbox is not a firewall.\n\nThis gate is not for interactive commands, long-running migrations, performance-sensitive benchmarks, or production changes where the probe environment differs from the real target. Use it as a preflight for generated command strings, not as a replacement for code review and a rollback plan.\n\nIf you are using MonkeyCode's free model access to draft server changes, copy the generated command into `probe_cmd.sh`\n\n, set `EXPECT_MARKER`\n\nfrom the effect the assistant claimed to perform, and run the probe on the free server option before the command becomes part of a real runbook. The free access gives you more candidate commands to test; the probe gives you a way to reject the ones whose runtime behavior does not match the claim.", "url": "https://wpnews.pro/news/run-ai-proposed-shell-commands-in-a-systemd-probe-before-you-run-them-for-real", "canonical_source": "https://dev.to/github_7727/run-ai-proposed-shell-commands-in-a-systemd-probe-before-you-run-them-for-real-2aed", "published_at": "2026-08-17 16:36:40+00:00", "updated_at": "2026-08-17 17:13:57.047097+00:00", "lang": "en", "topics": ["ai-safety", "developer-tools", "ai-tools"], "entities": ["MonkeyCode", "systemd", "ShellCheck"], "alternates": {"html": "https://wpnews.pro/news/run-ai-proposed-shell-commands-in-a-systemd-probe-before-you-run-them-for-real", "markdown": "https://wpnews.pro/news/run-ai-proposed-shell-commands-in-a-systemd-probe-before-you-run-them-for-real.md", "text": "https://wpnews.pro/news/run-ai-proposed-shell-commands-in-a-systemd-probe-before-you-run-them-for-real.txt", "jsonld": "https://wpnews.pro/news/run-ai-proposed-shell-commands-in-a-systemd-probe-before-you-run-them-for-real.jsonld"}}