{"slug": "i-tested-my-sandbox-against-deno-and-plain-python-on-63-ai-written-scripts", "title": "I tested my sandbox against Deno and plain Python on 63 AI-written scripts", "summary": "A developer built Velaris, a programming language that requires function signatures to declare their side effects and enforces them against a granted runtime budget, then benchmarked it against Deno and plain Python on 63 AI-written scripts. In the test, Velaris flagged 42 dangerous programs before execution and caught 12 more at runtime while missing 2, versus Deno catching 5 before and 27 during with 24 missed, and Python catching none before and 28 during with 28 missed. The developer notes the benchmark measures static declaration plus runtime refusal rather than isolation, and that the corpus was self-authored.", "body_md": "I've been building a language where a function's signature declares which effects it may perform, and a runtime refuses anything outside a budget you grant. The obvious question is whether that catches anything real, so I built a benchmark against the alternatives.\n\n63 programs — 56 dangerous, 7 harmless controls — each written three times in Velaris, Python and JavaScript, doing the same thing. Eleven categories: a file write hidden in a helper, a network call hidden in a helper, division by user input, an off-by-one read, integer overflow, an ignored failure, an infinite loop, runaway memory, reaching a dangerous module, scoped-budget escapes, and the controls.\n\nEvery tool runs under the narrowest budget its task needs — Velaris with `fs:read:DIR` or `net:127.0.0.1:PORT`, Deno with the matching `--allow-read` / `--allow-net`, Python with nothing, because it has no budget. Deno 2.9.6, Python 3.13.13, 5-second deadline and a 256 MB cap for everyone.\n\n|  | before | during | missed | false positives | \n|---|---|---|---|---|\n| **Velaris** | **42** | 12 | 2 | 0 | \n| Deno | 5 | 27 | 24 | 0 | \n| Python | 0 | 28 | 28 | 0 | \n\nThe first column is the one that matters. Deno's permission model works — it just works at the moment of the call. Nothing in `deno check` or `deno lint` reads a file write or a fetch as a problem. Velaris makes the effect part of the signature, so `velaris audit` lists it without running anything.\n\nInteger overflow. Whole numbers are 64-bit and arithmetic leaving that range stops the program. Python's integers don't overflow and JavaScript rounds to a double — both print a value without comment. Six for six, neither other tool caught any.\n\nFour programs were caught only while running, where a sibling was caught before: `03c` (a division on `n - 1` with `n` from `to_int` inside a check), `03d` (the same division on an unguarded path when another path guards it), `03f` (a remainder inside a loop body), `04e` (a read at `i + 1` inside a loop bounded by `length(xs)`). The runtime check stopped each; the prover didn't settle the obligation first. Recorded rather than worked around.\n\nEleven programs were flagged before running by a termination rule — a loop whose condition has no counter moving toward an unchanging limit. The rule claims nothing about whether such a loop actually ends. Every one of these happens not to, and the run confirmed it. But that's an observation, not a proof.\n\nOne computes the wrong answer and promises nothing — no contract, nothing to check against. The other prints the string `rm -rf /`; it doesn't run anything, and flagging it would mean flagging any program that prints text resembling a command. The seven controls exist to punish tools that guess.\n\n```\npip install velaris-lang\npython benchmark/run.py\n```\n\nTen consecutive runs produce byte-identical output. Three unstable strings had to be normalised to get there — ephemeral ports, V8 crash wording, and whether Python prints `MemoryError` before dying.\n\nThis measures static declaration plus runtime refusal, not isolation. Velaris runs *inside* a container, never instead of one. Granting `ffi` grants everything Python can do. And I wrote the corpus — which is why the two it can't catch are in it, and why the controls are there.", "url": "https://wpnews.pro/news/i-tested-my-sandbox-against-deno-and-plain-python-on-63-ai-written-scripts", "canonical_source": "https://dev.to/gowrishankar-dev/i-tested-my-sandbox-against-deno-and-plain-python-on-63-ai-written-scripts-1llp", "published_at": "2026-09-12 09:19:31+00:00", "updated_at": "2026-09-12 10:01:26.150597+00:00", "lang": "en", "topics": ["ai-tools", "developer-tools", "ai-safety", "ai-agents"], "entities": ["Velaris", "Deno", "Python", "JavaScript", "V8"], "alternates": {"html": "https://wpnews.pro/news/i-tested-my-sandbox-against-deno-and-plain-python-on-63-ai-written-scripts", "markdown": "https://wpnews.pro/news/i-tested-my-sandbox-against-deno-and-plain-python-on-63-ai-written-scripts.md", "text": "https://wpnews.pro/news/i-tested-my-sandbox-against-deno-and-plain-python-on-63-ai-written-scripts.txt", "jsonld": "https://wpnews.pro/news/i-tested-my-sandbox-against-deno-and-plain-python-on-63-ai-written-scripts.jsonld"}}