Sandboxed Code Evaluation for AI-Generated Outputs — How I Built SafeCode Arena

A developer built SafeCode Arena, an automated verifier that evaluates AI-generated code candidates across five axes—correctness, security, performance, maintainability, and resource usage—and scores them to help developers make defensible choices. The tool compiles candidates to WebAssembly with hard limits on instructions and memory, and logs evaluations to SQLite to detect regressions and build an audit trail. It supports both Rust and Python, allowing cross-language comparisons on the same rubric.

You're using Cursor, Claude Code, or GitHub Copilot. The AI gives you three implementation options for the same feature. AI: "Here are three approaches: A Quick but uses unsafe B Slower but memory-safe C Balanced tradeoffs" You: "Which one should I ship?" AI: "It depends..." That "it depends" is where responsibility falls through the cracks. Tests tell you if code compiles and passes specs. But they don't tell you about security, performance, maintainability, or resource limits — all at once. You end up making the call by gut feel. This essay is about building a system that doesn't let that happen. I built SafeCode Arena https://github.com/flipslidersand/safecode-arena — an automated verifier that evaluates code candidates across five axes simultaneously, scores each, and surfaces the tradeoffs. | Axis | Weight | Computation | |---|---|---| Correctness | 50% | compile 40% + tests 40% + property tests 20% | Security | 20% | unsafe heuristics 50% + clippy warnings 50% | Performance | 15% | relative compile+test time across candidates | Maintainability | 10% | function-length heuristics 60% + clippy 40% | Resource Usage | 5% | pass/fail of sandboxed Wasm execution | unsafe compiles fine, but you need to detect it yourself Candidate A: 85 points ├─ correctness: 100 all tests pass ├─ security: 60 2 unsafe blocks flagged ├─ performance: 70 10% slower than B ├─ maintainability: 85 avg function 25 lines └─ resource usage: 80 Wasm sandbox: 512MB, OK Candidate B: 92 points ✓ Recommended ├─ correctness: 95 1 edge case warning ├─ security: 95 no unsafe ├─ performance: 95 fastest ├─ maintainability: 88 avg function 20 lines └─ resource usage: 100 Wasm sandbox: 200MB Now the choice is defensible . You can explain why B won. AI-generated code can hang, crash, or consume unbounded memory. Running it bare-metal is reckless. js // Compile the candidate → link to Wasm let instance = linker.instantiate &module ?; let run fn = instance.get typed func::< , &mut store, "run" ?; // Set fuel instruction budget and memory limit store.set fuel 100 000 000 ?; // ~100M instructions store.set memory limit 1 000 000 000 ?; // 1GB max // Run with hard limits run fn.call &mut store, ?; // Will trap if it exceeds limits Result : The code runs in a box. If it tries to loop forever or allocate unbounded memory, it's killed cleanly with a trap. No crash, no OOM, just a resource usage score of 0. Comparing one candidate is useful. Comparing against past candidates is gold. // Log every evaluation to SQLite db.execute "INSERT INTO evals spec hash, code hash, correctness, security, perf, maintain, resource, timestamp VALUES ?, ?, ?, ?, ?, ?, ?, ? ", params hash spec , hash code , score.correctness, score.security, score.perf, score.maintain, score.resource, now , ?; // Detect regressions let best = db.query row "SELECT score FROM evals WHERE spec hash = ? ORDER BY timestamp DESC LIMIT 1", spec hash , |row| row.get 0 , ?; if new score < best - threshold { println "⚠️ Regression detected: {} → {}", best, new score ; } Why this matters : You can build a changelog of "every candidate for this spec, ranked". Future devs can see what was tried and why something was chosen. That's audit trail. Rust and Python coexist everywhere now. SafeCode Arena doesn't force you to rewrite everything in one language. js let scorer = match candidate.extension { "rs" = RustScorer::new , "py" = PythonScorer::new , = return Err "Unsupported" , }; let score = scorer.evaluate candidate, tests ?; Benefit : You can compare "the Rust solution" vs. "the Python solution" on the same rubric. Language is transparent to the ranking. .github/workflows/evaluate-candidate.yml name: Evaluate PR Candidate on: pull request jobs: safecode: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - name: Evaluate candidates run: | safecode evaluate pr-candidate.rs \ --tests tests/ \ --format json \ --db history.db score.json - name: Post score to PR run: | gh pr comment -b "$ cat score.json | jq -r '.report' " Result : Every PR that proposes code gets a scored report. Developers see the scorecard before merge. No surprises in production. Here's what kept me thinking while building this. Choosing a rubric is choosing what you trust. Reviewer: "This code feels unsafe." Why? Personal experience, gut instinct, luck Next reviewer: "Is that still true?" No way to know. Depends on their experience. This doesn't scale. It's not reproducible. It's not teachable. "Unsafe block found → -5 security points non-negotiable " "Property test passes → +10 correctness points" ... Now: In the age of AI-generated code, an explicit rubric is how you take responsibility for trust. You're not saying "AI is always right" or "AI is always wrong." You're saying: "Here's what I checked. Here's the score. Here's why I merged it." 1. AI generates 3 candidates you pick the most promising 2. Run the scorer $ safecode evaluate solution a.rs solution b.rs \ --tests tests/ \ --db evals.db 3. Results SafeCode Arena Comparison Report ───────────────────────────────── Solution A: 78 points correctness 85, security 60, perf 70, maintain 80, resource 95 Solution B: 91 points correctness 100, security 90, perf 95, maintain 85, resource 100 ← Winner 4. Confidence Run regression check: best previous score = 75, new = 91 → no regression, all good 5. Merge You ship Solution B, confident you can explain your choice. SafeCode Arena has completed phases 1–5: Coming next: Cursor, Claude Code, GitHub Copilot, and similar tools are shipping code daily. The industry collectively acts like "the AI picked it, so it's fine," which is how you end up with security holes in production. SafeCode Arena is my answer to that: Not "don't trust AI," but "trust AI + verify systematically." GitHub : https://github.com/flipslidersand/safecode-arena https://github.com/flipslidersand/safecode-arena License : MIT Stack : Rust, Wasm wasmtime , SQLite, Python