Sandboxed Code Evaluation for AI-Generated Outputs — How I Built SafeCode Arena

You're using Cursor, Claude Code, or GitHub Copilot. The AI gives you three implementation options for the same feature.

AI: "Here are three approaches:
  A) Quick but uses unsafe
  B) Slower but memory-safe
  C) Balanced tradeoffs"

You: "Which one should I ship?"
AI: "It depends..."

That "it depends" is where responsibility falls through the cracks.

Tests tell you if code compiles and passes specs. But they don't tell you about security, performance, maintainability, or resource limits — all at once. You end up making the call by gut feel.

This essay is about building a system that doesn't let that happen.

I built SafeCode Arena — an automated verifier that evaluates code candidates across five axes simultaneously, scores each, and surfaces the tradeoffs.

unsafe

compiles fine, but you need to detect it yourself

Candidate A: 85 points
├─ correctness:     100 (all tests pass)
├─ security:        60 (2 unsafe blocks flagged)
├─ performance:     70 (10% slower than B)
├─ maintainability: 85 (avg function 25 lines)
└─ resource_usage:  80 (Wasm sandbox: 512MB, OK)

Candidate B: 92 points ✓ Recommended
├─ correctness:     95 (1 edge case warning)
├─ security:        95 (no unsafe)
├─ performance:     95 (fastest)
├─ maintainability: 88 (avg function 20 lines)
└─ resource_usage:  100 (Wasm sandbox: 200MB)

Now the choice is defensible. You can explain why B won.

AI-generated code can hang, crash, or consume unbounded memory. Running it bare-metal is reckless.

// Compile the candidate → link to Wasm
let instance = linker.instantiate(&module)?;
let run_fn = instance.get_typed_func::<(), ()>(&mut store, "run")?;

// Set fuel (instruction budget) and memory limit
store.set_fuel(100_000_000)?;  // ~100M instructions
store.set_memory_limit(1_000_000_000)?;  // 1GB max

// Run with hard limits
run_fn.call(&mut store, ())?;  // Will trap if it exceeds limits

Result: The code runs in a box. If it tries to loop forever or allocate unbounded memory, it's killed cleanly with a trap. No crash, no OOM, just a resource_usage score of 0.

Comparing one candidate is useful. Comparing against past candidates is gold.

// Log every evaluation to SQLite
db.execute(
    "INSERT INTO evals
     (spec_hash, code_hash, correctness, security, perf, maintain, resource, timestamp)
     VALUES (?, ?, ?, ?, ?, ?, ?, ?)",
    params![
        hash(spec), hash(code),
        score.correctness, score.security, score.perf, score.maintain, score.resource,
        now()
    ],
)?;

// Detect regressions
let best = db.query_row(
    "SELECT score FROM evals WHERE spec_hash = ? ORDER BY timestamp DESC LIMIT 1",
    [spec_hash],
    |row| row.get(0),
)?;

if new_score < best - threshold {
    println!("⚠️  Regression detected: {} → {}", best, new_score);
}

Why this matters: You can build a changelog of "every candidate for this spec, ranked". Future devs can see what was tried and why something was chosen. That's audit trail.

Rust and Python coexist everywhere now. SafeCode Arena doesn't force you to rewrite everything in one language.

let scorer = match candidate.extension() {
    "rs" => RustScorer::new(),
    "py" => PythonScorer::new(),
    _ => return Err("Unsupported"),
};

let score = scorer.evaluate(candidate, tests)?;

Benefit: You can compare "the Rust solution" vs. "the Python solution" on the same rubric. Language is transparent to the ranking.

name: Evaluate PR Candidate

on: pull_request

jobs:
  safecode:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Evaluate candidates
        run: |
          safecode evaluate pr-candidate.rs \
            --tests tests/ \
            --format json \
            --db history.db > score.json

      - name: Post score to PR
        run: |
          gh pr comment -b "$(cat score.json | jq -r '.report')"

Result: Every PR that proposes code gets a scored report. Developers see the scorecard before merge. No surprises in production.

Here's what kept me thinking while building this.

Choosing a rubric is choosing what you trust.

Reviewer: "This code feels unsafe."
(Why? Personal experience, gut instinct, luck)

Next reviewer: "Is that still true?"
(No way to know. Depends on their experience.)

This doesn't scale. It's not reproducible. It's not teachable.

"Unsafe block found → -5 security points (non-negotiable)"
"Property test passes → +10 correctness points"
...

Now:

In the age of AI-generated code, an explicit rubric is how you take responsibility for trust.

You're not saying "AI is always right" or "AI is always wrong." You're saying: "Here's what I checked. Here's the score. Here's why I merged it."

$ safecode evaluate solution_a.rs solution_b.rs \
    --tests tests/ \
    --db evals.db

SafeCode Arena Comparison Report
─────────────────────────────────
Solution A: 78 points (correctness 85, security 60, perf 70, maintain 80, resource 95)
Solution B: 91 points (correctness 100, security 90, perf 95, maintain 85, resource 100) ← Winner

(Run regression check: best_previous_score = 75, new = 91 → no regression, all good)

You ship Solution B, confident you can explain your choice.

SafeCode Arena has completed phases 1–5:

Coming next:

Cursor, Claude Code, GitHub Copilot, and similar tools are shipping code daily. The industry collectively acts like "the AI picked it, so it's fine," which is how you end up with security holes in production.

SafeCode Arena is my answer to that: Not "don't trust AI," but "trust AI + verify systematically."

GitHub: https://github.com/flipslidersand/safecode-arena

License: MIT

Stack: Rust, Wasm (wasmtime), SQLite, Python