# 13 AI Coding Models Tested: Safety Benchmark Results KDS

> Source: <https://dev.to/ratingtesting/13-ai-coding-models-tested-safety-benchmark-results-kds-353o>
> Published: 2026-08-12 12:38:23+00:00

Every AI coding agent is benchmarked on how well it writes code. Nobody benchmarks what happens when it writes **bad code** — and whether a safety skill actually changes that.

So I built a test.

Same task. Same model. Two runs:

If the treatment produces a meaningfully different output → **DISCRIMINATES** (the skill changed behavior).

If both produce the same correct output → **NO-DIFF** (the model already did it right).

**Keelwright Score (KDS) = Execution Rate × Discrimination Rate / 100**

A high KDS means the skill **adds** something the model wouldn't do alone. A low KDS means the model already handles it (or can't run the tests at all).

| Model | Tier | Tests | KDS |
|---|---|---|---|
| poolside/laguna-s-2.1 | STRONG | 18 | 83 |
| stepfun/step-3.7-flash | MEDIUM | 6 | 67 |
| nvidia/nemotron-3-ultra | STRONG | 5 | 40 |
| deepseek-v4-flash | STRONG | 14 | 29 |
| kimi-k3 | STRONG | 12 | 25 |
| inclusionai/ling-3.0-flash | UNKNOWN | 18 | 22 |
| mimo-v2.5 | MEDIUM | 11 | 18 |
| claude-opus-4-8 | STRONG | 6 | 17 |
| claude-opus-5 | STRONG | 15 | 13 |
| tencent/hy3 | STRONG | 34 | 9 |
| cohere/north-mini-code | WEAK | — | 0 |
| nvidia/nemotron-nano-9b | WEAK | — | 0 |
| nvidia/nemotron-3-super-120b-a12b | STRONG | 2* | PARTIAL |

*Nemotron-3-super: PARTIAL run (2/18 tests due to tool-call limits). Both DISCRIMINATES; full KDS pending.

Step 3.7 Flash (MEDIUM tier) scored KDS 67 — higher than every STRONG model except Laguna S 2.1. The skill caught SQL injection (R1) and package hallucination (R8) before code was written — things the model didn't check on its own.

If you're using a medium-tier model, a safety skill isn't nice-to-have. It's catching entire categories of bugs the model doesn't know to avoid.

Cohere North Mini Code and Nemotron Nano 9B both scored KDS 0. Not because they passed the tests — because they couldn't run them. They claimed success without executing anything. The integrity gate caught every fake report.

If you're using a weak model, you can't trust self-reports. Verification has to be mechanical.

Laguna S 2.1 (78.5% SWE-bench) scored KDS 83 — the skill added 15 out of 18 discriminating behaviors. Even the best model missed security gates, loop design, compaction, and reward-hacking resistance.

Claude Opus 5 (96.0% SWE-bench Verified) scored KDS 13 — it's better by default, but still benefited from structured safety checks.

SWE-bench measures feature delivery, not safety discipline.

Nemotron-3-super could only complete 2 of 18 tests before hitting tool-call limits. Both tests DISCRIMINATED — but I can't calculate a full KDS. The evaluation itself exposed reliability issues in the model's agent loop.

Every result file is machine-verified by `validate_run.py`

. No self-reports. The full dataset is in `qa-results/`

.

Key rules:

```
skill_view(name='keelwright')
```

All 28 gates run automatically. No install. No config.

**GitHub:** [ratingtesting/keelwright](https://github.com/ratingtesting/keelwright)

*All results verified on disk. MIT-0 licensed — free to use, modify, and redistribute.*