By Vilius Vystartas | May 2026
I tested another 10 models across the same 10 agent coding tasks. Four of them were free-tier models — and the range was absurd: Owl Alpha scored 76.7% with zero hard fails, Laguna M.1 scored 10% and produced garbage on 9 out of 10 tasks. The free tier is not free if it costs you debugging time.
Total cost for all 10 models: $0.10. The paid models (6 of 10) came to $0.10 combined.
| # | Model | Score | P/P/F | Cost | Time | Category |
|---|---|---|---|---|---|---|
| 🥇 | Grok 4.3 | |||||
| 81.6% | ||||||
| 7/3/0 | $0.017 | 39.9s | Paid (xAI) | |||
| 🥈 | Perceptron Mk1 | 79.9% | 8/1/1 | $0.002 | 29.3s | Paid (Perceptron) |
| 🥉 | Owl Alpha (free) | 76.7% | 5/5/0 | Free | 83.0s | Free tier |
| 4 | xAI: Grok Build 0.1 | 75.0% | 5/4/1 | $0.034 | 95.3s | Paid (xAI) |
| 5 | OpenAI: GPT Chat Latest | 73.3% | 6/2/2 | $0.043 | 18.7s | Paid (OpenAI) |
| 6 | Mistral Medium 3.5 | 71.6% | 6/2/2 | $0.008 | 12.6s | Paid (Mistral) |
| 7 | Nemotron 3 Nano Omni (free) | 50.0% | 4/2/4 | Free | 23.5s | Free tier |
| 8 | Laguna XS.2 (free) | 49.7% | 3/3/4 | Free | 28.7s | Free tier |
| 9 | Baidu CoBuddy (free) | 40.0% | 4/0/6 | Free | 362.4s | Free tier |
| 10 | Laguna M.1 (free) | 10.0% | 1/0/9 | Free | 89.8s | Free tier |
Grok 4.3 (81.6%, $0.017, 39.9s) — Grok's latest release takes the batch with zero hard fails. Seven clean passes, three partials. Process-monitor was the only full pass it earned that 4.3's competitors missed. xAI's Grok line is quietly consistent — 4.1 Fast (76.7%), 4.20 (75%), and now 4.3 (81.6%) — all within striking distance of the 80%+ club without crossing into premium pricing.
Perceptron Mk1 (79.9%, $0.002, 29.3s) — A brand new family debuts at nearly 80%, with eight passes — the most in the batch — for two-tenths of a cent. The one failure (regex-extract at 17%) is a known weakness for small models. At this price-to-pass ratio, Perceptron Mk1 is the value story of this batch.
Owl Alpha (free, 76.7%, 83.0s) — A free model with zero hard fails and 5 full passes. That's the standout free-tier result. Takes 2x longer than paid models for some tasks (24s on csv-stats vs 1-3s for the field), but the code is functional. If latency isn't critical, this is usable.
Four free models. Results:
| Model | Score | Verdict |
|---|---|---|
| Owl Alpha | 76.7% | |
| Usable — zero hard fails, 5/10 full passes. Slow but functional. | ||
| Nemotron 3 Nano Omni | 50.0% | |
| Mixed — half of tasks hit output cap at 400 tokens. Hit or miss. | ||
| Laguna XS.2 | 49.7% | |
| Unreliable — 400-token cap kills complex responses. | ||
| Baidu CoBuddy | 40.0% | |
| Frustrating — 362 seconds total. Half the tasks hit output cap at 399 tokens. Waiting 6 minutes for 40% accuracy is not a good trade. | ||
| Laguna M.1 | 10.0% | |
| Broken — 1/10 passes. Every response capped at 400 tokens. Do not use. |
The free tier cap of 399-400 output tokens is the real problem. Models like Laguna M.1 and CoBuddy truncate every response, turning what could be a partial into a fail. Owl Alpha works despite the cap because its outputs are concise enough to fit.
Pay $0.002 for Perceptron Mk1 and get 8/10 passes, or use Laguna M.1 free and get 1/10. The math is not subtle.
GPT Chat Latest (73.3%, $0.043) — OpenAI's catch-all endpoint was solid on easy tasks (file-parse, csv-stats, sql-query all passed) but fell apart on fix-bug (0%) with a lengthy, expensive hallucination. The most expensive model in the batch and it doesn't crack 75%.
Mistral Medium 3.5 (71.6%, $0.008) — Fastest model in the batch at 12.6s total, but the process-monitor task hit a 504 Gateway Timeout and scored 0%. A timeout fail on a model that otherwise looks strong carries a disproportionate penalty — without it, Medium 3.5 would be at 79.5%.
Laguna M.1 (10%) — The worst score in any batch I've run. Seven of its task responses were blank 400-token output cap fills. Not worth listing on OpenRouter.
| Model | Score | Cost | $/%-pt |
|---|---|---|---|
| Owl Alpha (free) | 76.7% | $0 | $0 |
| Nemotron 3 Nano Omni (free) | 50.0% | $0 | $0 | | Laguna XS.2 (free) | 49.7% | $0 | $0 | | Baidu CoBuddy (free) | 40.0% | $0 | $0 | | Laguna M.1 (free) | 10.0% | $0 | $0 | | Perceptron Mk1 | 79.9% | $0.002 | $0.0024 | | Mistral Medium 3.5 | 71.6% | $0.008 | $0.0108 | | Grok 4.3 | 81.6% | $0.017 | $0.0209 | | xAI: Grok Build 0.1 | 75.0% | $0.034 | $0.0450 | | GPT Chat Latest | 73.3% | $0.043 | $0.0584 |
Free models dominate the $/%-pt table by definition, but only Owl Alpha is actually usable. Among paid models, Perceptron Mk1 at $0.0024/%-pt is the efficiency winner — 24x cheaper per point than GPT Chat Latest.
Same setup as previous batches: ten real-world agent coding tasks — file operations, shell commands, error recovery, data parsing, SQL queries — tested via OpenRouter. Max tokens: 400. Temperature: 0.1. Pattern-matching scoring against expected outputs.
Pre-flight verification caught zero failures this batch. Total cost: $0.10. Total dataset: 168 models tested across cloud and local.
Full results and per-task scores: benchmarks.workswithagents.dev