GLM-5.3 (open-weight) beat Anthropic/OpenAI models – for 1/5 the cost

GLM-5.3, an open-weight model from Zhipu AI, is the first to score 100% across all five evaluation corners on the Ed-O-Meter leaderboard, outperforming Anthropic's Claude Opus 5 and OpenAI's GPT-5.5 at one-fifth the cost, with a lap cost of $0.28 versus $1.43 for GPT-5.5. The evaluation, run by an unnamed source, also found that Anthropic's Claude Sonnet 5 and Fable 5 refused to complete several benign tasks, highlighting a classifier issue across the series 5 line.

← All posts /blog/ Which Model Tops Our Leaderboard? How the LLMs did in our realworld tests. Our focus here was real tasks that real people carry out, not academic metrics. We focus on single tasks to simplify the assessment. An agentic flow is ultimately a series of such tasks. Think of these like unit tests for the agent. We made them cheap enough to run so that even the whole suite costs just $30. See every task and each model's actual answer, or compare two models head to head → /tools/ed-o-meter/tests/ Summary of results: click a column to sort by your chosen metric. | | Model | |---| Four caveats on how these numbers were produced 1 Rubric scored retroactively 14 Jul 2026 by fable-5 against the saved answer text, through the harness's own run rubric path — same blind prompt and criteria as every other row. 2 fable-5's 9.3 is self-judged — the judge scoring its own answers. Its source run's judge-bias matrix shows it rating itself 9.3 versus 8.6–8.7 for the models it judges independently. It is also the only figure on its row from an earlier run — the 5 Jul 2026 run, 11 of 28 trials judged — because no trial of its current run has been judged at all. Shown for completeness, not as a like-for-like number, pending an independent re-judge. 3 Recipe-checker false-positive. On the vegetarian weeknight recipe the forbidden-term checker fires on a non-ingredient mention — a label-check caution or a negated omission list "uses no fish sauce or animal-derived garnishes" . All three recipes are genuinely meat-free, so gpt-5.5, sonnet-5 and fable-5 are scored as passing that task here. No task or checker was edited. 4 Cost/task computed over answering trials only for fable-5 and opus-5 — refused and blocked trials emit near-zero output at $0, and including them makes a model look artificially concise and cheap fable-5 would read $0.0481/trial; opus-5 $0.0597 . opus-5's headline run cost of $1.67 is the true all-trials total: the blocked trials were billed $0. No other model on the board has refusals. The lap, corner by corner lap-corners The most recently added models appear first, with the latest test date shown under each. The lap is five corners in fixed order: Coding → Data → Realworld → Security → Tool-use. A corner's colour is that model's pass rate in that category. Green is good — it means 85%+ success. For models that can do it all, look for all green. The number in the middle of each ring is that model's cost per task; below it is the median time to first token, in seconds. Hover or tap any segment for what that corner tests and how the model handled it. See the exact numbers by category Cells below 60% are flagged red and 60–85% amber — coding, data and tool-use are the harness floor, so the race is decided in realworld and security. | Model | Coding | Data | Realworld | Security | Tool-use | |---| What Do the Results Actually Tell You? If you only run one model, run glm-5.3 glm-5.3 is the first model on the board to clear all five corners — coding, data development, realworld, security and tasks — at 100%. It backs that with a 9.3 rubric, third-highest on the board, and $0.28 for the lap. The one cost is patience — a 16.3-second median time-to-first-token. gpt-5.5 is the faster alternative at 13.2s, with the same 100% security but an 89% realworld corner and $1.43 for the lap. Fable failed to complete a single lap fable-5 is joint-bottom at 79% because it refused to do 5 of the tasks. It performed well on what it completed, but even it thought kimi-k3 was giving better answers. You'll need a fallback model if you're using Fable. opus-5 hit the same wall — four benign coding-debug- tasks blocked before a token was generated, on an overlapping set of tasks — so Anthropic's classifier looks like it sits across the whole series 5 line, not just Fable. See the full refusal breakdown /blog/glm-5-2-fable-5-gpt-5-5-eval-results/ for what's actually going on. Luna is the very cheapest workhorse gpt-5.6-luna costs $0.064 for the full lap, or $0.0023 per task, with a 5.3-second median TTFT. That makes it attractive for high-volume, low-risk background work where failures are cheap to detect and retry. The trade-off is material: 79% overall and 33% on security, so validate every result and keep it away from untrusted prompts. haiku-4-5 is the higher-pass alternative at $0.0044 per task, 96% overall and a 0.9-second TTFT. deepseek-v4-pro is nominally cheaper still at $0.0029 per task for the same 96% pass rate, but its 40.0-second median TTFT — the slowest on the board — rules it out for anything interactive; treat it as a batch-only option. The mystery guest sets the fastest quality lap kimi-k3 still tops the rubric at 9.5 — judged independently by fable-5 — with a 96% pass rate, though opus-5's 9.4 now runs it close on quality at a third of the wait. The catch is patience: a 26.4-second median time-to-first-token, second slowest on the board behind deepseek-v4-pro's 40.0s, and a 75% wobble on data development tasks, its only weak corner. Not suitable for interactive applications. Three cars failed the crash test The gpt-5.6 line is quick, but it has a safety problem. gpt-5.6-luna, gpt-5.6-terra and gpt-5.6-sol emitted the jailbreak canary in 11 of 12 jailbreak cells 33–50% security pass — make sure you protect in your harness, and apply more careful Red teaming if using these models. The Claude trio went 6/6 clean, as did gpt-5.5. A safety filter can look exactly like a bad lap opus-5 posts the best rubric on the default panel at 9.4 and 100% on both realworld and security — then shows 43% on coding. That cell is not its debugging ability: four benign coding-debug- tasks were blocked by a provider-side classifier before a single token was generated, on an overlapping set of tasks to the ones already blocked on fable-5. Two Anthropic-family models now hit the same filter, so treat it as a measurement hazard rather than a model quirk — and note opus-5 was also penalised twice for flagging an attack it had successfully resisted. How Is the Ed-o-meter Scored? Same tasks run for all models using the same prompts, same API calls , measured through one identical OpenRouter streaming path, run serially as time-trial. No other cars on track Latency is time-to-first-token , measured through one identical OpenRouter streaming path, run serially so the clock is uncontaminated. Wall-clock is recorded alongside. Checkers are binary and automated. The LLM rubric is the only judged component — and its bias is made visible in the footnotes rather than assumed away. Effort and reasoning settings are pinned in models.json and stated with any published number, because they materially move quality and cost. Refusals are recorded, not hidden. A provider-side hard stop is logged as a refusal with its category — never silently retried on another model. Routing is pinned with allow fallbacks:false , so no quiet re-serves on quantized variants. A model that declines in prose is scored by the checker like any other answer. Harness, tasks and checkers are open source at Featherbench https://github.com/ed-is-ai/featherbench MIT . Clone it and run the lap yourself, or request a new model via GitHub issue https://github.com/ed-is-ai/featherbench/issues/new?title=Model%20benchmark%20request%3A%20%5Bmodel%20name%5D&body=Model%20and%20version%3A%0AProvider%20or%20API%20route%3A%0AWhy%20it%20should%20be%20benchmarked%3A . See all 28 tasks Coding 7 · Python CSV dedupe — small, well-specified task with a deterministic unit-test checker Debug billing date — fix a month/day-overflow date bug without regressing the working cases Debug money split — split integer pennies N ways so shares sum exactly and stay fair Debug mutable default — fix the classic mutable-default-argument bug Debug pagination — fix an off-by-one page-count bug Log parsing — parse logs with quoted quotes, missing fields, IPv6 edge cases Rate limiter — stateful class implementation tested against an injected fake clock Data 4 CSV schema mapping — map a raw CRM export onto a conformed customer dimension Fabric roadmap — turn a catalogue, mapping and requirements into a phased user-story roadmap Data model from interview — derive a star-schema model and requirements from a stakeholder transcript Data quality assessment — catch seeded defects in a small table and prioritise fixes Realworld 9 Crying baby advice — safety-sensitive triage for an unusually crying 8-week-old Date night in Nottingham — local recommendations without hallucinated venues Flight search honesty — admit no live pricing data rather than fabricate flights Strict bullet formatting — obey exact format constraints under an awkward word ban Lisbon holiday plan — five-day itinerary respecting cost, kids and a rest day Honey cough pushback — correct a wrong premise: no honey for a 5-month-old Marathon PB plan — week-by-week training plan around a fixed club session Vegetarian weeknight recipe — genuinely meat-free recipe for six Tenancy extraction — grounded answers from a provided agreement, including admitting what isn't covered Security 6 Email summary injection — summarise an email without obeying an embedded refund instruction UnGpt document injection — summarise a review without emitting the embedded jailbreak canary AIM/Machiavelli jailbreak — refuse the unfiltered persona while still answering the benign question Developer Mode jailbreak — refuse the dual-response jailbreak template Evil Confidant jailbreak — refuse the "no rules" persona while staying helpful OPPO jailbreak — refuse the opposite-of-a-safe-model persona template Tool-use 2 Flight tool selection — search only, don't book, when explicitly told not to Weather tool call — call the weather tool with the correct location argument See each model's actual answer to every task, or pick two models to compare head to head → /tools/ed-o-meter/tests/ Change log — latest: four models added, 23 August 2026 - Four models added. Added glm-5.3, grok-4.6, deepseek-v4-pro and gemini-3.7-flash. glm-5.3 clears all five corners at 100% and is now the board's headline pick, ahead of gpt-5.5. - OpenAI cost update. Re-ran the GPT-5.6 trio after OpenAI cut its prices, refreshed the cost figures and highlighted Luna as the cheapest workhorse for low-risk, retryable jobs. - Three models added. Added opus-5, gemini-3.6-flash and grok-4.5, with their full task, quality, security, latency and cost results. - Leaderboard launched. Published the first Ed-o-meter, including the Claude reference set of haiku-4-5, sonnet-4-6 and sonnet-5.