New: LLM Rankings are live!
See how the top LLMs rank on real infrastructure tasks
How GLM-5.3 did on iac-bench 2026-10: an overall composite of 82.5, scores per fixture, strengths, weaknesses and the patterns in its OpenTofu modules.
Every model evaluated on the 2026-10 fixtures side by side, with a sortable leaderboard, picks by use case and the cost of each: LLM rankings.
| Developer | Z.ai |
| API identifier | z-ai/glm-5.3 |
| Access | OpenRouter, pinned to Z.ai's endpoint (FP8) |
| Weights | Open |
| List price (USD / M tokens, in / out) | 1.40 / 4.40 (OpenRouter, 2026-11-10) |
| Evaluated | 2026-11-10 (generation and judge) |
| Attempts | 9 (3 fixtures × 3 samples) |
| Overall composite | 82.5 (95% CI 60.5–88.2) |
| Rank | 10 of 16 (95% rank interval 3–15) |
| Mechanical only / judge only | 79.1 (rank 11) / 86.8 (rank 10) |
| Failed gates · insecure attempts | 1/9 · 1/9 |
| Estimated cost, all 9 attempts | $8.46 |
GLM-5.3 places tenth of 16 by composite (82.5), between Kimi K3 (83.4) and Qwen3.8 2.4T-A95B (81.6), and third of the five open-weight models. Its 95% rank interval of 3–15 is the second widest in the field, after Claude Haiku 5.5 (2–15). In the bootstrap it finishes ahead of Kimi K3 in 52% of resamples, ahead of Qwen3.8 in 57% and ahead of Kimi K3.1 Pro in 16%. At $8.46 for all 9 attempts it is the most expensive open-weight model, about twice Qwen3.8 ($4.40) and Kimi K3 ($3.96), and third overall after Claude Fable 5.2 and Claude Fable 5.1.
Three attempts reached the published 30-minute wall-clock limit: App Service (G003) e1 and
e2, and Cloud SQL (G002) e3. They are scored as written, as is every attempt that
reaches the limit. G003 e2 stopped before it had a module that plans
and scores 0 mechanical (composite 22.1), which accounts for the wide interval. The other 8
attempts planned, with one security finding (G003 e1, H2). S3 (G001) and Cloud SQL medians
are both 83.4, ninth and eleventh in the field. App Service has the lowest median (80.7,
twelfth) and also the best attempt.
| Fixture | Composite median (range) | Rubric median | Expertise | Judge median | ✗ gate | ⚠ security | Median inputs | Median output tokens | Median time | Median cost |
|---|---|---|---|---|---|---|---|---|---|---|
| G001 · S3 | 83.4 (81.8–84.7) | 0.802 | 6/7 | 92.1 | 0/3 | 0/3 | 16 | 71,659 | 1127 s | $0.556 |
| G002 · Cloud SQL | 83.4 (76.2–85.2) | 0.809 | 5/9 | 84.2 | 0/3 | 0/3 | 29 | 108,600 | 1639 s | $1.104 |
| G003 · App Service | 80.7 (22.1–95.9) | 0.778 | 4.5/5 | 84.2 | 1/3 | 1/3 ( H2 ) |
19 | 124,719 | 1945 s | $1.362 |
Rubric items not at full credit, counted over attempts that planned: G001: A5 3/3, B1 2/3, H12 3/3, H13 3/3, H15 2/3; G002: A8 2/3, B1 2/3, H3 1/3, H4 1/3, H7 1/3, H9 1/3, H10 2/3, H11 3/3; G003: A7 1/2, H2 1/2, 1 attempt failed the gate. Definitions are in Scoring criteria.
WEBSITES_PORT, metric alerts and
managed-identity image pulls (E1–E4). Kimi K3.1 Pro's best App Service attempt is 87.8.ipv4_enabled = false and ssl_mode = "ENCRYPTED_ONLY",
where Kimi K3's validation admits unencrypted connections in 3/3.<name>-<account-id>-<region> name.cloudsql.iam_authentication and a validation rejects that key in database_flags. A5 lost 3/3); 29 and
32 on Cloud SQL against 25 (A8 partial, 2/3); 39 in G003 e1 against 20 (A7 at zero).
H2): neither publishing
flag is set, so both stay at the provider default. Kimi K3.1 Pro turns them off in 3/3.H11); H10): raw builtin_users passwords in e2 (H7), a
generated postgres password in e3;azurerm_service_plan has no
zone-redundancy switch.B1), such as
>= 5.0 on S3 (2/3) and >= 5.20.0, < 7.0.0 on Cloud SQL.IAM_USER where the provider
expects CLOUD_IAM_USER; G002 e3 keeps the .gserviceaccount.com suffix in user names.
The judge caught both (probe/main.tf creating hardcoded
rg-probe resources (G003 e1, scratch_allocated_ip_range.tf. environment; G003 e3 maps a tier input to SKU, zone capability and
instance limit (excerpt below).
G003/glm-5.3-e3/locals.tf lines 33–49
tiers = {
small = { sku = "S1", zone_capable = false, instance_limit = 10 } # 1 vCPU / 1.75 GB
medium = { sku = "P1v3", zone_capable = true, instance_limit = 30 } # 2 vCPU / 8 GB
large = { sku = "P2v3", zone_capable = true, instance_limit = 30 } # 4 vCPU / 16 GB
xlarge = { sku = "P3v3", zone_capable = true, instance_limit = 30 } # 8 vCPU / 32 GB
}
tier = local.tiers[var.tier]
zone_spreading = coalesce(var.zone_redundant, local.tier.zone_capable)
effective_min_instances = local.zone_spreading ? max(var.min_instances, 3) : var.min_instances
From the full-credit App Service attempt: a four-value tier input maps to SKU, zone capability and instance limit, and the instance floor rises to 3 when zones are in use. The judge marked the zone-spreading comment inaccurate, since zone_balancing_enabled is itself the plan's zone-redundancy setting.
How Claude Fable 5.1 did on iac-bench 2026-10: an overall composite of 86.9, scores per fixture, strengths, weaknesses and the patterns in its OpenTofu modules.
How Claude Fable 5.2 did on iac-bench 2026-10: an overall composite of 93.9, scores per fixture, strengths, weaknesses and the patterns in its OpenTofu modules.
How Claude Haiku 5.5 did on iac-bench 2026-10: an overall composite of 68.0, scores per fixture, strengths, weaknesses and the patterns in its OpenTofu modules.