New: LLM Rankings are live!
See how the top LLMs rank on real infrastructure tasks
How Gemini 3.1 Pro did on iac-bench 2026-10: an overall composite of 73.0, scores per fixture, strengths, weaknesses and the patterns in its OpenTofu modules.
Every model evaluated on the 2026-10 fixtures side by side, with a sortable leaderboard, picks by use case and the cost of each: LLM rankings.
| Developer | |
| API identifier | gemini-3.1-pro-preview |
| Access | Gemini API |
| Weights | Closed |
| List price (USD / M tokens, in / out) | 2.00 / 12.00 (OpenRouter, 2026-10-23) |
| Evaluated | 2026-10-22 (generation) · 2026-10-23 (judge) |
| Attempts | 9 (3 fixtures × 3 samples) |
| Overall composite | 73.0 (95% CI 69.2–76.2) |
| Rank | 13 of 16 (95% rank interval 11–15) |
| Mechanical only / judge only | 71.9 (rank 13) / 71.9 (rank 14) |
| Failed gates · insecure attempts | 0/9 · 5/9 |
| Estimated cost, all 9 attempts | $2.26 |
Gemini 3.1 Pro writes the leanest modules among the full-size models: few inputs, short files, low token use. All 9 of its modules planned. Five of the 9 had security findings, all from settings left unset rather than settings exposed as inputs:
https_only and the basic-auth publishing flags are never set, so
the provider defaults leave HTTP and shared-credential publishing open.ssl_mode is never set, so unencrypted connections are accepted.
It is also the only model with no validation block in any of its nine modules.
| Fixture | Composite median (range) | Rubric median | Expertise | Judge median | ✗ gate | ⚠ security | Median inputs | Median output tokens | Median time | Median cost |
|---|---|---|---|---|---|---|---|---|---|---|
| G001 · S3 | 74.2 (73.4–80.1) | 0.733 | 3/7 | 78.9 | 0/3 | 0/3 | 5 | 4,677 | 140 s | $0.171 |
| G002 · Cloud SQL | 79.3 (73.5–82.0) | 0.815 | 3/9 | 76.3 | 0/3 | 2/3 ( H2 ) |
18 | 6,768 | 128 s | $0.271 |
| G003 · App Service | 65.4 (59.9–69.4) | 0.722 | 0/5 | 60.5 | 0/3 | 3/3 ( H1 ,H2 ) |
11 | 7,736 | 231 s | $0.238 |
Rubric items not at full credit, counted over attempts that planned: G001: H6 2/3, H7 2/3, H9 3/3, H12 3/3, H13 3/3, H15 3/3; G002: A4 1/3, B1 2/3, H2 2/3, H4 1/3, H5 1/3, H8 1/3, H9 3/3, H11 1/3; G003: H1 3/3, H2 3/3, H4 1/3, H7 1/3, H8 3/3. Definitions are in Scoring criteria.
ipv4_enabled = false is a
literal, and the IAM authentication flag is merged after consumer flags so it cannot be
overridden.H1 and H2 (security findings, 3/3).ip_configuration omits H8, 0/3); H9, H13). sku_name, disk_type, availability_type). G003/gemini-3.1-pro-preview-e1/main.tf lines 45–58
resource "azurerm_linux_web_app" "this" {
name = var.app_name
location = azurerm_resource_group.this.location
resource_group_name = azurerm_resource_group.this.name
service_plan_id = azurerm_service_plan.this.id
identity {
type = "SystemAssigned"
}
site_config {
application_stack {
docker_image_name = var.container_image
docker_registry_url = var.container_registry_url
An otherwise complete resource that never sets https_only or the basic-auth publishing flags. The same omission appears on all six App Service web apps and slots this model wrote.
Insecure configuration is scored by the rubric, not the judge, so the security findings move this model's judge grade very little. On Cloud SQL, the attempt with a security finding received the highest judge grade of the three, and the secure attempt the lowest. The composite takes both into account. Read either component alone and you get a misleading picture.
How Claude Fable 5.1 did on iac-bench 2026-10: an overall composite of 86.9, scores per fixture, strengths, weaknesses and the patterns in its OpenTofu modules.
How Claude Fable 5.2 did on iac-bench 2026-10: an overall composite of 93.9, scores per fixture, strengths, weaknesses and the patterns in its OpenTofu modules.
How Claude Haiku 5.5 did on iac-bench 2026-10: an overall composite of 68.0, scores per fixture, strengths, weaknesses and the patterns in its OpenTofu modules.