{"slug": "gemini-3-1-pro-on-iac-bench-2026-10", "title": "Gemini 3.1 Pro on iac-bench 2026-10", "summary": "Gemini 3.1 Pro scored an overall composite of 73.0 (95% CI 69.2–76.2) on Massdriver's iac-bench 2026-10 infrastructure-as-code benchmark, ranking 13th of 16 models, with 5 of its 9 OpenTofu attempts flagged for security findings. All 9 modules planned and none failed a gate, but the model never set `https_only`, basic-auth publishing flags or `ssl_mode`, leaving HTTP and shared-credential publishing open, and it was the only model with no `validation` block in any of its nine modules. The benchmark, run on 2026-10-22 generation and 2026-10-23 judging at a list price of $2.00/$12.00 per million input/output tokens, cost $2.26 for all 9 attempts.", "body_md": "New: LLM Rankings are live!\n\n[See how the top LLMs rank on real infrastructure tasks](https://www.massdriver.cloud/llm-rankings)\n\nHow Gemini 3.1 Pro did on iac-bench 2026-10: an overall composite of 73.0, scores per fixture, strengths, weaknesses and the patterns in its OpenTofu modules.\n\nEvery model evaluated on the 2026-10 fixtures side by side, with a sortable leaderboard, picks by use case and the cost of each: [LLM rankings](https://www.massdriver.cloud/llm-rankings).\n\n| Developer |  | \n| API identifier | `gemini-3.1-pro-preview` | \n| Access | Gemini API | \n| Weights | Closed | \n| List price (USD / M tokens, in / out) | 2.00 / 12.00 (OpenRouter, 2026-10-23) | \n| Evaluated | 2026-10-22 (generation) · 2026-10-23 (judge) | \n| Attempts | 9 (3 fixtures × 3 samples) | \n| **Overall composite** | **73.0** (95% CI 69.2–76.2) | \n| Rank | 13 of 16 (95% rank interval 11–15) | \n| Mechanical only / judge only | 71.9 (rank 13) / 71.9 (rank 14) | \n| Failed gates · insecure attempts | 0/9 · 5/9 | \n| Estimated cost, all 9 attempts | $2.26 | \n\nGemini 3.1 Pro writes the leanest modules among the full-size models: few inputs, short files, low token use. All 9 of its modules planned. Five of the 9 had security findings, all from settings left unset rather than settings exposed as inputs:\n\n`https_only` and the basic-auth publishing flags are never set, so\nthe provider defaults leave HTTP and shared-credential publishing open.`ssl_mode` is never set, so unencrypted connections are accepted.\nIt is also the only model with no `validation` block in any of its nine modules.\n\n| Fixture | Composite median (range) | Rubric median | Expertise | Judge median | ✗ gate | ⚠ security | Median inputs | Median output tokens | Median time | Median cost | \n|---|---|---|---|---|---|---|---|---|---|---|\n| G001 · S3 | 74.2 (73.4–80.1) | 0.733 | 3/7 | 78.9 | 0/3 | 0/3 | 5 | 4,677 | 140 s | $0.171 | \n| G002 · Cloud SQL | 79.3 (73.5–82.0) | 0.815 | 3/9 | 76.3 | 0/3 | 2/3 ( `H2` ) | 18 | 6,768 | 128 s | $0.271 | \n| G003 · App Service | 65.4 (59.9–69.4) | 0.722 | 0/5 | 60.5 | 0/3 | 3/3 ( `H1` ,`H2` ) | 11 | 7,736 | 231 s | $0.238 | \n\nRubric items not at full credit, counted over attempts that planned: G001: `H6` 2/3, `H7` 2/3, `H9` 3/3, `H12` 3/3, `H13` 3/3, `H15` 3/3; G002: `A4` 1/3, `B1` 2/3, `H2` 2/3, `H4` 1/3, `H5` 1/3, `H8` 1/3, `H9` 3/3, `H11` 1/3; G003: `H1` 3/3, `H2` 3/3, `H4` 1/3, `H7` 1/3, `H8` 3/3. Definitions are in [Scoring criteria](https://www.massdriver.cloud/blogs/iac-bench-2026-10-scoring-criteria).\n\n`ipv4_enabled = false` is a\nliteral, and the IAM authentication flag is merged after consumer flags so it cannot be\noverridden.`H1` and `H2` (security findings, 3/3).`ip_configuration` omits `H8`, 0/3);` H9`, `H13`).` sku_name`, `disk_type`, `availability_type`). `G003/gemini-3.1-pro-preview-e1/main.tf` lines 45–58\n\n```\nresource \"azurerm_linux_web_app\" \"this\" {\n  name                = var.app_name\n  location            = azurerm_resource_group.this.location\n  resource_group_name = azurerm_resource_group.this.name\n  service_plan_id     = azurerm_service_plan.this.id\n\n  identity {\n    type = \"SystemAssigned\"\n  }\n\n  site_config {\n    application_stack {\n      docker_image_name   = var.container_image\n      docker_registry_url = var.container_registry_url\n```\n\nAn otherwise complete resource that never sets `https_only` or the basic-auth publishing flags. The same omission appears on all six App Service web apps and slots this model wrote.\n\nInsecure configuration is scored by the rubric, not the judge, so the security findings move this model's judge grade very little. On Cloud SQL, the attempt with a security finding received the highest judge grade of the three, and the secure attempt the lowest. The composite takes both into account. Read either component alone and you get a misleading picture.\n\nHow Claude Fable 5.1 did on iac-bench 2026-10: an overall composite of 86.9, scores per fixture, strengths, weaknesses and the patterns in its OpenTofu modules.\n\nHow Claude Fable 5.2 did on iac-bench 2026-10: an overall composite of 93.9, scores per fixture, strengths, weaknesses and the patterns in its OpenTofu modules.\n\nHow Claude Haiku 5.5 did on iac-bench 2026-10: an overall composite of 68.0, scores per fixture, strengths, weaknesses and the patterns in its OpenTofu modules.", "url": "https://wpnews.pro/news/gemini-3-1-pro-on-iac-bench-2026-10", "canonical_source": "https://www.massdriver.cloud/blogs/iac-bench-2026-10-gemini-3-1-pro-preview", "published_at": "2026-11-04 00:00:00+00:00", "updated_at": "2026-11-21 01:13:25.584939+00:00", "lang": "en", "topics": ["ai-research", "large-language-models", "ai-tools"], "entities": ["Gemini 3.1 Pro", "Google", "Massdriver", "iac-bench", "OpenTofu", "Claude Fable 5.1", "Claude Fable 5.2", "OpenRouter"], "also_reported_by": [], "takeaway": "Gemini 3.1 Pro's 73.0 composite came with 5 of 9 insecure attempts, all from unset settings — `https_only`, basic-auth publishing flags and `ssl_mode` — rather than exposed inputs, and the Cloud SQL attempt with a security finding drew the highest judge grade of its three.", "recommendation": {"advice": "Score insecure IaC configuration with rubric rules, not an LLM judge: on Cloud SQL the attempt with a security finding got the highest judge grade, so combine both components before ranking models.", "playbook": "evals-testing", "playbook_url": "https://wpnews.pro/playbooks/evals-testing"}, "alternates": {"html": "https://wpnews.pro/news/gemini-3-1-pro-on-iac-bench-2026-10", "markdown": "https://wpnews.pro/news/gemini-3-1-pro-on-iac-bench-2026-10.md", "text": "https://wpnews.pro/news/gemini-3-1-pro-on-iac-bench-2026-10.txt", "jsonld": "https://wpnews.pro/news/gemini-3-1-pro-on-iac-bench-2026-10.jsonld"}}