{"slug": "deepseek-v4-pro-on-iac-bench-2026-10", "title": "DeepSeek V4 Pro on iac-bench 2026-10", "summary": "DeepSeek V4 Pro scored an overall composite of 69.2 (95% CI 64.4–79.3) on the iac-bench 2026-10 evaluation of OpenTofu infrastructure modules, ranking 8th of 10 models with a 95% rank interval of 6–9, according to the benchmark results published for the 2026-10-22 generation and 2026-10-23 judging runs. Across 9 attempts (3 fixtures × 3 samples), the open-weights model, accessed via OpenRouter at a list price of $0.66 per million input tokens and $1.98 per million output tokens, produced 0 failed gates but 5 of 9 insecure attempts, and the judge scored its Documentation 0 in all six Cloud SQL and App Service attempts because the documentation described behavior the code did not implement. DeepSeek V4 Pro reached a median of 82 on the S3 fixture (G001) but fell behind on Cloud SQL (G002, composite median 62.1) and App Service (G003, 63.2), with the full 9-attempt run estimated at $0.88.", "body_md": "Upcoming Workshop: Citizen Developer Agentic Software Factory on AWS 11.4.26 @ 12PM PST\n\n[Register Here!](https://streamyard.com/watch/umMggUFkVMNu)\n\nHow DeepSeek V4 Pro did on iac-bench 2026-10: an overall composite of 69.2, scores per fixture, strengths, weaknesses and the patterns in its OpenTofu modules.\n\n| Developer | DeepSeek | \n| API identifier | `deepseek/deepseek-v4-pro-0813` | \n| Access | OpenRouter | \n| Weights | Open | \n| List price (USD / M tokens, in / out) | 0.66 / 1.98 (OpenRouter, 2026-10-23) | \n| Evaluated | 2026-10-22 (generation) · 2026-10-23 (judge) | \n| Attempts | 9 (3 fixtures × 3 samples) | \n| **Overall composite** | **69.2** (95% CI 64.4–79.3) | \n| Rank | 8 of 10 (95% rank interval 6–9) | \n| Mechanical only / judge only | 69.5 (rank 8) / 67.5 (rank 9) | \n| Failed gates · insecure attempts | 0/9 · 5/9 | \n| Estimated cost, all 9 attempts | $0.88 | \n\nDeepSeek V4 Pro is close to the leading group on S3 (G001, median 82) and falls well behind on Cloud SQL and App Service. All 9 of its modules planned. Five of the 9 had a security finding, each caused by a security setting exposed as an input or left at an insecure provider default.\n\nIts most distinctive weakness is documentation that describes behavior the code does not\nimplement. The judge scored *Documentation* 0 in all six of its Cloud SQL and App Service\nattempts.\n\n| Fixture | Composite median (range) | Rubric median | Expertise | Judge median | ✗ gate | ⚠ security | Median inputs | Median output tokens | Median time | Median cost | \n|---|---|---|---|---|---|---|---|---|---|---|\n| G001 · S3 | 82.5 (69.6–82.6) | 0.791 | 4/7 | 84.2 | 0/3 | 1/3 ( `H1` ) | 8 | 27,086 | 432 s | $0.087 | \n| G002 · Cloud SQL | 62.1 (60.9–73.8) | 0.641 | 4/9 | 65.8 | 0/3 | 2/3 ( `H1` ,`H2` ) | 22 | 14,092 | 192 s | $0.097 | \n| G003 · App Service | 63.2 (61.4–81.8) | 0.708 | 2/5 | 52.6 | 0/3 | 2/3 ( `H1` ,`H2` ) | 21 | 19,960 | 493 s | $0.102 | \n\nRubric items not at full credit, counted over attempts that planned: G001: `A5` 1/3, `B1` 2/3, `H1` 1/3, `H9` 2/3, `H12` 3/3, `H13` 3/3, `H15` 1/3; G002: `A2` 1/3, `A3` 2/3, `A8` 1/3, `B1` 3/3, `H1` 2/3, `H2` 1/3, `H5` 1/3, `H6` 1/3, `H9` 3/3, `H11` 2/3; G003: `A4` 1/3, `A6` 1/3, `A7` 2/3, `B1` 1/3, `H1` 1/3, `H2` 2/3, `H3` 1/3, `H7` 1/3. Definitions are in [Scoring criteria](https://www.massdriver.cloud/blogs/iac-bench-2026-10-scoring-criteria).\n\n`var.block_public_access` (`H1`).` ipv4_enabled` is an input. In one it defaults to `true`, so\nthe defaults plan has a public IP, while the description says \"Private-only\nregardless of this flag\" (`require_ssl = false` maps to\nunencrypted connections (`H2`).` https_only` an input (`staging_traffic` input documented to route traffic, which only feeds an output;`prevent_destroy = var.deletion_protection` in 3/3 Cloud SQL\nattempts, duplicating the native `deletion_protection` attribute. `zone`, `allocated_ip_range`,\n`service_tag`).`_tmp_prod.tf` and `_full_test.tfvars`, and a pinned\n`azurerm ~> 3.0` with deprecated arguments in one attempt.\n`G002/deepseek-v4-pro-0813-e3/main.tf` lines 30–38\n\n```\n    ip_configuration {\n      ipv4_enabled    = var.ipv4_enabled\n      private_network = var.network\n\n      allocated_ip_range = var.private_ip_allocated_range\n\n      # TLS posture on the (opt-in) public path. Private path traffic never\n      # leaves the VPC, but SSL is still enforced end-to-end for defense in depth.\n      ssl_mode = var.require_ssl ? \"TRUSTED_CLIENT_CERTIFICATE_REQUIRED\" : \"ALLOW_UNENCRYPTED_AND_ENCRYPTED\"\n```\n\nThe comment says SSL is enforced end to end, and the next line lets a caller turn it off. Public IP is also an input.\n\nHow Claude Fable 5.1 did on iac-bench 2026-10: an overall composite of 86.9, scores per fixture, strengths, weaknesses and the patterns in its OpenTofu modules.\n\nHow Claude Haiku 5.5 did on iac-bench 2026-10: an overall composite of 68.0, scores per fixture, strengths, weaknesses and the patterns in its OpenTofu modules.\n\nHow Claude Opus 5.5 did on iac-bench 2026-10: an overall composite of 88.9, scores per fixture, strengths, weaknesses and the patterns in its OpenTofu modules.", "url": "https://wpnews.pro/news/deepseek-v4-pro-on-iac-bench-2026-10", "canonical_source": "https://www.massdriver.cloud/blogs/iac-bench-2026-10-deepseek-v4-pro-0813", "published_at": "2026-11-04 00:00:00+00:00", "updated_at": "2026-11-04 20:21:32.041308+00:00", "lang": "en", "topics": ["large-language-models", "ai-research", "ai-tools", "mlops"], "entities": ["DeepSeek", "DeepSeek V4 Pro", "OpenRouter", "iac-bench", "OpenTofu", "Claude Fable 5.1", "Claude Haiku 5.5", "Claude Opus 5.5"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/deepseek-v4-pro-on-iac-bench-2026-10", "markdown": "https://wpnews.pro/news/deepseek-v4-pro-on-iac-bench-2026-10.md", "text": "https://wpnews.pro/news/deepseek-v4-pro-on-iac-bench-2026-10.txt", "jsonld": "https://wpnews.pro/news/deepseek-v4-pro-on-iac-bench-2026-10.jsonld"}}