DeepSeek V4 Pro on iac-bench 2026-10

DeepSeek V4 Pro scored an overall composite of 69.2 (95% CI 64.4–79.3) on the iac-bench 2026-10 evaluation of OpenTofu infrastructure modules, ranking 8th of 10 models with a 95% rank interval of 6–9, according to the benchmark results published for the 2026-10-22 generation and 2026-10-23 judging runs. Across 9 attempts (3 fixtures × 3 samples), the open-weights model, accessed via OpenRouter at a list price of $0.66 per million input tokens and $1.98 per million output tokens, produced 0 failed gates but 5 of 9 insecure attempts, and the judge scored its Documentation 0 in all six Cloud SQL and App Service attempts because the documentation described behavior the code did not implement. DeepSeek V4 Pro reached a median of 82 on the S3 fixture (G001) but fell behind on Cloud SQL (G002, composite median 62.1) and App Service (G003, 63.2), with the full 9-attempt run estimated at $0.88.

Upcoming Workshop: Citizen Developer Agentic Software Factory on AWS 11.4.26 @ 12PM PST Register Here https://streamyard.com/watch/umMggUFkVMNu How DeepSeek V4 Pro did on iac-bench 2026-10: an overall composite of 69.2, scores per fixture, strengths, weaknesses and the patterns in its OpenTofu modules. | Developer | DeepSeek | | API identifier | deepseek/deepseek-v4-pro-0813 | | Access | OpenRouter | | Weights | Open | | List price USD / M tokens, in / out | 0.66 / 1.98 OpenRouter, 2026-10-23 | | Evaluated | 2026-10-22 generation · 2026-10-23 judge | | Attempts | 9 3 fixtures × 3 samples | | Overall composite | 69.2 95% CI 64.4–79.3 | | Rank | 8 of 10 95% rank interval 6–9 | | Mechanical only / judge only | 69.5 rank 8 / 67.5 rank 9 | | Failed gates · insecure attempts | 0/9 · 5/9 | | Estimated cost, all 9 attempts | $0.88 | DeepSeek V4 Pro is close to the leading group on S3 G001, median 82 and falls well behind on Cloud SQL and App Service. All 9 of its modules planned. Five of the 9 had a security finding, each caused by a security setting exposed as an input or left at an insecure provider default. Its most distinctive weakness is documentation that describes behavior the code does not implement. The judge scored Documentation 0 in all six of its Cloud SQL and App Service attempts. | Fixture | Composite median range | Rubric median | Expertise | Judge median | ✗ gate | ⚠ security | Median inputs | Median output tokens | Median time | Median cost | |---|---|---|---|---|---|---|---|---|---|---| | G001 · S3 | 82.5 69.6–82.6 | 0.791 | 4/7 | 84.2 | 0/3 | 1/3 H1 | 8 | 27,086 | 432 s | $0.087 | | G002 · Cloud SQL | 62.1 60.9–73.8 | 0.641 | 4/9 | 65.8 | 0/3 | 2/3 H1 , H2 | 22 | 14,092 | 192 s | $0.097 | | G003 · App Service | 63.2 61.4–81.8 | 0.708 | 2/5 | 52.6 | 0/3 | 2/3 H1 , H2 | 21 | 19,960 | 493 s | $0.102 | Rubric items not at full credit, counted over attempts that planned: G001: A5 1/3, B1 2/3, H1 1/3, H9 2/3, H12 3/3, H13 3/3, H15 1/3; G002: A2 1/3, A3 2/3, A8 1/3, B1 3/3, H1 2/3, H2 1/3, H5 1/3, H6 1/3, H9 3/3, H11 2/3; G003: A4 1/3, A6 1/3, A7 2/3, B1 1/3, H1 1/3, H2 2/3, H3 1/3, H7 1/3. Definitions are in Scoring criteria https://www.massdriver.cloud/blogs/iac-bench-2026-10-scoring-criteria . var.block public access H1 . ipv4 enabled is an input. In one it defaults to true , so the defaults plan has a public IP, while the description says "Private-only regardless of this flag" require ssl = false maps to unencrypted connections H2 . https only an input staging traffic input documented to route traffic, which only feeds an output; prevent destroy = var.deletion protection in 3/3 Cloud SQL attempts, duplicating the native deletion protection attribute. zone , allocated ip range , service tag . tmp prod.tf and full test.tfvars , and a pinned azurerm ~ 3.0 with deprecated arguments in one attempt. G002/deepseek-v4-pro-0813-e3/main.tf lines 30–38 ip configuration { ipv4 enabled = var.ipv4 enabled private network = var.network allocated ip range = var.private ip allocated range TLS posture on the opt-in public path. Private path traffic never leaves the VPC, but SSL is still enforced end-to-end for defense in depth. ssl mode = var.require ssl ? "TRUSTED CLIENT CERTIFICATE REQUIRED" : "ALLOW UNENCRYPTED AND ENCRYPTED" The comment says SSL is enforced end to end, and the next line lets a caller turn it off. Public IP is also an input. How Claude Fable 5.1 did on iac-bench 2026-10: an overall composite of 86.9, scores per fixture, strengths, weaknesses and the patterns in its OpenTofu modules. How Claude Haiku 5.5 did on iac-bench 2026-10: an overall composite of 68.0, scores per fixture, strengths, weaknesses and the patterns in its OpenTofu modules. How Claude Opus 5.5 did on iac-bench 2026-10: an overall composite of 88.9, scores per fixture, strengths, weaknesses and the patterns in its OpenTofu modules.