# How Well Do LLMs Write Infrastructure Modules? iac-bench 2026-10 Results

> Source: <https://www.massdriver.cloud/blogs/iac-bench-2026-10-results>
> Published: 2026-11-04 00:00:00+00:00

Upcoming Workshop: Citizen Developer Agentic Software Factory on AWS 11.4.26 @ 12PM PST

[Register Here!](https://streamyard.com/watch/umMggUFkVMNu)

We asked ten language models to write reusable OpenTofu modules for AWS, GCP and Azure, then scored 90 modules against real plans and a calibrated judge. Here is how they did and the patterns behind the scores.

**iac-bench release 2026-10 · run 2026-10-22 · 10 models · 3 clouds · 90 attempts**

We asked ten language models to write reusable OpenTofu modules for three common cloud services intended for a specific audience:

Each request was a few sentences describing the high level use case of the module and the intended audience. We never listed the specific security, durability or hygiene requirements a senior reviewer would expect.

We scored the 90 resulting modules on two components:

`tofu plan`
output.
This article reports the results and the patterns behind them:

`H13`).` H11`).` B1`).
| Rank | Model | Developer | Weights | Overall (95% CI) | Rank interval | G001 · S3 | G002 · Cloud SQL | G003 · App Service | ✗ gate | ⚠ security | Cost, 9 attempts | 
|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | [Claude Opus 5.5](https://www.massdriver.cloud/blogs/iac-bench-2026-10-claude-opus-5-5) | Anthropic | Closed | **88.9** (87.5–91.5) | 1–3 | 87.9 | 88.6 | 90.2 | 0/9 | 0/9 | $3.58 | 
| 2 | [Claude Fable 5.1](https://www.massdriver.cloud/blogs/iac-bench-2026-10-claude-fable-5-1) | Anthropic | Closed | **86.9** (83.7–90.2) | 1–5 | 85.6 | 85.6 | 89.5 | 0/9 | 0/9 | $13.97 | 
| 3 | [GPT-6 Astra](https://www.massdriver.cloud/blogs/iac-bench-2026-10-gpt-6-astra) | OpenAI | Closed | **86.7** (86.0–89.8) | 1–5 | 72.8 | 91.3 | 95.9 | 0/9 | 0/9 | $7.26 | 
| 4 | [Claude Sonnet 5.5](https://www.massdriver.cloud/blogs/iac-bench-2026-10-claude-sonnet-5-5) | Anthropic | Closed | **86.1** (84.8–88.2) | 2–5 | 84.8 | 90.6 | 82.8 | 0/9 | 2/9 | $1.52 | 
| 5 | [Kimi K3](https://www.massdriver.cloud/blogs/iac-bench-2026-10-kimi-k3) | Moonshot AI | Open | **82.4** (57.3–84.7) | 5–9 | 81.1 | 79.2 | 86.9 | 1/9 | 4/9 | $3.96 | 
| 6 | [GPT-6 Luna](https://www.massdriver.cloud/blogs/iac-bench-2026-10-gpt-6-luna) | OpenAI | Closed | **81.5** (79.8–82.1) | 5–7 | 74.3 | 84.5 | 85.7 | 0/9 | 0/9 | $0.05 | 
| 7 | [Gemini 3.1 Pro](https://www.massdriver.cloud/blogs/iac-bench-2026-10-gemini-3-1-pro-preview) |  | Closed | **73.0** (69.2–76.2) | 6–9 | 74.2 | 79.3 | 65.4 | 0/9 | 5/9 | $2.26 | 
| 8 | [DeepSeek V4 Pro](https://www.massdriver.cloud/blogs/iac-bench-2026-10-deepseek-v4-pro-0813) | DeepSeek | Open | **69.2** (64.4–79.3) | 6–9 | 82.5 | 62.1 | 63.2 | 0/9 | 5/9 | $0.88 | 
| 9 | [Claude Haiku 5.5](https://www.massdriver.cloud/blogs/iac-bench-2026-10-claude-haiku-5-5) | Anthropic | Closed | **68.0** (66.1–90.6) | 1–9 | 87.8 | 26.8 | 89.4 | 2/9 | 1/9 | $0.11 | 
| 10 | [Gemini 3.6 Flash](https://www.massdriver.cloud/blogs/iac-bench-2026-10-gemini-3-6-flash) |  | Closed | **54.3** (40.0–61.5) | 10–10 | 62.8 | 49.2 | 51.0 | 1/9 | 8/9 | $3.14 | 

Composite = 0.70 × mechanical + 0.30 × judge. Fixture columns are medians of three attempts. ✗ = attempts whose module did not plan; ⚠ = attempts failing at least one mandatory security item. Costs are estimates at list prices on 2026-10-23 and exclude judging.

*Figure 1. Overall composite (mean of the three per-fixture medians). Whiskers show the
bootstrap 95% CI; the right-hand labels give the 95% rank interval.*

**How to read the ranking.** Three samples per model per fixture is enough to separate
groups but not neighbors:

Claude Haiku 5.5's interval is the widest in the release. It is driven by one fixture where two of three modules failed to plan. Pairwise win probabilities are in the data release.

*Figure 2. Median composite of three attempts, per model and fixture.*

| Fixture | Field median | Best median | Most-missed rubric item (attempts that planned) | Most-missed mandatory item | 
|---|---|---|---|---|
| G001 · S3 | 81.8 | 87.9 (Claude Opus 5.5) | `H13` log delivery permitted: 30/30 missed | `H1` public access block: 6/30 | 
| G002 · Cloud SQL | 82.2 | 91.3 (GPT-6 Astra) | `H11` databases dropped on removal: 24/28 | `H2` encrypted connections: 9/28 | 
| G003 · App Service | 86.2 | 95.9 (GPT-6 Astra) | `A7` interface budget: 19/28 | `H2` basic-auth publishing: 10/28 | 

Fixture scores are not interchangeable. **G001 (S3)** punishes omission: the brief's
audience is developers who "don't know S3 well," so every hygiene decision falls to the
module. The models that lose the most here:

**G002 (Cloud SQL)** rewards locking down a specialist's interface without taking their
controls away. **G003 (App Service)** depends heavily on two settings the Azure provider
leaves insecure by default.

Some criteria were missed by nearly every model, regardless of vendor, size or overall rank. These are the clearest signal in the release, because they do not depend on which model you pick.

*Figure 3. Share of attempts earning full credit on the rubric items most often missed,
by model. Item definitions are in [Scoring criteria](https://www.massdriver.cloud/blogs/iac-bench-2026-10-scoring-criteria).*

These criteria were missed by most of the field.

**Access logs that cannot be delivered (G001 `H13`, 30/30 missed).**

`H9`).
**Databases dropped when removed from the input (G002 `H11`, 24/28 missed).**

`google_sql_database` defaults to deleting the database, and its data, when the resource
is removed.`databases` list therefore drops that database's
data on the next apply.`deletion_policy = "ABANDON"` or tying it to deletion
protection.
**Backups that cannot be turned off (G002 `H4`/` H5`).** This one splits by rank:

`E1`, 0/28). By default Cloud SQL deletes them with the instance.
**Provider constraints that admit the next major version (`B1`).**

| Cloud | Attempts with such a constraint | 
|---|---|
| AWS | 19/30 | 
| GCP | 22/28 | 
| Azure | 2/28 | 

`>= 5.0`, or a window such as
`>= 5.0, < 7.0` that spans two majors.`~> 4.0`.
The habit looks provider-specific rather than model-specific: the same models that bound
`azurerm` correctly left `aws` and `google` open.

**Expertise that only a few models showed:**

`roles/cloudsql.instanceUser`, G002 `E6`): 13/28 granted it for
the IAM database users they created. Without it, those users cannot log in.`E8`): 12/28.` E5`): 6/28.

*Figure 4. Attempts with at least one mandatory security failure, and attempts that did
not plan, out of 9 per model.*

Each fixture has two **mandatory** items: its most severe security failures. Twenty-five of
90 attempts failed at least one.

| Pattern | Attempts | What it looks like | Models | 
|---|---|---|---|
| **Secure default, open override** | 15 | The secure value is the default, but the setting is an input any consumer can change. For example, `ssl_mode` defaults to`ENCRYPTED_ONLY` while the validation also accepts`ALLOW_UNENCRYPTED_AND_ENCRYPTED` . | Gemini 3.6 Flash, Kimi K3, DeepSeek V4 Pro | 
| **Insecure provider default left in place** | 10 | The module never sets the attribute, and the provider's default is insecure. On Azure, basic-auth publishing for FTP and Web Deploy is on unless disabled. | Gemini 3.1 Pro, Claude Sonnet 5.5, Claude Haiku 5.5, plus some attempts by DeepSeek V4 Pro and Gemini 3.6 Flash | 

**Secure default, open override.** The doctrine treats this as a defect even when the
default is safe. A module's security posture has to hold for every consumer, and an input
that can silently weaken it will eventually be used to do so. Models in this pattern often
document the risk in the same breath: one description reads "Do not weaken this without a
security review" above a validation list that allows exactly that.

**Insecure provider default left in place.** This failure cannot be seen by reading the
source, because nothing is written down. The most-missed mandatory item in the release is
G003 `H2` (basic-auth publishing, 10/28). Several of these modules set HTTPS-only, TLS 1.2
and FTPS-off correctly in the same block, and still miss the two publishing flags the Azure
provider enables by default.

**Failed gates.** Four attempts produced modules that did not plan:

*Figure 5. Mechanical-only and judge-only scores per model (mean of per-fixture medians).*

**The ranking holds without the judge.** Ranked by the mechanical score alone, the ten models
fall in exactly the composite order. The judge's 30% shifts scores but never reorders the
leaderboard.

| Model | Mechanical only | Judge only | Composite rank | 
|---|---|---|---|
| Claude Opus 5.5 | 86.4 (#1) | 94.7 (#1) | 1 | 
| Claude Fable 5.1 | 85.0 (#2) | 91.2 (#5) | 2 | 
| GPT-6 Astra | 83.2 (#3) | 94.7 (#2) | 3 | 
| Claude Sonnet 5.5 | 82.5 (#4) | 94.7 (#3) | 4 | 
| Kimi K3 | 80.6 (#5) | 86.0 (#6) | 5 | 
| GPT-6 Luna | 79.7 (#6) | 84.2 (#7) | 6 | 
| Gemini 3.1 Pro | 71.9 (#7) | 71.9 (#8) | 7 | 
| DeepSeek V4 Pro | 69.5 (#8) | 67.5 (#9) | 8 | 
| Claude Haiku 5.5 | 59.2 (#9) | 91.2 (#4) | 9 | 
| Gemini 3.6 Flash | 50.9 (#10) | 62.3 (#10) | 10 | 

The two components disagree on individual modules, and the disagreements are systematic.

**What the judge misses.** It reads source, so it cannot see:

**What the rubric misses.** It checks planned outcomes, so it cannot see:

The judge scored *Documentation* 0 in 26 of 90 attempts. It found, for example, a databases
input that fails as soon as one database is declared, which the rubric credited.

**Disagreement about restraint.** The largest single gap is GPT-6 Astra on S3:

A reader who rejects the rubric's view can read the judge-only column, and the reverse.

**Self-preference.** The judge is Claude Opus 5.5. Claude models take three of the top four
judge-only ranks, alongside GPT-6 Astra. But the composite order, and Claude Opus 5.5's
first place, are identical under the mechanical score alone, where no model judges any
other.

*Figure 6. Overall composite against the estimated cost of one attempt at each fixture, at
list prices on 2026-10-23. The dashed line is the Pareto frontier: models no other model
beats on both cost and score.*

Generation cost varies by a factor of about 280 across the field, and it is only loosely related to score.

| Model | Overall | Estimated cost, 9 attempts | 
|---|---|---|
| GPT-6 Luna | 81.5 | $0.05 | 
| Claude Haiku 5.5 | 68.0 | $0.11 | 
| DeepSeek V4 Pro | 69.2 | $0.88 | 
| Claude Sonnet 5.5 | 86.1 | $1.52 | 
| Gemini 3.1 Pro | 73.0 | $2.26 | 
| Gemini 3.6 Flash | 54.3 | $3.14 | 
| Claude Opus 5.5 | 88.9 | $3.58 | 
| Kimi K3 | 82.4 | $3.96 | 
| GPT-6 Astra | 86.7 | $7.26 | 
| Claude Fable 5.1 | 86.9 | $13.97 | 

The Pareto frontier, the models no other model beats on both cost and score, runs through:

Claude Fable 5.1 costs about four times as much as Claude Opus 5.5 for a slightly lower score. Claude Haiku 5.5 is the second-cheapest model and would sit near the frontier, but for its two failed Cloud SQL attempts.

Costs are estimates: token counts multiplied by list prices. They exclude judging and any provider-specific caching discounts beyond a flat cache-read and cache-write multiplier.

*Documentation* is the judged section most often scored 0: in 26 of 90 attempts. Nearly every
module has documentation. The zeros come from **documentation that is wrong**:

Under the doctrine, any wrong statement scores 0, however small.

| Pattern | Examples | 
|---|---|
| Limits misstated | 1–35 days of WAL retention described for an edition that allows 1–7 | 
| Inverted meaning | A deletion-protection input described as "whether or not to allow Terraform to destroy the instance" | 
| Features that do not exist | "Least-requests load balancing" that is never configured; a staging-traffic input that feeds only an output | 

**Longer documentation produces more of these errors.** The models that write long READMEs
(Claude Fable 5.1, GPT-6 Astra, Kimi K3) give the judge more claims to check:

**The same model, prompt and fixture can produce very different modules.** Nine of the 30
model-fixture cells spread by more than 10 points across their three attempts. Four spread
by more than 20:

| Model · fixture | Range | Cause | 
|---|---|---|
| Claude Haiku 5.5 · Cloud SQL | 25.3–91.0 | Two attempts did not plan | 
| Kimi K3 · App Service | 25.3–89.4 | One attempt did not plan | 
| Gemini 3.6 Flash · App Service | 15.0–62.4 | One attempt did not plan | 
| DeepSeek V4 Pro · App Service | 57.8–81.8 | Basic-auth flags set in one attempt, absent in two | 

Two consequences:

Every model in the top group has all three attempts within 11 points on every fixture. Claude Opus 5.5's widest spread is 8.5 points.

The main limitations of this release:

Full discussion: [Methodology §7–8](https://www.massdriver.cloud/blogs/iac-bench-2026-10-methodology#7-limitations).

The [data release](https://github.com/massdriver-cloud/iac-bench/tree/2026-10/results/2026-10) (`results/2026-10/` in the repository) contains:

It is licensed CC BY 4.0. The evaluation code, fixtures, rubrics, reference modules and doctrine are in the repository.

```
@misc{iacbench2026,
  title  = {iac-bench: Evaluating LLM-authored OpenTofu modules},
  author = {{Massdriver}},
  year   = {2026},
  month  = {10},
  note   = {Release 2026-10},
  url    = {https://github.com/massdriver-cloud/iac-bench}
}
```

How Claude Fable 5.1 did on iac-bench 2026-10: an overall composite of 86.9, scores per fixture, strengths, weaknesses and the patterns in its OpenTofu modules.

How Claude Haiku 5.5 did on iac-bench 2026-10: an overall composite of 68.0, scores per fixture, strengths, weaknesses and the patterns in its OpenTofu modules.

How Claude Opus 5.5 did on iac-bench 2026-10: an overall composite of 88.9, scores per fixture, strengths, weaknesses and the patterns in its OpenTofu modules.