GPT-6.1 Sol on iac-bench 2026-10

OpenAI's GPT-6.1 Sol scored an overall composite of 87.2 on the iac-bench 2026-10 infrastructure-as-code benchmark, ranking third of 16 models with a 95% rank interval of 2–8, according to Massdriver's LLM rankings. GPT-6.1 Sol passed all 9 attempts (3 fixtures × 3 samples) with 0 failed gates and 0 insecure attempts, at an estimated cost of $1.02 for all 9 attempts — the cheapest in its group, against $1.52 for Claude Sonnet 5.5 and $7.26 for GPT-6 Astra. Its strongest fixture was App Service (G003) at a 94.1 median composite, while S3 (G001) was its weakest at 78.1, placing eleventh in the field.

New: LLM Rankings are live See how the top LLMs rank on real infrastructure tasks https://www.massdriver.cloud/llm-rankings How GPT-6.1 Sol did on iac-bench 2026-10: an overall composite of 87.2, scores per fixture, strengths, weaknesses and the patterns in its OpenTofu modules. Every model evaluated on the 2026-10 fixtures side by side, with a sortable leaderboard, picks by use case and the cost of each: LLM rankings https://www.massdriver.cloud/llm-rankings . | Developer | OpenAI | | API identifier | gpt-6.1-sol | | Access | OpenAI API Responses | | Weights | Closed | | List price USD / M tokens, in / out | 2.00 / 10.00 OpenRouter, 2026-11-10 | | Evaluated | 2026-11-10 generation and judge | | Attempts | 9 3 fixtures × 3 samples | | Overall composite | 87.2 95% CI 84.9–92.3 | | Rank | 3 of 16 95% rank interval 2–8 | | Mechanical only / judge only | 83.9 rank 5 / 94.7 rank 3 | | Failed gates · insecure attempts | 0/9 · 0/9 | | Estimated cost, all 9 attempts | $1.02 | GPT-6.1 Sol places third of 16 by composite 87.2 and sits in the second group with Claude Opus 5.5, Claude Fable 5.1, GPT-6 Astra, Kimi K3.1 Pro and Claude Sonnet 5.5, with a 95% rank interval of 2–8. In the bootstrap it finishes ahead of Astra in 49% of resamples, ahead of Fable 5.1 in 54%, ahead of Sonnet in 71% and ahead of Kimi K3.1 Pro in 67%. It is the cheapest model in that group: $1.02 for all 9 attempts, against $1.52 for Sonnet and $7.26 for Astra. All 9 attempts planned, with no security findings. App Service G003 is its strongest fixture, third in the field by median composite. S3 G001 is eleventh. | Fixture | Composite median range | Rubric median | Expertise | Judge median | ✗ gate | ⚠ security | Median inputs | Median output tokens | Median time | Median cost | |---|---|---|---|---|---|---|---|---|---|---| | G001 · S3 | 78.1 72.8–91.6 | 0.721 | 4/7 | 94.7 | 0/3 | 0/3 | 6 | 4,434 | 129 s | $0.066 | | G002 · Cloud SQL | 89.3 87.7–89.3 | 0.875 | 5/9 | 100.0 | 0/3 | 0/3 | 17 | 7,559 | 165 s | $0.110 | | G003 · App Service | 94.1 94.1–95.9 | 0.972 | 4/5 | 89.5 | 0/3 | 0/3 | 17 | 10,013 | 379 s | $0.168 | Rubric items not at full credit, counted over attempts that planned: G001: A5 1/3, B1 2/3, H6 2/3, H7 2/3, H9 2/3, H12 3/3, H13 3/3, H14 1/3, H15 1/3; G002: B1 3/3, H4 3/3, H5 3/3, H11 3/3; G003: B2 2/3. Definitions are in Scoring criteria https://www.massdriver.cloud/blogs/iac-bench-2026-10-scoring-criteria . connector enforcement = "REQUIRED" and validation that rejects overrides of IAM authentication. On App Service, both basic-auth publishing flags are off on the app and the slot; Claude Sonnet 5.5 leaves them on in 2/3 H2 . cloudsql.client and cloudsql.instanceUser E6 , the grant GPT-6 Luna omits. AcrPull goes to the identities the app and the staging slot pull with, where GPT-6 Luna uses registry passwords. A7 , which Sonnet exceeds in 3/3. H13 , 3/3 ; H9 lost 2/3 ; H6 , H7 lost 2/3 . The other two expire only incomplete multipart uploads. H11 , 3/3 . google sql database sets deletion policy . Backups and point-in-time recovery cannot be turned off H4 / H5 at 0.75 in 3/3 . B1 , including = 6.0, < 8.0 in all three Cloud SQL modules. cors rules mirrors the provider's cors rule block. registry url that can disagree. iam principals passes IAM USER -style values to google sql user.type , which expects CLOUD IAM USER . The judge caught it capacity object, and the S3 e3 attempt enables access logging when an access logging object is supplied. The rubric credits both WEBSITE SWAP WARMUP PING PATH at the health-check path, so a swap waits for the staging slot to respond. G001/gpt-6.1-sol-e1/main.tf lines 63–75 Statement = { Sid = "DenyInsecureTransport" Effect = "Deny" Principal = " " Action = "s3: " Resource = aws s3 bucket.assets.arn, "${aws s3 bucket.assets.arn}/ " Condition = { Bool = { "aws:SecureTransport" = "false" "aws:PrincipalIsAWSService" = "false" } } } The TLS-deny statement carries a second condition that exempts AWS service principals, with the comment "Exempt direct AWS service principals as their network context can be redacted." It appears in all three S3 attempts and in no other model's modules. How Claude Fable 5.1 did on iac-bench 2026-10: an overall composite of 86.9, scores per fixture, strengths, weaknesses and the patterns in its OpenTofu modules. How Claude Fable 5.2 did on iac-bench 2026-10: an overall composite of 93.9, scores per fixture, strengths, weaknesses and the patterns in its OpenTofu modules. How Claude Haiku 5.5 did on iac-bench 2026-10: an overall composite of 68.0, scores per fixture, strengths, weaknesses and the patterns in its OpenTofu modules.