{"slug": "gpt-6-1-sol-on-iac-bench-2026-10", "title": "GPT-6.1 Sol on iac-bench 2026-10", "summary": "OpenAI's GPT-6.1 Sol scored an overall composite of 87.2 on the iac-bench 2026-10 infrastructure-as-code benchmark, ranking third of 16 models with a 95% rank interval of 2–8, according to Massdriver's LLM rankings. GPT-6.1 Sol passed all 9 attempts (3 fixtures × 3 samples) with 0 failed gates and 0 insecure attempts, at an estimated cost of $1.02 for all 9 attempts — the cheapest in its group, against $1.52 for Claude Sonnet 5.5 and $7.26 for GPT-6 Astra. Its strongest fixture was App Service (G003) at a 94.1 median composite, while S3 (G001) was its weakest at 78.1, placing eleventh in the field.", "body_md": "New: LLM Rankings are live!\n\n[See how the top LLMs rank on real infrastructure tasks](https://www.massdriver.cloud/llm-rankings)\n\nHow GPT-6.1 Sol did on iac-bench 2026-10: an overall composite of 87.2, scores per fixture, strengths, weaknesses and the patterns in its OpenTofu modules.\n\nEvery model evaluated on the 2026-10 fixtures side by side, with a sortable leaderboard, picks by use case and the cost of each: [LLM rankings](https://www.massdriver.cloud/llm-rankings).\n\n| Developer | OpenAI | \n| API identifier | `gpt-6.1-sol` | \n| Access | OpenAI API (Responses) | \n| Weights | Closed | \n| List price (USD / M tokens, in / out) | 2.00 / 10.00 (OpenRouter, 2026-11-10) | \n| Evaluated | 2026-11-10 (generation and judge) | \n| Attempts | 9 (3 fixtures × 3 samples) | \n| **Overall composite** | **87.2** (95% CI 84.9–92.3) | \n| Rank | 3 of 16 (95% rank interval 2–8) | \n| Mechanical only / judge only | 83.9 (rank 5) / 94.7 (rank 3) | \n| Failed gates · insecure attempts | 0/9 · 0/9 | \n| Estimated cost, all 9 attempts | $1.02 | \n\nGPT-6.1 Sol places third of 16 by composite (87.2) and sits in the second group with Claude Opus 5.5, Claude Fable 5.1, GPT-6 Astra, Kimi K3.1 Pro and Claude Sonnet 5.5, with a 95% rank interval of 2–8. In the bootstrap it finishes ahead of Astra in 49% of resamples, ahead of Fable 5.1 in 54%, ahead of Sonnet in 71% and ahead of Kimi K3.1 Pro in 67%. It is the cheapest model in that group: $1.02 for all 9 attempts, against $1.52 for Sonnet and $7.26 for Astra.\n\nAll 9 attempts planned, with no security findings. App Service (G003) is its strongest fixture, third in the field by median composite. S3 (G001) is eleventh.\n\n| Fixture | Composite median (range) | Rubric median | Expertise | Judge median | ✗ gate | ⚠ security | Median inputs | Median output tokens | Median time | Median cost | \n|---|---|---|---|---|---|---|---|---|---|---|\n| G001 · S3 | 78.1 (72.8–91.6) | 0.721 | 4/7 | 94.7 | 0/3 | 0/3 | 6 | 4,434 | 129 s | $0.066 | \n| G002 · Cloud SQL | 89.3 (87.7–89.3) | 0.875 | 5/9 | 100.0 | 0/3 | 0/3 | 17 | 7,559 | 165 s | $0.110 | \n| G003 · App Service | 94.1 (94.1–95.9) | 0.972 | 4/5 | 89.5 | 0/3 | 0/3 | 17 | 10,013 | 379 s | $0.168 | \n\nRubric items not at full credit, counted over attempts that planned: G001: `A5` 1/3, `B1` 2/3, `H6` 2/3, `H7` 2/3, `H9` 2/3, `H12` 3/3, `H13` 3/3, `H14` 1/3, `H15` 1/3; G002: `B1` 3/3, `H4` 3/3, `H5` 3/3, `H11` 3/3; G003: `B2` 2/3. Definitions are in [Scoring criteria](https://www.massdriver.cloud/blogs/iac-bench-2026-10-scoring-criteria).\n\n`connector_enforcement = \"REQUIRED\"` and validation that rejects overrides of IAM\nauthentication. On App Service, both basic-auth publishing flags are off on the app and\nthe slot; Claude Sonnet 5.5 leaves them on in 2/3 (`H2`).` cloudsql.client` and `cloudsql.instanceUser` (E6), the grant GPT-6 Luna omits.`AcrPull` goes to the identities the app and\nthe staging slot pull with, where GPT-6 Luna uses registry passwords.`A7`), which Sonnet exceeds in 3/3.` H13`, 3/3);` H9` lost 2/3);`H6`, `H7` lost 2/3). The\nother two expire only incomplete multipart uploads.`H11`, 3/3).` google_sql_database` sets\n`deletion_policy`. Backups and point-in-time recovery cannot be turned off (` H4`/` H5` at\n0.75 in 3/3).`B1`), including\n`>= 6.0, < 8.0` in all three Cloud SQL modules.`cors_rules` mirrors the provider's `cors_rule` block. `registry_url` that can disagree. `iam_principals` passes `IAM_USER`-style values to\n`google_sql_user.type`, which expects `CLOUD_IAM_USER`. The judge caught it (` capacity` object, and the S3 e3 attempt enables\naccess logging when an `access_logging` object is supplied. The rubric credits both\n(`WEBSITE_SWAP_WARMUP_PING_PATH`\nat the health-check path, so a swap waits for the staging slot to respond.\n`G001/gpt-6.1-sol-e1/main.tf` lines 63–75\n\n```\n    Statement = [{\n      Sid       = \"DenyInsecureTransport\"\n      Effect    = \"Deny\"\n      Principal = \"*\"\n      Action    = \"s3:*\"\n      Resource  = [aws_s3_bucket.assets.arn, \"${aws_s3_bucket.assets.arn}/*\"]\n      Condition = {\n        Bool = {\n          \"aws:SecureTransport\"       = \"false\"\n          \"aws:PrincipalIsAWSService\" = \"false\"\n        }\n      }\n    }]\n```\n\nThe TLS-deny statement carries a second condition that exempts AWS service principals, with the comment \"Exempt direct AWS service principals as their network context can be redacted.\" It appears in all three S3 attempts and in no other model's modules.\n\nHow Claude Fable 5.1 did on iac-bench 2026-10: an overall composite of 86.9, scores per fixture, strengths, weaknesses and the patterns in its OpenTofu modules.\n\nHow Claude Fable 5.2 did on iac-bench 2026-10: an overall composite of 93.9, scores per fixture, strengths, weaknesses and the patterns in its OpenTofu modules.\n\nHow Claude Haiku 5.5 did on iac-bench 2026-10: an overall composite of 68.0, scores per fixture, strengths, weaknesses and the patterns in its OpenTofu modules.", "url": "https://wpnews.pro/news/gpt-6-1-sol-on-iac-bench-2026-10", "canonical_source": "https://www.massdriver.cloud/blogs/iac-bench-2026-10-gpt-6-1-sol", "published_at": "2026-11-04 00:00:00+00:00", "updated_at": "2026-11-11 18:54:00.246356+00:00", "lang": "en", "topics": ["large-language-models", "ai-research", "ai-tools", "artificial-intelligence"], "entities": ["OpenAI", "GPT-6.1 Sol", "Massdriver", "iac-bench 2026-10", "Claude Opus 5.5", "Claude Sonnet 5.5", "GPT-6 Astra", "Kimi K3.1 Pro"], "also_reported_by": [{"url": "https://www.massdriver.cloud/blogs/iac-bench-2026-10-gpt-6-1-terra", "source": "massdriver.cloud", "title": "GPT-6.1 Terra on iac-bench 2026-10", "published_at": "2026-11-04T00:00:00+00:00"}], "alternates": {"html": "https://wpnews.pro/news/gpt-6-1-sol-on-iac-bench-2026-10", "markdown": "https://wpnews.pro/news/gpt-6-1-sol-on-iac-bench-2026-10.md", "text": "https://wpnews.pro/news/gpt-6-1-sol-on-iac-bench-2026-10.txt", "jsonld": "https://wpnews.pro/news/gpt-6-1-sol-on-iac-bench-2026-10.jsonld"}}