{"slug": "gpt-6-astra-on-iac-bench-2026-10", "title": "GPT-6 Astra on iac-bench 2026-10", "summary": "OpenAI's GPT-6 Astra scored an overall composite of 86.7 (95% CI 86.0–89.8) on the iac-bench 2026-10 evaluation, ranking 3rd of 10 models across 9 attempts on three OpenTofu infrastructure fixtures, with zero failed gates and zero insecure attempts. GPT-6 Astra posted the highest median composite on App Service (95.9) and Cloud SQL (91.3), narrowly ahead of Claude Sonnet 5.5, but fell to the lower half of the field on S3 (72.8) because its bucket module included no retention controls, costing 21 of that fixture's 86 weight points. The 9 attempts cost an estimated $7.26 in total at a list price of $10.00 per million input tokens and $50.00 per million output tokens, evaluated on 2026-10-22 and judged on 2026-10-23.", "body_md": "Upcoming Workshop: Citizen Developer Agentic Software Factory on AWS 11.4.26 @ 12PM PST\n\n[Register Here!](https://streamyard.com/watch/umMggUFkVMNu)\n\nHow GPT-6 Astra did on iac-bench 2026-10: an overall composite of 86.7, scores per fixture, strengths, weaknesses and the patterns in its OpenTofu modules.\n\n| Developer | OpenAI | \n| API identifier | `gpt-6-astra` | \n| Access | OpenAI API (Responses) | \n| Weights | Closed | \n| List price (USD / M tokens, in / out) | 10.00 / 50.00 (OpenRouter, 2026-10-23) | \n| Evaluated | 2026-10-22 (generation) · 2026-10-23 (judge) | \n| Attempts | 9 (3 fixtures × 3 samples) | \n| **Overall composite** | **86.7** (95% CI 86.0–89.8) | \n| Rank | 3 of 10 (95% rank interval 1–5) | \n| Mechanical only / judge only | 83.2 (rank 3) / 94.7 (rank 2) | \n| Failed gates · insecure attempts | 0/9 · 0/9 | \n| Estimated cost, all 9 attempts | $7.26 | \n\nGPT-6 Astra has the highest median composite on App Service (G003) and on Cloud SQL (G002), where it is narrowly ahead of Claude Sonnet 5.5. It falls into the lower half of the field on S3 (G001). The spread across fixtures is the widest of any model in the top group, and it comes from a single design stance rather than from errors. On S3 it builds a bucket that never deletes anything and offers no retention controls. The rubric treats that as missing data-hygiene items. The judge reads the same module as restrained and free of gold-plating.\n\nAll 9 attempts planned. No security setting is exposed as an input in any of them.\n\n| Fixture | Composite median (range) | Rubric median | Expertise | Judge median | ✗ gate | ⚠ security | Median inputs | Median output tokens | Median time | Median cost | \n|---|---|---|---|---|---|---|---|---|---|---|\n| G001 · S3 | 72.8 (72.0–79.6) | 0.640 | 4/7 | 94.7 | 0/3 | 0/3 | 4 | 4,361 | 189 s | $0.318 | \n| G002 · Cloud SQL | 91.3 (89.9–93.9) | 0.908 | 5/9 | 100.0 | 0/3 | 0/3 | 19 | 6,411 | 158 s | $0.447 | \n| G003 · App Service | 95.9 (95.2–98.3) | 1.000 | 4/5 | 89.5 | 0/3 | 0/3 | 19 | 21,718 | 561 s | $1.802 | \n\nRubric items not at full credit, counted over attempts that planned: G001: `B1` 2/3, `H6` 3/3, `H7` 3/3, `H9` 3/3, `H12` 3/3, `H13` 3/3, `H14` 3/3, `H15` 2/3; G002: `B1` 2/3, `H4` 3/3, `H5` 3/3, `H11` 2/3; G003: `A7` 1/3. Definitions are in [Scoring criteria](https://www.massdriver.cloud/blogs/iac-bench-2026-10-scoring-criteria).\n\n`connector_enforcement = \"REQUIRED\"`, and validation rejects any attempt to override IAM\nauthentication. On App Service, HTTPS-only and disabled basic-auth publishing are set on\nboth the app and the slot.`cloudsql.client` and\n`cloudsql.instanceUser` with an IAM condition on the instance name, in 3/3 attempts. This\ncloses a cross-instance access path that the rubric does not test (excerpt below).\n**S3 has no retention controls (3/3).** It has:\n\n`H6`);` H7`), so with versioning on, old versions accumulate\nwithout limit;`H9`, `H13`);` force_destroy` (`H14`).\nTogether these cost 21 of the fixture's 86 weight points. A code comment gives the reason: the module \"cannot know the application's retention requirements.\"\n\n**Provider constraints that admit a new major:** `>= 5.0, < 7.0` and `>= 6.0, < 8.0` in\n4 of 9 attempts (`B1`).\n\n**Backups and point-in-time recovery cannot be turned off** on Cloud SQL (`H4`/` H5` at\n0.75 in 3/3).\n\n**Duplicate inputs on App Service.** Every attempt takes the registry both as a login URL\nand as an ACR resource ID, though the URL can be looked up from the ID. The judge marked\nthis down under *Data sources vs inputs* in 8 of 9 readings.\n\n`optional()` defaults`capacity`, `staging`, `registry`),\nchecked with preconditions.\n`G002/gpt-6-astra-e3/main.tf` lines 102–116\n\n```\n# Additive grants, not authoritative policies. Conditions prevent this module's\n# grants from allowing connection/login to other instances in the project.\nresource \"google_project_iam_member\" \"database_access\" {\n  for_each = local.iam_grants\n\n  project = var.project_id\n  member  = each.value.member\n  role    = each.value.role\n\n  condition {\n    title       = \"cloudsql-${var.instance_name}\"\n    description = \"Connection and IAM database login to ${var.instance_name} only\"\n    expression  = \"resource.name == 'projects/${var.project_id}/instances/${google_sql_database_instance.postgres.name}' && resource.service == 'sqladmin.googleapis.com'\"\n  }\n}\n```\n\nProject-level Cloud SQL roles granted with an IAM condition on this one instance. Without the condition, the same grant would allow login to every instance in the project.\n\nThe S3 fixture shows the largest gap between the two components for this model: mechanical about 64 against a judge grade of 94.7. The doctrine the judge applies does not require retention controls on a general-purpose bucket. The rubric does, because the brief names logs and generated files, data that accumulates.\n\nHow Claude Fable 5.1 did on iac-bench 2026-10: an overall composite of 86.9, scores per fixture, strengths, weaknesses and the patterns in its OpenTofu modules.\n\nHow Claude Haiku 5.5 did on iac-bench 2026-10: an overall composite of 68.0, scores per fixture, strengths, weaknesses and the patterns in its OpenTofu modules.\n\nHow Claude Opus 5.5 did on iac-bench 2026-10: an overall composite of 88.9, scores per fixture, strengths, weaknesses and the patterns in its OpenTofu modules.", "url": "https://wpnews.pro/news/gpt-6-astra-on-iac-bench-2026-10", "canonical_source": "https://www.massdriver.cloud/blogs/iac-bench-2026-10-gpt-6-astra", "published_at": "2026-11-04 00:00:00+00:00", "updated_at": "2026-11-04 20:21:39.800325+00:00", "lang": "en", "topics": ["ai-research", "large-language-models", "artificial-intelligence", "ai-infrastructure"], "entities": ["OpenAI", "GPT-6 Astra", "iac-bench 2026-10", "Claude Sonnet 5.5", "OpenRouter", "OpenTofu", "AWS App Service", "Cloud SQL"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/gpt-6-astra-on-iac-bench-2026-10", "markdown": "https://wpnews.pro/news/gpt-6-astra-on-iac-bench-2026-10.md", "text": "https://wpnews.pro/news/gpt-6-astra-on-iac-bench-2026-10.txt", "jsonld": "https://wpnews.pro/news/gpt-6-astra-on-iac-bench-2026-10.jsonld"}}