{"slug": "claude-haiku-5-5-on-iac-bench-2026-10", "title": "Claude Haiku 5.5 on iac-bench 2026-10", "summary": "Anthropic's Claude Haiku 5.5 scored an overall composite of 68.0 (95% CI 66.1–90.6) on iac-bench 2026-10, ranking 9th of 10 models evaluated on generating OpenTofu infrastructure modules, with 2 of 9 attempts failing the gate and 1 of 9 flagged as insecure. The low rank was driven by the Cloud SQL fixture, where two of three modules failed because the model looked up the environment's network inside a required project_id that its own example inputs set to the placeholder \"example-dbops-project\"; the model saw the tofu_plan error and submitted anyway, while its third attempt avoided the lookup and scored 0.929. Claude Haiku 5.5 tied Claude Opus 5.5 for the field-best median of 88 on the S3 fixture and placed fourth on App Service, at a list price of $0.10/$0.50 per million input/output tokens and an estimated $0.11 for all 9 attempts.", "body_md": "Upcoming Workshop: Citizen Developer Agentic Software Factory on AWS 11.4.26 @ 12PM PST\n\n[Register Here!](https://streamyard.com/watch/umMggUFkVMNu)\n\nHow Claude Haiku 5.5 did on iac-bench 2026-10: an overall composite of 68.0, scores per fixture, strengths, weaknesses and the patterns in its OpenTofu modules.\n\n| Developer | Anthropic | \n| API identifier | `claude-haiku-5-5` | \n| Access | Anthropic API | \n| Weights | Closed | \n| List price (USD / M tokens, in / out) | 0.10 / 0.50 (OpenRouter, 2026-10-23) | \n| Evaluated | 2026-10-22 (generation) · 2026-10-23 (judge) | \n| Attempts | 9 (3 fixtures × 3 samples) | \n| **Overall composite** | **68.0** (95% CI 66.1–90.6) | \n| Rank | 9 of 10 (95% rank interval 1–9) | \n| Mechanical only / judge only | 59.2 (rank 9) / 91.2 (rank 4) | \n| Failed gates · insecure attempts | 2/9 · 1/9 | \n| Estimated cost, all 9 attempts | $0.11 | \n\nClaude Haiku 5.5's overall rank is driven by one fixture. On S3 (G001) its median of 88 ties Claude Opus 5.5 for the best in the field, and on App Service it places fourth, a tenth of a point behind Claude Fable 5.1. On Cloud SQL, two of its three modules failed the gate. Its rank interval (1–9) is the widest in the release, because one more good Cloud SQL attempt would have put it in the top group.\n\nBoth failures share one cause. The module looks up the environment's network inside a\nrequired `project_id`, and the model's own example inputs set that project ID to a\nplaceholder. `tofu_plan` failed, the model saw the error, and it submitted anyway, saying\nso in its summary. The third attempt avoided the lookup and scored 0.929.\n\nIt is among the cheapest models in the release, at about 1–2 cents per attempt.\n\n| Fixture | Composite median (range) | Rubric median | Expertise | Judge median | ✗ gate | ⚠ security | Median inputs | Median output tokens | Median time | Median cost | \n|---|---|---|---|---|---|---|---|---|---|---|\n| G001 · S3 | 87.8 (87.2–90.2) | 0.860 | 5/7 | 100.0 | 0/3 | 0/3 | 8 | 9,297 | 102 s | $0.006 | \n| G002 · Cloud SQL | 26.8 (25.3–91.0) | 0.000 | 7/9 | 89.5 | 2/3 | 0/3 | 22 | 17,089 | 84 s | $0.011 | \n| G003 · App Service | 89.4 (85.2–93.0) | 0.967 | 2/5 | 84.2 | 0/3 | 1/3 ( `H2` ) | 22 | 21,616 | 367 s | $0.017 | \n\nRubric items not at full credit, counted over attempts that planned: G001: `A5` 1/3, `H9` 3/3, `H12` 2/3, `H13` 3/3, `H14` 2/3, `H15` 1/3; G002: `H4` 1/1, `H5` 1/1, `H11` 1/1, 2 attempts failed the gate; G003: `A7` 2/3, `H2` 1/3. Definitions are in [Scoring criteria](https://www.massdriver.cloud/blogs/iac-bench-2026-10-scoring-criteria).\n\n`ENCRYPTED_ONLY`; App Service HTTPS and transport\nsettings.`asset_class` enum drives lifecycle through a map of\ndefaults. The judge scored `minimal.tfvars` to plan cleanly, and credentials come\nfrom the environment.`H2` (security finding, 1/3).`ftp_publish_basic_authentication_enabled = false` but omits the Web Deploy flag, which\nstays at the provider default (on).\n`G002/claude-haiku-5-5-e1/main.tf` lines 4–7\n\n```\ndata \"google_compute_network\" \"apps\" {\n  name    = local.network_name\n  project = coalesce(var.network_project_id, var.project_id)\n}\n```\n\n`G002/claude-haiku-5-5-e1/minimal.tfvars` lines 1–4\n\n```\nproject_id    = \"example-dbops-project\"\nregion        = \"europe-west1\"\nenvironment   = \"dev\"\ninstance_name = \"orders-dev-pg\"\n```\n\nA reasonable network lookup, which the reference module also uses, made unplannable by a required `project_id` that the model filled with a placeholder.\n\nHow Claude Fable 5.1 did on iac-bench 2026-10: an overall composite of 86.9, scores per fixture, strengths, weaknesses and the patterns in its OpenTofu modules.\n\nHow Claude Opus 5.5 did on iac-bench 2026-10: an overall composite of 88.9, scores per fixture, strengths, weaknesses and the patterns in its OpenTofu modules.\n\nHow Claude Sonnet 5.5 did on iac-bench 2026-10: an overall composite of 86.1, scores per fixture, strengths, weaknesses and the patterns in its OpenTofu modules.", "url": "https://wpnews.pro/news/claude-haiku-5-5-on-iac-bench-2026-10", "canonical_source": "https://www.massdriver.cloud/blogs/iac-bench-2026-10-claude-haiku-5-5", "published_at": "2026-11-04 00:00:00+00:00", "updated_at": "2026-11-04 20:21:27.506068+00:00", "lang": "en", "topics": ["large-language-models", "ai-research", "ai-tools", "ai-agents"], "entities": ["Anthropic", "Claude Haiku 5.5", "Claude Opus 5.5", "Claude Fable 5.1", "Claude Sonnet 5.5", "OpenTofu", "OpenRouter", "AWS"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/claude-haiku-5-5-on-iac-bench-2026-10", "markdown": "https://wpnews.pro/news/claude-haiku-5-5-on-iac-bench-2026-10.md", "text": "https://wpnews.pro/news/claude-haiku-5-5-on-iac-bench-2026-10.txt", "jsonld": "https://wpnews.pro/news/claude-haiku-5-5-on-iac-bench-2026-10.jsonld"}}