Upcoming Workshop: Citizen Developer Agentic Software Factory on AWS 11.4.26 @ 12PM PST
How Claude Haiku 5.5 did on iac-bench 2026-10: an overall composite of 68.0, scores per fixture, strengths, weaknesses and the patterns in its OpenTofu modules.
| Developer | Anthropic |
| API identifier | claude-haiku-5-5 |
| Access | Anthropic API |
| Weights | Closed |
| List price (USD / M tokens, in / out) | 0.10 / 0.50 (OpenRouter, 2026-10-23) |
| Evaluated | 2026-10-22 (generation) · 2026-10-23 (judge) |
| Attempts | 9 (3 fixtures × 3 samples) |
| Overall composite | 68.0 (95% CI 66.1–90.6) |
| Rank | 9 of 10 (95% rank interval 1–9) |
| Mechanical only / judge only | 59.2 (rank 9) / 91.2 (rank 4) |
| Failed gates · insecure attempts | 2/9 · 1/9 |
| Estimated cost, all 9 attempts | $0.11 |
Claude Haiku 5.5's overall rank is driven by one fixture. On S3 (G001) its median of 88 ties Claude Opus 5.5 for the best in the field, and on App Service it places fourth, a tenth of a point behind Claude Fable 5.1. On Cloud SQL, two of its three modules failed the gate. Its rank interval (1–9) is the widest in the release, because one more good Cloud SQL attempt would have put it in the top group.
Both failures share one cause. The module looks up the environment's network inside a
required project_id, and the model's own example inputs set that project ID to a
placeholder. tofu_plan failed, the model saw the error, and it submitted anyway, saying
so in its summary. The third attempt avoided the lookup and scored 0.929.
It is among the cheapest models in the release, at about 1–2 cents per attempt.
| Fixture | Composite median (range) | Rubric median | Expertise | Judge median | ✗ gate | ⚠ security | Median inputs | Median output tokens | Median time | Median cost |
|---|---|---|---|---|---|---|---|---|---|---|
| G001 · S3 | 87.8 (87.2–90.2) | 0.860 | 5/7 | 100.0 | 0/3 | 0/3 | 8 | 9,297 | 102 s | $0.006 |
| G002 · Cloud SQL | 26.8 (25.3–91.0) | 0.000 | 7/9 | 89.5 | 2/3 | 0/3 | 22 | 17,089 | 84 s | $0.011 |
| G003 · App Service | 89.4 (85.2–93.0) | 0.967 | 2/5 | 84.2 | 0/3 | 1/3 ( H2 ) |
22 | 21,616 | 367 s | $0.017 |
Rubric items not at full credit, counted over attempts that planned: G001: A5 1/3, H9 3/3, H12 2/3, H13 3/3, H14 2/3, H15 1/3; G002: H4 1/1, H5 1/1, H11 1/1, 2 attempts failed the gate; G003: A7 2/3, H2 1/3. Definitions are in Scoring criteria.
ENCRYPTED_ONLY; App Service HTTPS and transport
settings.asset_class enum drives lifecycle through a map of
defaults. The judge scored minimal.tfvars to plan cleanly, and credentials come
from the environment.H2 (security finding, 1/3).ftp_publish_basic_authentication_enabled = false but omits the Web Deploy flag, which
stays at the provider default (on).
G002/claude-haiku-5-5-e1/main.tf lines 4–7
data "google_compute_network" "apps" {
name = local.network_name
project = coalesce(var.network_project_id, var.project_id)
}
G002/claude-haiku-5-5-e1/minimal.tfvars lines 1–4
project_id = "example-dbops-project"
region = "europe-west1"
environment = "dev"
instance_name = "orders-dev-pg"
A reasonable network lookup, which the reference module also uses, made unplannable by a required project_id that the model filled with a placeholder.
How Claude Fable 5.1 did on iac-bench 2026-10: an overall composite of 86.9, scores per fixture, strengths, weaknesses and the patterns in its OpenTofu modules.
How Claude Opus 5.5 did on iac-bench 2026-10: an overall composite of 88.9, scores per fixture, strengths, weaknesses and the patterns in its OpenTofu modules.
How Claude Sonnet 5.5 did on iac-bench 2026-10: an overall composite of 86.1, scores per fixture, strengths, weaknesses and the patterns in its OpenTofu modules.