New: LLM Rankings are live!
See how the top LLMs rank on real infrastructure tasks
How Kimi K3.1 Pro did on iac-bench 2026-10: an overall composite of 86.5, scores per fixture, strengths, weaknesses and the patterns in its OpenTofu modules.
Every model evaluated on the 2026-10 fixtures side by side, with a sortable leaderboard, picks by use case and the cost of each: LLM rankings.
| Developer | Moonshot AI |
| API identifier | moonshotai/kimi-k3.1-pro |
| Access | OpenRouter, pinned to Moonshot AI's endpoint (MXFP4) |
| Weights | Open |
| List price (USD / M tokens, in / out) | 2.00 / 10.00 (OpenRouter, 2026-11-10) |
| Evaluated | 2026-11-10 (generation and judge) |
| Attempts | 9 (3 fixtures × 3 samples) |
| Overall composite | 86.5 (95% CI 84.8–88.7) |
| Rank | 6 of 16 (95% rank interval 3–9) |
| Mechanical only / judge only | 85.6 (rank 3) / 88.6 (rank 9) |
| Failed gates · insecure attempts | 0/9 · 0/9 |
| Estimated cost, all 9 attempts | $3.52 |
Kimi K3.1 Pro places sixth of 16 by composite (86.5), the highest open-weight model, with a 95% rank interval of 3–9. It is third on the mechanical score alone (85.6, behind Claude Fable 5.2 and Claude Opus 5.5) and ninth on the judge score (88.6). In the bootstrap it finishes ahead of Claude Sonnet 5.5 in 54% of resamples, ahead of Claude Fable 5.1 in 35%, GPT-6.1 Sol in 33% and GPT-6 Astra in 27%, and ahead of Kimi K3 in more than 99%. All 9 attempts cost $3.52, about the same as Claude Opus 5.5 ($3.58), against $1.52 for Sonnet, $1.02 for Sol and $3.96 for Kimi K3.
All 9 attempts planned, with no security findings. Kimi K3 had four findings and one failed gate. S3 (G001) is its strongest fixture, tied fourth in the field by median composite (85.6, with Claude Fable 5.1). App Service (G003) has its highest median (87.8) but places eighth, and Cloud SQL (G002) places seventh (86.0, up from 79.2 for Kimi K3).
| Fixture | Composite median (range) | Rubric median | Expertise | Judge median | ✗ gate | ⚠ security | Median inputs | Median output tokens | Median time | Median cost |
|---|---|---|---|---|---|---|---|---|---|---|
| G001 · S3 | 85.6 (84.0–86.4) | 0.849 | 4/7 | 92.1 | 0/3 | 0/3 | 12 | 12,640 | 526 s | $0.174 |
| G002 · Cloud SQL | 86.0 (84.2–91.8) | 0.875 | 7/9 | 86.8 | 0/3 | 0/3 | 24 | 19,216 | 720 s | $0.300 |
| G003 · App Service | 87.8 (84.7–87.8) | 0.889 | 4/5 | 86.8 | 0/3 | 0/3 | 51 | 42,983 | 1866 s | $0.669 |
Rubric items not at full credit, counted over attempts that planned: G001: A5 3/3, H12 3/3, H13 3/3, H15 3/3; G002: A8 1/3, B1 3/3, H4 3/3, H5 3/3, H9 1/3, H11 2/3; G003: A7 3/3. Definitions are in Scoring criteria.
ssl_mode = "ENCRYPTED_ONLY" is a
literal in 3/3 attempts, next to ipv4_enabled = false (H2 passes 3/3; Kimi K3 failed
it 3/3 by validating an input that admitted unencrypted connections). No S3 attempt offers
public reads (H1, which Kimi K3 lost in 1/3). cloudsql.iam_authentication = "on" last, so
it cannot be overridden. log_connections and log_disconnections are on in 3/3 (E8,
which Kimi K3 misses in 3/3).health_check_eviction_time_in_min, the pairing that failed Kimi K3's gate. Autoscale,
WEBSITES_PORT, metric alerts and AcrPull (E1–E4) appear in 3/3; Key Vault references
(E5) in none.B1 is lost in 3 of 9 attempts, all on Cloud SQL,
against 6 of 9 for Kimi K3 and Claude Fable 5.1 and 5 of 9 for GPT-6.1 Sol.A7 scores zero in 3/3. Kimi K3 used 20–34, and the next-largest median is
Gemini 3.6 Flash at 40. Many are attribute pass-throughs (ip_restrictions,
vnet_route_all_enabled, container_start_time_limit). container_registry_id beside
container_registry_url in 3/3, and a managed identity's client ID beside its principal
ID in e2. A registry password is accepted as an alternative to .gserviceaccount.com suffix, which Cloud SQL rejects,
in 3/3 (Kimi K3 trims it in 3/3). The e1 and e3 READMEs document the full email as the
username: H4/ H5 at 0.75,
3/3), where Kimi K3 offered one;ENTERPRISE_PLUS (35 days) is offered; Cloud SQL provider ranges such as
>= 5.20.0, < 7.0.0 that span two majors; no AWS provider block (H15, 3/3); S3 over
budget with 12–13 inputs (A5, 3/3). H11 lost 2/3).deletion_policy.
e2 defaults it to ABANDON through an optional() attribute of the databases map,
which earns full credit.H13, 3/3). logging.amazonaws.com
instead of logging.s3.amazonaws.com (excerpt below).cloudsql.instanceUser and
cloudsql.client to each IAM user (E6), which Kimi K3 never does. Validation stays as
dense as Kimi K3's: 20–21 blocks per Cloud SQL module, 22–28 per App Service module.
G001/kimi-k3.1-pro-e2/main.tf lines 254–270
statement {
sid = "AllowServerAccessLogDelivery"
effect = "Allow"
principals {
type = "Service"
identifiers = ["logging.amazonaws.com"]
}
actions = ["s3:PutObject"]
resources = ["${aws_s3_bucket.logs[0].arn}/access-logs/*"]
condition {
test = "ArnLike"
variable = "aws:SourceArn"
values = [aws_s3_bucket.this.arn]
}
The one S3 attempt that creates its own log bucket also writes a delivery grant for it, scoped to this bucket's ARN, but names the service principal logging.amazonaws.com. S3 server access logs are delivered by logging.s3.amazonaws.com, so the statement admits nothing and the rubric's delivery check (H13) fails.
How Claude Fable 5.1 did on iac-bench 2026-10: an overall composite of 86.9, scores per fixture, strengths, weaknesses and the patterns in its OpenTofu modules.
How Claude Fable 5.2 did on iac-bench 2026-10: an overall composite of 93.9, scores per fixture, strengths, weaknesses and the patterns in its OpenTofu modules.
How Claude Haiku 5.5 did on iac-bench 2026-10: an overall composite of 68.0, scores per fixture, strengths, weaknesses and the patterns in its OpenTofu modules.