# Giraffebench 1.0 – The quality of local and cloud LLMs in a way you can see

> Source: <https://keno303.github.io/giraffebench/index.html>
> Published: 2026-11-20 04:09:50+00:00

**Recommendation:** Route heavy agent runs through harnesses that maximize cached input: Qwen4 Max billed cached tokens at $0.075 versus $1.50 per million new input, and the pi run's 1,295,616 cached tokens held total cost to $0.371.
More in the playbook "Controlling LLM cost": <https://wpnews.pro/playbooks/llm-cost>

[Run the giraffe](g/qwen--qwen4-max-pi/index.html)

### Qwen4 Max

Qwen · OpenRouter · 2400 B MoE

Released November 16, 2026

**25 min**

**26.4 tok/s**

**$0.371**

## Details

- Model on OpenRouter
- [qwen/qwen4-max](https://openrouter.ai/qwen/qwen4-max)
- Weights
- Closed model, weights not published
- Released
- November 16, 2026 (date the model was added to OpenRouter)
- Hardware
- Cloud via OpenRouter, served by Alibaba
- Harness
- pi 1.5.0
- Thinking
- Model default (pi and OpenCode sent no thinking setting to OpenRouter, so the model default applied)
19,240 reasoning tokens billed
- Coding time
- 25 min, 39 requests
- Tokens
- 1,411,522 (75,942 new input, 1,295,616 cached, 39,964 output)
- OpenRouter cost (real, via API)
- $0.371 (list price per million tokens: $1.50 input, $0.075 cached, $4.50 output)

[Run the giraffe](g/qwen--qwen4-max-oc/index.html)

### Qwen4 Max

Qwen · OpenRouter · 2400 B MoE

Released November 16, 2026

**20 min**

**24.7 tok/s**

**$0.549**

## Details

- Model on OpenRouter
- [qwen/qwen4-max](https://openrouter.ai/qwen/qwen4-max)
- Weights
- Closed model, weights not published
- Released
- November 16, 2026 (date the model was added to OpenRouter)
- Hardware
- Cloud via OpenRouter, served by Alibaba
- Harness
- OpenCode 2.2.5
- Thinking
- Model default (pi and OpenCode sent no thinking setting to OpenRouter, so the model default applied)
28,161 reasoning tokens billed
- Coding time
- 20 min, 49 requests
- Tokens
- 2,358,792 (100,580 new input, 2,228,736 cached, 29,476 output)
- OpenRouter cost (real, via API)
- $0.549 (list price per million tokens: $1.50 input, $0.075 cached, $4.50 output)
