# TTFT benchmark: LLM Gateway vs. OpenRouter (Claude-haiku-4.5, 150 runs)

> Source: <https://gist.github.com/smakosh/0ca360230fb267d375b91dbd50548b67>
> Published: 2026-07-22 16:41:00+00:00

# LLM Gateway vs. OpenRouter — TTFT benchmark

Phase-by-phase latency comparison using [rbadillap/ai-gateways-benchmark](https://github.com/rbadillap/ai-gateways-benchmark) (Python stdlib only, raw sockets — it splits every request into DNS / TCP / TLS / TTFB / TTFT so connection cost and gateway overhead don't get conflated).

**Method**: `claude-haiku-4.5` on both gateways · same prompt · `max_tokens=16` · 75 cold + 75 warm runs, interleaved round-robin so time-of-day drift hits both equally · cold = fresh connection (DNS+TCP+TLS, new TLS context, no session resumption) · warm = second request on an already-open socket · measured 2026-07-22 from a single residential connection · **zero errors across all 300 requests**.

## Results

| TTFT end-to-end (medians) | TTFB | TTFT (cold) | TTFT (warm) |
|---|---:|---:|---:|
| **LLM Gateway** | 201ms | 906ms | 814ms |
| **OpenRouter (direct)** | 1369ms | 1392ms | 1232ms |

TTFT (cold) is end-to-end: DNS + TCP + TLS + time to first content token — what a short-lived process pays. TTFT (warm) is the connection-pool case.

### Full phase breakdown (medians, ms)

| Gateway | DNS | TCP | TLS | TTFB | TTFT | Cold e2e TTFT | Warm TTFB | Warm TTFT |
|---|---:|---:|---:|---:|---:|---:|---:|---:|
| LLM Gateway | 3.2 | 20.9 | 40.2 | 200.9 | 829.5 | 906.1 | 175.7 | 813.8 |
| OpenRouter | 3.4 | 7.2 | 12.7 | 1369.4 | 1369.8 | 1392.0 | 1228.7 | 1232.1 |

### Spread — median (p10–p90), ms

| | LLM Gateway | OpenRouter |
|---|---:|---:|
| Cold TTFB | 201 (194–240) | 1369 (979–1643) |
| Cold e2e TTFT | 906 (803–1380) | 1392 (1002–1675) |
| Warm TTFB | 176 (171–193) | 1229 (924–1500) |
| Warm TTFT | 814 (673–1379) | 1232 (924–1501) |

## Observations

- **LLM Gateway is ~35% faster to first token cold (906 vs 1392ms) and ~34% faster warm (814 vs 1232ms)** at the median. Its p90 cold e2e TTFT (1380ms) is still below OpenRouter's median.
- **The TTFB gap is architectural, not just network.** LLM Gateway starts the response stream in ~200ms, before the first upstream token, and its warm TTFB is extremely stable (171–193ms p10–p90). OpenRouter's first byte arrives together with its first token (TTFB ≈ TTFT on every run) — it holds the response until the upstream answers. TTFT is the honest headline number; TTFB mostly tells you who streams headers early.
- **Different upstreams.** OpenRouter picks its upstream per request (a test request was served via Amazon Bedrock); LLM Gateway's runs were pinned to Anthropic's API. Both are each gateway's default behavior for this model.
- **Vantage point matters.** As the benchmark's README says, results are a property of where you measure from, not a global ranking. OpenRouter's edge terminated in Copenhagen for these runs (`cf-ray: …-CPH`); request receipts were captured per run.

## Reproduce it

```sh
git clone https://github.com/rbadillap/ai-gateways-benchmark
cd ai-gateways-benchmark
# config.json:
{
	"runs_cold": 75,
	"runs_warm": 75,
	"prompt": "Reply with the single word: pong",
	"max_tokens": 16,
	"gateways": [
		{
			"name": "llmgateway",
			"host": "api.llmgateway.io",
			"path": "/v1/chat/completions",
			"model": "anthropic/claude-haiku-4-5",
			"auth_value": "Bearer $LLMGATEWAY_API_KEY"
		},
		{
			"name": "openrouter",
			"host": "openrouter.ai",
			"path": "/api/v1/chat/completions",
			"model": "anthropic/claude-haiku-4.5",
			"auth_value": "Bearer $OPENROUTER_API_KEY"
		}
	]
}

LLMGATEWAY_API_KEY=... OPENROUTER_API_KEY=... python3 bench.py config.json
```