Disclosure: I maintain Lynkr, the router being benchmarked, so read everything here with that in mind. The mitigation: every number in this post comes from RouterArena's automated evaluation pipeline, run by their CI on their infrastructure, not by me — including the numbers that make us look mediocre.
Every LLM router's README — ours included — makes the same claim: it sends easy queries to cheap models and hard queries to good ones, and saves you money without hurting quality. Almost none of them attach evidence. The numbers that do exist are self-reported, measured on datasets the vendor picked, with baselines the vendor chose.
So when RouterArena showed up — an open, standardized benchmark for LLM routers out of the RouteWorks group (paper, leaderboard) — we submitted Lynkr to it. This post is what we learned, including the metrics where we did badly, because a benchmark result you only quote selectively is just marketing with extra steps.
RouterArena evaluates a router over 8,400 queries spanning 9 domains and 44 categories at three difficulty levels, and scores it on five axes:
One rule matters more than the metrics: RouterArena is evaluation-only. Any router component trained, fitted, or tuned on their data or labels gets rejected, and violations found later get withdrawn. Keep that rule in mind — it comes back at the end of this post.
We submitted Lynkr's untuned production default — the same complexity scorer and the same tier boundaries that ship in the repo. The adapter is ~70 lines of Python that sends each query to a live Lynkr instance's /routing/analyze
endpoint and maps the returned tier to a model. No benchmark-special code path: the endpoint runs the exact intent scorer the live proxy uses (local embeddings, no LLM call in the routing decision itself).
The model pool, all via OpenRouter:
| Lynkr tier | Model |
|---|---|
| SIMPLE | gpt-oss-120b |
| MEDIUM | Qwen3-235B-A22B |
| COMPLEX / REASONING | GLM-4.7 |
Running all 8,400 queries cost $2.46 total. That's the entire eval bill, which says something on its own about where inference prices are in mid-2026.
From RouterArena's automated evaluation on our submission PR:
| Metric | Lynkr |
|---|---|
| Arena Score | 67.65 |
| Accuracy | 68.41% |
| Cost per 1K queries | $0.29 |
| Robustness | 92.38 |
| Opt.Acc (accuracy vs. optimal) | 84.48 |
| Opt.Cost (cost efficiency vs. optimal) | 16.08 |
| Opt.Sel (optimal model selection) | 10.97 |
On the official leaderboard (our submission merged 2026-07-23), that lands mid-table — 15th of 27 routers. We beat GPT-5 used as a router — at 34× lower cost. The leaderboard includes GPT-5 itself as a routing baseline: Arena score 64.32 at $10.02 per 1K queries. Lynkr scores 67.65 at $0.29 per 1K. A local intent scorer making the decision without any LLM call outperforms asking a frontier model to route — which is the entire thesis Lynkr is built on, so we're relieved the benchmark agrees. Several well-known systems (NotDiamond at 57.29, RouteLLM at 48.07, both RouterBench baselines) also land below us.
Robustness 92.38 is top-five territory. Most leaderboard routers sit between 22 and 72 — meaning a rephrased query frequently flips their model choice. Lynkr's routing decision survives rephrasing 92% of the time. For an interactive tool where users iterate on prompts, decision stability is arguably worth more than a couple of accuracy points: a router that sends your reworded follow-up to a different model mid-conversation is a router you turn off.
Opt.Sel is 10.97. When multiple models in our pool could answer a query correctly, we picked the cheapest correct one about 11% of the time. Lynkr routes conservatively — when in doubt, it escalates a tier. That's a deliberate live-serving bias (a wrong cheap answer costs more user trust than an unnecessarily good one costs dollars), and $0.29/1K shows the absolute spend stays low. But the oracle comparison is unambiguous: there was money on the table we didn't pick up.
The accuracy gap to the top is real. Cross-Router leads at 78.14% accuracy; we're at 68.41%. Part of that is the pool (three self-hostable open-weight models — no frontier closed models to escalate to), but part is genuinely the router. Mid-table is where we are, not where we'd spin it.
Reading our failure cases was the most valuable part of the exercise. The clearest pattern: our SIMPLE→MEDIUM boundary is conservative — a chunk of queries scored just past the boundary and got a mid-tier model when the cheap tier would have answered correctly.
So we prototyped a shifted boundary and ran it on a ~10% subsample locally: Arena score 69.93, accuracy 71.07%, Opt.Sel jumping from ~11 to ~66. A two-point arena gain from moving one threshold.
That config will never appear on the leaderboard. It was diagnosed from RouterArena's own failure cases, which makes it exactly what their evaluation-only rule exists to prevent — a router fitted to the eval. The leaderboard entry is the untuned default, and stays that way. The honest generalization claim is much narrower: the benchmark showed us which knob matters, and we'll validate new defaults on our own traffic, not on theirs.
If you maintain a router: this is the trap. The moment a public benchmark exists, the incentive is to tune against it, and every point you gain that way is a point of overfitting you ship to your actual users.
If you're evaluating routers — including ours — ask every vendor for their RouterArena number. It costs a few dollars and a PR to get one. "We haven't submitted" is also an answer.
Links: RouterArena paper (arXiv:2510.00202) · leaderboard · our submission PR · Lynkr