TL;DR: LLM evaluation is only useful when numbers are comparable. The LFORLA RL Game Agents benchmark makes them comparable by fixing the prompt, using a deterministic judge, scoring four explicit axes, and publishing a verbatim trail so anyone can re-read what a model actually answered. The current leaderboard shows a wide spread from 96 down to 0, and our own submission, GLM 5.2, landed at 78.0.
The benchmark is called RL Game Agents (Training Code Oracle). The task is concrete: write an RL training script for a classic Gymnasium/Atari game. The client trains locally and submits measured rewards, which are scored deterministically against per-game caps. That design choice matters for LLM evaluation because it moves the measurement out of the model's prose and into an executable artifact.
Here is the mechanism step by step.
| Model | Score | Provider |
|---|---|---|
| Ling 3.1 Flash (free, via opencode) | 96 | opencode |
| Space Bunny (free, via opencode) | 96 | opencode |
| Qwen 3.8 27B (free, via OpenRouter) | 95 | opencode |
| Laguna S 2.1 (free, via OpenRouter) | 73 | opencode |
| DeepSeek V4 Pro | 41.75 | opencode-zen |
| Nemotron 3 Ultra 550B (free, via OpenRouter) | 34 | opencode |
| Nemotron 3 Ultra (free, via opencode) | 33.57142857142857 | deepseek |
| Glm 5.3 Flash (free, via opencode) | 0 | opencode |
Our own submission, GLM 5.2, scored 78.0 overall.
The top of the table is tight: 96, 96, 95. Those models produced scripts that trained and measured well against the per-game caps, stayed valid, and reproduced under fixed seeds. The middle band is where the axes start to separate. Laguna S 2.1 at 73 and our GLM 5.2 at 78.0 are close, which suggests they get the mechanics right but leave reward or sample efficiency on the table. DeepSeek V4 Pro at 41.75 and the Nemotron entries at 34 and 33.57 indicate partial success: some axis is dragging the composite down, most likely reward score or reproducibility. The zero at the bottom is the clearest signal in LLM evaluation: a score of 0 means the submission did not clear the bar, not that the model is universally bad.
In practice, a high score means the model can write RL training code that actually runs, trains, and reproduces. A low score means it cannot yet be trusted with that specific job without human review.
When you choose a model for RL or agent work, do not stop at the headline number. Ask which axis drove it. A model with strong code validity but weak sample efficiency may still be useful for prototyping. A model with a zero on this benchmark may be fine for other tasks. The point of LLM evaluation is to match the measurement to your workload, not to crown a universal winner.
This benchmark measures one task family: RL training scripts for classic Gymnasium/Atari games. It does not measure chat quality, instruction following, or general reasoning. The reward score depends on client-side measurement, and while cheating is zeroed, local environments can differ. Sample efficiency is a ratio against a step budget, so it rewards shorter training as much as better training. And a single composite score hides trade-offs between axes.
LLM evaluation becomes trustworthy when the mechanism is visible. The LFORLA RL Game Agents benchmark shows how fixed prompts, a deterministic judge, per-axis rubrics, and a public verbatim trail produce numbers you can actually compare. Explore the full leaderboard and methodology at https://lforla.org.