Calibrated Decision Rewards: Training Language Models with a Judge That Knows How Sure It Is Training Qwen3-1.7B with GRPO on MATH using RLCDR (Reinforcement Learning from Calibrated Decision Rewards), a reference-free reward built from a calibrated LLM decision model, raised MATH500 accuracy from 72.2% to 76.8% (+4.65 points, p < 1e-4), according to the arXiv paper 2611.25135v1. The gain nearly matched training on the answer key (76.9%), while a random reward added only +0.65 points, and the improvement grew with difficulty, from 44.8% to 51.7% on the hardest level. The calibrated reward's expected calibration error was 0.074 versus 0.141 for a softmax judge on the same 27B backbone, and without step training the model identified the first wrong step better than a dedicated 7B process reward model (ProcessBench F1 0.772 vs. 0.722, 20/20 splits). arXiv:2611.25135v1 Announce Type: cross Abstract: Reinforcement learning RL can only teach a language model what its reward can recognize. Answer checkers are exact but need a reference answer; learned reward models and LLM judges work everywhere, but their scores are not probabilities. We study RLCDR Reinforcement Learning from Calibrated Decision Rewards . An LLM converted into a decision model answers a typed question about a response "is the final answer correct?", "which step is the first error?" with a probability distribution, read in one forward pass, debiased over option orders and calibrated with about a hundred labels. Training Qwen3-1.7B with GRPO on MATH, this reference-free reward raises MATH500 accuracy from 72.2% to 76.8% +4.65 points, p < 1e-4 , matching training on the answer key 76.9% , while a random reward gains only +0.65. The gain grows with difficulty, from 44.8% to 51.7% on the hardest level. On the policy's own changing samples the reward stays calibrated: its expected calibration error is half that of a softmax judge on the same 27B backbone 0.074 vs. 0.141 , and it shows no sign of reward hacking. Without step training, the same model finds the first wrong step better than a dedicated 7B process reward model ProcessBench F1 0.772 vs. 0.722, 20/20 splits . We also map where the approach stops being enough: trained only on problems the base model mostly fails, the reference-free reward still improves the policy +1.9 MATH500 points, p = 0.03 but trails the answer key, because its probabilities become overconfident on that distribution. We state six hypotheses in advance and test each with a control designed to falsify it.