{"slug": "c-grpo-conformal-group-relative-policy-optimization", "title": "C-GRPO: Conformal Group Relative Policy Optimization", "summary": "A new arXiv paper (2611.05591v1) proposes CGRPO, a method that replaces the fixed rollout budget of Group Relative Policy Optimization with calibrated conformal thresholds over task-specific nonconformity scores. Across six benchmarks spanning mathematics, graduate-level science, and code generation, CGRPO reduced mean training rollouts by 49-70% relative to fixed-budget GRPO while retaining strong fixed-budget accuracy, and transferred across model families with up to 82% rollout savings. The authors prove finite-sample coverage for the resulting stopping rule and show that, under monotone policy improvement, recalibration induces a non-increasing expected training budget.", "body_md": "arXiv:2611.05591v1 Announce Type: new \nAbstract: Reasoning-oriented language model training increasingly relies on multiple on-policy rollouts, but standard Group Relative Policy Optimization (GRPO) uses a fixed group size, wasting compute on easy prompts and under-exploring hard ones. Existing adaptive-budget methods alleviate this inefficiency, yet their stopping rules are heuristic and do not provide a finite-sample calibration guarantee. We propose CGRPO, which replaces the fixed rollout budget of standard GPRO with calibrated conformal thresholds over task-specific nonconformity scores, combining Adaptive Prediction Set (APS)-style calibration for closed-form reasoning, a first-success score for execution-based code, automatic miscoverage selection, and periodic re-calibration. We prove finite-sample coverage for the resulting stopping rule and show that, under monotone policy improvement, recalibration induces a non-increasing expected training budget. Across six benchmarks spanning mathematics, graduate-level science, and code generation, CGRPO reduces mean training rollouts by 49-70% relative to fixed-budget GRPO while retaining strong fixed-budget accuracy, achieves the strongest pass@1 among evaluated static and dynamic baselines on different datasets, and transfers across model families with up to 82% rollout savings.", "url": "https://wpnews.pro/news/c-grpo-conformal-group-relative-policy-optimization", "canonical_source": "https://arxiv.org/abs/2611.05591", "published_at": "2026-11-05 05:00:00+00:00", "updated_at": "2026-11-05 05:22:23.717033+00:00", "lang": "en", "topics": ["machine-learning", "large-language-models", "ai-research", "artificial-intelligence"], "entities": ["CGRPO", "Group Relative Policy Optimization", "arXiv"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/c-grpo-conformal-group-relative-policy-optimization", "markdown": "https://wpnews.pro/news/c-grpo-conformal-group-relative-policy-optimization.md", "text": "https://wpnews.pro/news/c-grpo-conformal-group-relative-policy-optimization.txt", "jsonld": "https://wpnews.pro/news/c-grpo-conformal-group-relative-policy-optimization.jsonld"}}