{"slug": "daca-grpo-denoising-aware-credit-assignment-for-reinforcement-learning-in-models", "title": "DACA-GRPO: Denoising-Aware Credit Assignment for Reinforcement Learning in Diffusion Language Models", "summary": "Researchers affiliated with The Ohio State University and Apple proposed DACA-GRPO, a plug-and-play enhancement to GRPO-style trainers for diffusion language models that adds Denoising Progress Scores and Stratified Masking Likelihood to fix the absence of temporal credit assignment and mean-field likelihood bias. Applied on top of three GRPO base methods, DACA-GRPO improved results across seven benchmarks, with gains of up to 5.6 percentage points on math reasoning, 7.4pp on code generation, 36.3pp on constraint satisfaction, and 5.9pp on JSON schema adherence.", "body_md": "Diffusion large language models are a compelling alternative to autoregressive models, yet existing RL methods for diffusion treat all denoising steps as equally important and rely on biased, high-variance likelihood estimates. We identify two fundamental weaknesses: the absence of temporal credit assignment across the denoising trajectory, and the systematic bias of mean-field likelihood estimates used for policy optimization. To address these, we propose Denoising-Aware Credit Assignment for GRPO (DACA-GRPO), a lightweight, plug-and-play enhancement for any GRPO-style trainer. DACA-GRPO introduces two complementary mechanisms: Denoising Progress Scores, which extract per-token importance weights from intermediate predictions at no additional forward cost, and Stratified Masking Likelihood, which partitions token positions into strata so that each token is predicted with most of the sequence as context, reducing the mean-field bias. Applied on top of three GRPO base methods, DACA-GRPO achieves consistent improvements across seven benchmarks spanning mathematical reasoning, code generation, constraint satisfaction, and constrained generation, with gains of up to 5.6pp on math reasoning, 7.4pp on code generation, 36.3pp on constraint satisfaction, and 5.9pp on JSON schema adherence. \n\n- ‡ Equal contribution\n- † The Ohio State University\n- ** Work done while at Apple", "url": "https://wpnews.pro/news/daca-grpo-denoising-aware-credit-assignment-for-reinforcement-learning-in-models", "canonical_source": "https://machinelearning.apple.com/research/denoising-aware-credit-assignment", "published_at": "2026-09-16 00:00:00+00:00", "updated_at": "2026-09-16 21:55:29.986798+00:00", "lang": "en", "topics": ["large-language-models", "machine-learning", "ai-research", "natural-language-processing"], "entities": ["DACA-GRPO", "GRPO", "The Ohio State University", "Apple"], "alternates": {"html": "https://wpnews.pro/news/daca-grpo-denoising-aware-credit-assignment-for-reinforcement-learning-in-models", "markdown": "https://wpnews.pro/news/daca-grpo-denoising-aware-credit-assignment-for-reinforcement-learning-in-models.md", "text": "https://wpnews.pro/news/daca-grpo-denoising-aware-credit-assignment-for-reinforcement-learning-in-models.txt", "jsonld": "https://wpnews.pro/news/daca-grpo-denoising-aware-credit-assignment-for-reinforcement-learning-in-models.jsonld"}}