{"slug": "deepo-dual-entropy-enhanced-policy-optimization-for-hallucination-in-mllms", "title": "DEEPO: Dual-Entropy Enhanced Policy Optimization for Hallucination in MLLMs", "summary": "Researchers introduced Dual-Entropy Enhanced Policy Optimization (DEEPO), a dual-stage reinforcement learning method that reduces hallucination in multimodal large language models by combining semantic-entropy-triggered expert prefixes with advantage-sign-aware Renyi gradient preconditioning. DEEPO's two branches each improved over GRPO individually, and their interaction was statistically significant on VideoMMMU, the most complex long-horizon task in the evaluation suite, at +4.0 points (95% CI [1.1, 6.9]), with additive gains elsewhere. The method reduces hallucination while preserving accuracy and training stability, according to the arXiv:2609.28570v1 paper.", "body_md": "arXiv:2609.28570v1 Announce Type: new \nAbstract: Reinforcement learning (RL) is widely used to sharpen reasoning in multimodal large language models (MLLMs), yet its effect on hallucination is uneven. We trace this to two weak points in the \\emph{correction chain} from reward to parameter update. At the rollout level, hard queries---those with high semantic entropy---frequently produce unanimously wrong sample groups, collapsing the group-relative\n  advantage to zero exactly where hallucination risk is highest. At the optimization level, confident-but-wrong tokens are gradient-invisible: a categorical policy's expected score-gradient norm vanishes as its distribution sharpens, so the predictions that most need correction receive the weakest updates. We propose Dual-Entropy Enhanced Policy Optimization (DEEPO), a dual-stage enhancement combining signal\n  variance regularization with gradient preconditioning: semantic-entropy-triggered expert prefixes inject grounded continuations on high-uncertainty queries, providing direct supervision and restoring advantage variance, while advantage-sign-aware Renyi preconditioning counteracts logit-level saturation so correction reaches confident errors in the operational confidence regime. Both branches improve over GRPO individually; their interaction is statistically significant on VideoMMMU---the most complex long-horizon task in our evaluation suite (+4.0$, 95\\% CI [1.1, 6.9])---and additive elsewhere. DEEPO reduces hallucination while preserving accuracy and training stability.", "url": "https://wpnews.pro/news/deepo-dual-entropy-enhanced-policy-optimization-for-hallucination-in-mllms", "canonical_source": "https://arxiv.org/abs/2609.28570", "published_at": "2026-09-25 04:00:00+00:00", "updated_at": "2026-09-25 04:29:20.851622+00:00", "lang": "en", "topics": ["artificial-intelligence", "machine-learning", "large-language-models", "ai-research", "natural-language-processing"], "entities": ["DEEPO", "GRPO", "VideoMMMU", "arXiv"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/deepo-dual-entropy-enhanced-policy-optimization-for-hallucination-in-mllms", "markdown": "https://wpnews.pro/news/deepo-dual-entropy-enhanced-policy-optimization-for-hallucination-in-mllms.md", "text": "https://wpnews.pro/news/deepo-dual-entropy-enhanced-policy-optimization-for-hallucination-in-mllms.txt", "jsonld": "https://wpnews.pro/news/deepo-dual-entropy-enhanced-policy-optimization-for-hallucination-in-mllms.jsonld"}}