{"slug": "when-to-communicate-belief-distributions-and-kl-divergence-for-principled-gating", "title": "When to Communicate: Belief Distributions and KL Divergence for Principled Gating in Multi-Agent RL", "summary": "A new arXiv preprint (2608.14559v1) proposes a principled gating mechanism for multi-agent reinforcement learning where agents communicate only when the KL divergence between their learned belief distributions exceeds a fixed threshold, instead of using high-variance REINFORCE gradients. On the Predator-Prey 20x20 benchmark, the method with threshold 0.5 achieves 73.84 average steps and 42% success rate versus IC3Net's 75.31 steps and 31%, a gap of 1.47 steps and 11 percentage points with tighter seed variance. On MPE simple_spread, the belief head improves mean reward by 12 points and reduces variance by 26x even when gating is inactive.", "body_md": "arXiv:2608.14559v1 Announce Type: new\nAbstract: Effective communication in multi-agent reinforcement learning requires agents to decide not only \\textit{what} to communicate, but when? Existing approaches either communicate at every timestep or learn a binary gate through REINFORCE policy gradients \\cite{singh2019}, a high-variance signal that produces unstable and uninterpretable gating behavior. I propose a principled alternative: agents communicate only when the KL divergence between their learned belief distributions exceeds a fixed threshold. Each agent maintains a belief distribution over a latent world state computed as a softmax over its LSTM hidden state, and communicates only when belief disagreement is large enough to justify information exchange. I evaluate this approach on the Predator-Prey benchmark from IC3Net \\cite{singh2019} across two environment sizes with 5 seeds each, and on MPE simple\\_spread \\cite{lowe2017}, comparing against IC3Net, CommNet, and an independent controller. On PP 10$\\times$10, IC3Net outperforms KL-belief at all thresholds. On the harder PP 20$\\times$20, a threshold ablation over $\\varepsilon \\in \\{0.1, 0.3, 0.5, 1.0\\}$ reveals an inverted U-shape: $\\varepsilon=0.5$ achieves 73.84 average steps and 42\\% success rate versus IC3Net's 75.31 steps and 31\\%, a gap of 1.47 steps and 11 percentage points with tighter seed variance. On MPE, the belief head improves mean reward by 12 points and reduces variance by 26$\\times$ even when gating is inactive, suggesting two orthogonal contributions: principled gating when beliefs can converge, and improved latent representations that benefit coordination regardless.", "url": "https://wpnews.pro/news/when-to-communicate-belief-distributions-and-kl-divergence-for-principled-gating", "canonical_source": "https://arxiv.org/abs/2608.14559", "published_at": "2026-08-18 04:00:00+00:00", "updated_at": "2026-08-18 04:13:35.269782+00:00", "lang": "en", "topics": ["artificial-intelligence", "machine-learning"], "entities": ["arXiv", "IC3Net", "CommNet", "Predator-Prey", "MPE simple_spread"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/when-to-communicate-belief-distributions-and-kl-divergence-for-principled-gating", "markdown": "https://wpnews.pro/news/when-to-communicate-belief-distributions-and-kl-divergence-for-principled-gating.md", "text": "https://wpnews.pro/news/when-to-communicate-belief-distributions-and-kl-divergence-for-principled-gating.txt", "jsonld": "https://wpnews.pro/news/when-to-communicate-belief-distributions-and-kl-divergence-for-principled-gating.jsonld"}}