{"slug": "certifying-lower-bounds-for-risk-sensitive-reinforcement-learning-under-state", "title": "Certifying Lower Bounds for Risk-Sensitive Reinforcement Learning under Adversarial State Perturbations", "summary": "A new arXiv paper (arXiv:2609.10866v1) extends certification methods for reinforcement learning to risk-sensitive objectives, establishing lower bounds on the exponential utility of cumulative rewards under l_p-norm-bounded adversarial state perturbations for 1 ≤ p < ∞. The authors formulate the risk-sensitive certification problem as a convex optimization via a φ-divergence relaxation of the perturbation set and derive its dual for a tractable approximation of the certified lower bound. Experiments on OpenAI Gym environments and a machine replacement problem show risk-averse training generally yields higher certified lower bounds than risk-neutral training, particularly under larger perturbation budgets, while increasing risk aversion during training produces non-monotonic certification performance as overly conservative policies eventually reduce the bounds.", "body_md": "arXiv:2609.10866v1 Announce Type: new \nAbstract: Reinforcement learning (RL) agents deployed in real-world environments are often vulnerable to adversarial perturbations in state observations, creating risks in safety-critical applications. Certification methods can improve robustness against adversarial perturbations by providing lower bounds on expected cumulative rewards. Existing certification methods, however, mainly focus on risk-neutral objectives. In this paper, we extend certification methods to risk-sensitive objectives by establishing lower bounds on the exponential utility of cumulative rewards under $l_{p}$-norm-bounded state adversarial perturbations ($1\\leq p <\\infty$). By introducing a $\\phi$-divergence relaxation of the perturbation set, we formulate the risk-sensitive certification problem as a convex optimization and derive its dual to obtain a tractable approximation of the certified lower bound. We further propose an empirical method that improves certified lower bounds by selecting the training risk-aversion parameter $\\beta$ independently of the risk level used during evaluation. Experiments on both OpenAI Gym environments and a machine replacement problem show that, compared to risk-neutral training, risk-averse training generally yields policies with higher certified lower bounds, particularly under larger perturbation budgets. Moreover, under both risk-neutral and risk-averse evaluation settings, increasing risk aversion during training leads to non-monotonic certification performance, where certified lower bounds initially improve but eventually decrease due to overly conservative policies.", "url": "https://wpnews.pro/news/certifying-lower-bounds-for-risk-sensitive-reinforcement-learning-under-state", "canonical_source": "https://arxiv.org/abs/2609.10866", "published_at": "2026-09-11 04:00:00+00:00", "updated_at": "2026-09-11 04:28:49.257207+00:00", "lang": "en", "topics": ["ai-safety", "ai-research", "machine-learning", "artificial-intelligence"], "entities": ["arXiv", "OpenAI Gym"], "alternates": {"html": "https://wpnews.pro/news/certifying-lower-bounds-for-risk-sensitive-reinforcement-learning-under-state", "markdown": "https://wpnews.pro/news/certifying-lower-bounds-for-risk-sensitive-reinforcement-learning-under-state.md", "text": "https://wpnews.pro/news/certifying-lower-bounds-for-risk-sensitive-reinforcement-learning-under-state.txt", "jsonld": "https://wpnews.pro/news/certifying-lower-bounds-for-risk-sensitive-reinforcement-learning-under-state.jsonld"}}