# Robust Asynchronous Q-Learning under Reward and State Corruption via Batching

> Source: <https://www.machinebrief.com/news/robust-asynchronous-q-learning-under-reward-and-state-corrup-31vx>
> Published: 2026-07-24 04:00:00+00:00

arXiv:2607.20822v1 Announce Type: new
Abstract: Motivated by reinforcement learning in harsh environments, we consider the problem of learning an optimal policy subject to adversarially corrupted feedback. Specifically, at each time-step, an adversary can perturb both the reward and state observations of the learner following the Huber contamination model. To defend against such data corruption, we propose {{\texttt{BR-Async-Q}}}: a novel, epoch-based, robust \(Q\)-learning algorithm built upon two key ideas: (i) partitioning the online data stream into batches to reduce variance, and (ii) constructing robust estimates of the Bellman optimality operator using such batched data. We prove a high-probability $\ell_\infty$ error bound for {{\texttt{BR-Async-Q}}} that matches that for vanilla \(Q\)-learning, up to a small additive term that scales with the fraction of corrupted samples. To our knowledge, this provides the first robustness guarantee for asynchronous \(Q\)-learning subject to both reward and state corruption. Furthermore, when only rewards are corrupted, the dependence of our algorithm's bound on the corruption fraction is minimax optimal.