R2D-RL: A RoboCup 2D Soccer Environment for Multi-Agent Reinforcement Learning

Researchers introduced R2D-RL, a reinforcement learning environment that bridges the RoboCup 2D Soccer Simulation platform with Python-based multi-agent reinforcement learning workflows. The environment supports full-field and scenario-based training with configurable opponents, discrete and hybrid action spaces, and reward shaping, providing a benchmark for multi-agent RL in robot soccer.

arXiv:2606.18786v1 Announce Type: new Abstract: Robot soccer is a challenging testbed for multi-agent reinforcement learning because it combines partial observability, cooperative and adversarial interaction, sparse rewards, and long-horizon tactical behavior. RoboCup 2D Soccer Simulation RCSS2D provides a mature robot-soccer platform, but its competition-oriented server-client architecture is difficult to use directly with modern Python-based MARL workflows. We introduce R2D-RL, a reinforcement learning environment that connects RCSS2D and HELIOS-based player clients to a Python MARL interface through shared-memory communication and cycle-level synchronization. R2D-RL supports full-field and scenario-based training with configurable opponents, Base discrete and Hybrid parameterized action spaces, action masks, expected possession value EPV -based reward shaping, and parallel execution. We provide front-goal scenarios and an 11-vs-11 full-field benchmark, together with baseline results.