Imitation Bootstrapped Multi-Agent Reinforcement Learning A paper submitted to arXiv on 13 Nov 2026 proposes Imitation Bootstrapped Multi-Agent Reinforcement Learning (IBMARL), a framework that extends Imitation Bootstrapped Reinforcement Learning to cooperative teams and uses Round-Robin Behavior Cloning (R2BC) to train imitation policies. On four cooperative multi-agent RL benchmarks, IBMARL surpasses its imitation policies 2.6-4.2x faster than demonstration-based baselines on two tasks, is the only method to surpass them on the other two, and exceeds the demonstrator itself on two of the four; with real human demonstrations it still beats imitation learning alone and does so more than twice as fast as the best baseline. Each agent receives one action proposal from its imitation policy and one from its RL policy per time step, with an ensemble of centralized critics selecting among the per-agent combinations and a gated imitation penalty regularizing each RL policy only where the critics value the imitation action more. Computer Science Artificial Intelligence Submitted on 13 Nov 2026 Title:Imitation Bootstrapped Multi-Agent Reinforcement Learning View PDF http://arxiv.org/pdf/2611.18543v1 HTML experimental https://arxiv.org/html/2611.18543v1 Abstract:Demonstrations are an effective way to accelerate single-agent reinforcement learning RL , but extending this idea to multi-agent reinforcement learning MARL is non-trivial as it is difficult and often impossible for a single person to teleoperate multiple agents simultaneously. Round-Robin Behavior Cloning R2BC addresses this by letting a single demonstrator control one agent at a time while the others execute their current policies. However, the resulting imitation policies are limited by the quality of the demonstrations. We propose Imitation Bootstrapped Multi-Agent Reinforcement Learning IBMARL , a framework that extends Imitation Bootstrapped Reinforcement Learning IBRL to cooperative teams and uses R2BC to train the imitation policies. At each time step, every agent receives one action proposal from its imitation policy and one from its RL policy, and an ensemble of centralized critics selects among the per-agent combinations of the two. A gated imitation penalty, a soft Q-filtered behavior cloning loss, regularizes each RL policy toward its imitation policy only where the critics value the imitation action more, so that after training each agent executes its RL policy alone, without the centralized critics. On four cooperative multi-agent RL benchmarks, IBMARL surpasses its imitation policies $2.6$-$4.2\times$ faster than demonstration-based baselines on two tasks, is the only method to surpass it on the other two, and exceeds the demonstrator itself on two of the four. With real human demonstrations IBMARL still surpasses imitation learning alone, and does so more than twice as fast as the best baseline. Additional Features References & Citations Loading... Bibliographic and Citation Tools Bibliographic Explorer What is the Explorer? https://info.arxiv.org/labs/showcase.html arxiv-bibliographic-explorer Connected Papers What is Connected Papers? https://www.connectedpapers.com/about Litmaps What is Litmaps? https://www.litmaps.co/ scite Smart Citations What are Smart Citations? https://www.scite.ai/ Code, Data and Media Associated with this Article alphaXiv What is alphaXiv? https://alphaxiv.org/ CatalyzeX Code Finder for Papers What is CatalyzeX? https://www.catalyzex.com DagsHub What is DagsHub? https://dagshub.com/ Gotit.pub What is GotitPub? http://gotit.pub/faq Hugging Face What is Huggingface? https://huggingface.co/huggingface ScienceCast What is ScienceCast? https://sciencecast.org/welcome Demos Recommenders and Search Tools Influence Flower What are Influence Flowers? https://influencemap.cmlab.dev/ CORE Recommender What is CORE? https://core.ac.uk/services/recommender arXivLabs: experimental projects with community collaborators arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website. Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them. Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs https://info.arxiv.org/labs/index.html .