Semi-pessimistic Reinforcement Learning paper accepted in JASA.

A new reinforcement learning method called semi-pessimistic pseudo labelling (SPL), developed by researchers including Omar Rivasplata of the University of Manchester, has been accepted for publication in the Journal of the American Statistical Association (JASA). The method enables offline reinforcement learning to effectively use both labelled and unlabelled data, addressing distributional shift and improving decision-making in real-world applications such as adaptive deep brain stimulation for Parkinson's disease. The paper provides theoretical guarantees and demonstrates effectiveness through simulations and benchmarks.

Semi-pessimistic Reinforcement Learning paper accepted in JASA. Omar Rivasplata's key paper on Offline Reinforcement Learning via semi-pessimistic pseudo labelling has been accepted in JASA, the Journal of the American Statistical Association. Many real-world decision-making problems rely on large amounts of data but only a small proportion of that data is fully labelled. This paper introduces a new reinforcement learning method that can learn effectively from both labelled with rewards and unlabelled data, making it better suited to real-world applications where collecting reward information is expensive or impractical, including healthcare. Reinforcement learning RL is a powerful paradigm for policy optimisation in sequential decision making, it is used to train an intelligent agent policy to make optimal decisions by maximising some long-term reward. While online RL learns a policy by actively interacting with an environment and collecting new data through exploration and exploitation, offline RL learns a policy from a fixed, pre-collected dataset without further interaction with the environment. A typical challenge in offline RL is distributional shift, where the learned policy may encounter scenarios not covered by the offline data. This paper tackles distributional shift and integration of labelled with rewards data and unlabelled without rewards data in offline RL. The proposed method, called semi-pessimistic pseudo labelling SPL , makes it possible to take advantage of the unlabelled data to find a better policy than would be possible only using the labelled data. This improves the reliability of decision-making by improving estimation and accounting for uncertainty while remaining computationally efficient. The authors provide theoretical guarantees for the method and demonstrate its effectiveness through simulations, standard reinforcement learning benchmarks, and an application to adaptive deep brain stimulation for Parkinson's disease. By making better use of available data, the approach has the potential to improve AI systems in healthcare and other high-stakes domains. You can find the full ‘Semi-pessimistic Reinforcement Learning’ paper here . Dr Omar Rivasplata https://research.manchester.ac.uk/en/persons/omar-rivasplata/ is a Senior Lecturer in Machine Learning at The University of Manchester, and a key academic member of the Manchester Centre for AI Fundamentals https://www.ai-fun.manchester.ac.uk/ and ELLIS Unit Manchester https://www.ellismcr.org/ . Jin Zhu1,2, Xin Zhou3, Jiaang Yao4, Gholamali Aminian5, Omar Rivasplata6, Simon Little4, Lexin Li3∗, Chengchun Shi1∗ 1London School of Economics and Political Science, 2University of Birmingham, 3University of California at Berkeley, 4University of California at San Francesco, 5The Alan Turing Institute, 6University of Manchester. Corresponding authors.