Training a Chess Reasoning Agent

TLDR: I trained an LLM reasoning agent to play chess, with model sizes 0.8B, 2B and 9B. The agent calls tools which help it keep track of the board state, which enables good performance with a relatively small number of parameters.

The demo code is available on Github. I became excited about the power of RLVR after the Deepseek R1 paper came out. They demonstrated that through the use of “thinking” tokens, an LLM could solve verifiable tasks like math problems. Additionally, they described the necessity of bootstrapping the thinking process with supervised examples, which prevents the model from generating a nonsense chain of thought.