Multi-Turn Agentic Reinforcement Learning with Tunix and Gemma 4 on Google TPUs Google's Tunix agentic reinforcement learning pipeline raised the win rate of the Gemma 4 E2B model from 9.4% to 95% on the Plane Strike board game after 100 steps of multi-turn GRPO training on a single Cloud TPU v6e-8 host, cutting the average game from 22.44 strikes to about 16. The walkthrough, published on the Google Developers Blog, shows how to train a multi-turn tool-using LLM agent by subclassing Tunix's BaseTaskEnv for the 6x6 grid game, where the model calls a strike(cell="...") tool, with complete code in a public GitHub repository. In our previous posts - Easy FunctionGemma finetuning with Tunix on Google TPUs https://developers.googleblog.com/easy-functiongemma-finetuning-with-tunix-on-google-tpus/ and Autonomous LLM post-training with Tunix on TPUs https://developers.googleblog.com/autonomous-llm-post-training-with-tunix-on-tpus/ - we showed how Google Tunix https://github.com/google/tunix streamlines Supervised Fine-Tuning SFT and single-turn Group Relative Policy Optimization GRPO on Cloud TPUs. Real-world AI agents, however, rarely finish their job in a single turn. They have to reason over partial observations, invoke tools, observe environment feedback, and adapt their strategy across multiple steps. So in this post, we will walk through how to train a multi-turn, tool-using LLM agent using Tunix's agentic reinforcement learning pipeline on Cloud TPU v6e-8. You can find the complete code in this GitHub repository https://github.com/windmaple/planestrike-gemma4-rl . We will revisit the Plane Strike board game from an earlier TensorFlow/JAX reinforcement learning blog https://blog.tensorflow.org/2022/09/building-reinforcement-learning-agent-with-JAX-and-deploying-it-on-android-with-tensorflow-lite.html , replacing the custom MLP policy with Gemma 4 E2B in thinking mode on a single Cloud TPU v6e-8 host. Plane Strike is played on a 6x6 grid A1 to F6 where one 8-cell plane is hidden in one of 4 orientations. Each turn, the agent sees the result of its previous strike and the raw visible board . untried, X hit, O miss , thinks through candidate placements, and calls the strike cell="..." tool to strike one cell. The game ends when all 8 plane cells are hit, or after 23 strikes. Although google/gemma-4-E2B-it has decent spatial reasoning and tool calling capabilities, out of the box, it wins only 9.4% of games on held-out boards averaging 22.44 strikes . After 100 steps of agentic GRPO with Tunix, its win rate reaches 95% while cutting the average game length to ~16 strikes. We set up the multi-turn RL pipeline in four modular components: BaseTaskEnv in BaseTool and Gemma4ChatTemplateParser in ConversationAgentBase in TrajectoryCollectEngine and GRPOLearner in Let's look at the key steps for each piece. env.py In env.py https://github.com/windmaple/planestrike-gemma4-rl/blob/main/env.py , we subclass Tunix's BaseTaskEnv to implement PlaneStrikeEnv . Each environment instance receives a task dictionary with a seed that fixes the hidden plane position so all 8 rollouts in a GRPO group play the exact same board: class PlaneStrikeEnv BaseTaskEnv : """One Plane Strike game. entry "seed" fixes the hidden plane position.""" def init self, entry: dict str, Any , group id: int | None = None, pair index: int | None = None, max steps: int = MAX STEPS, kwargs, : entry = {k: unwrap v for k, v in entry.items } self.seed = int entry.get "seed", 0 self.split = str entry.get "split", "train" super . init task=dict entry , max steps=max steps, kwargs self.extra kwargs "group id" = group id self.extra kwargs "pair index" = pair index self. reset board On each turn, step impl https://github.com/windmaple/planestrike-gemma4-rl/blob/main/env.py L150-L179 evaluates the struck cell index against the hidden board, updates the visible grid, computes the step reward summed into the trajectory-level reward over the episode , and returns EnvStepResult observation, reward, done, info . For standard multi-tool agents, tunix.rl.agentic also provides ready-to-use ToolEnvironment , ToolAgent and ToolManager abstractions out of the box so we don't have to write custom message/observation plumbing unless we need custom turn control. strike tool.py In Gemma 4, tool schemas are declared in the system prompt inside <|tool declaration:...