Google has published a walkthrough of Tunix agentic reinforcement learning, training a tool-using Gemma 4 agent over many turns on a single Cloud TPU v6e-8 host. The test case is a board game, and the result is large: the model’s win rate on held-out boards goes from 9.4% to 95% after 100 training steps.
The guide builds on two earlier Tunix tutorials, one on FunctionGemma fine-tuning and one on autonomous post-training. Those covered supervised fine-tuning and single-turn Group Relative Policy Optimization (GRPO). This one tackles agents that must act across several steps.
The Plane Strike task
Plane Strike is played on a 6×6 grid, A1 to F6. An 8-cell plane hides in one of four orientations. Each turn the agent sees the result of its last strike plus the visible board, where a dot is untried, X a hit and O a miss. It reasons about where the plane could be and calls a strike tool on one cell.
The game ends when all 8 plane cells are hit or after 23 strikes. The setup revisits an older TensorFlow and JAX reinforcement learning example but swaps the custom neural policy for Gemma 4 E2B in thinking mode.
Results before and after training
The base google/gemma-4-E2B-it model already has decent spatial reasoning and tool calling. Even so, it wins only 9.4% of held-out games, averaging 22.44 strikes. After 100 steps of agentic GRPO, it wins 95% and finishes in about 16 strikes.
The four building blocks
The pipeline has four modular parts:
- An environment, PlaneStrikeEnv, built on Tunix’s BaseTaskEnv. A seed fixes the hidden plane so all 8 rollouts in a GRPO group play the same board. Each step scores the struck cell, updates the grid and returns a reward.
- A tool and parser. StrikeTool defines the action, and a subclass of the Gemma 4 chat template parser adds tool declarations to the system prompt and renders tool messages as in-turn responses.
- An agent, built on ConversationAgentBase, that converts board observations into messages and model output into actions.
- A trainer that combines a trajectory collection engine with GRPOLearner and an asynchronous vLLM rollout engine.
Teams with standard multi-tool agents can skip custom plumbing, since tunix.rl.agentic ships ready-made ToolEnvironment, ToolAgent and ToolManager classes.
How Gemma 4 handles tool calls
Gemma 4 declares tool schemas in the system prompt. A model turn that calls a tool ends at a native stop token, so the model pauses and waits for the tool’s response before continuing in the same assistant turn.
The agent code ignores the reasoning inside the thought channel and parses only the final answer after it. An answer counts only if it contains a single valid strike call with a real board coordinate. Anything else returns an invalid action. When the model stops at the tool-response token, the next board observation is added with the tool role so the model continues its own turn.
Why multi-turn training is harder
Single-turn tuning scores one answer. An agent that plays a game or uses tools has to carry information from step to step, read the feedback after each action and change course. Rewards arrive after a whole sequence of moves, which makes it harder to tell which decisions helped. Running 8 rollouts of the same board in each GRPO group gives the learner a fair comparison, since only the agent’s choices differ.
What this means for developers
The main lesson is that a small model can gain a lot from reinforcement learning on a narrow task with a clear reward. The code is in a public GitHub repository, so developers can adapt the environment and tool to their own tasks. Keep the caveat in mind: Plane Strike is a small, simple game, and the guide doesn’t test how well the approach carries over to larger agent workloads.
Where to start
Anyone planning to try Tunix agentic reinforcement learning needs access to a Cloud TPU v6e-8 host and the example repository. The sensible next step is to replace the game environment with a task of their own, reuse the tool and parser pattern, and watch the win-rate equivalent for their reward.
Source: developers.googleblog.com