Autonomous LLM post-training with Tunix on TPUs

The autofinetune project applies autonomous research loops to LLM post-training, using Google's Tunix library, Gemma models, and Cloud TPUs orchestrated with Antigravity CLI and Gemini Flash 3.7. In a supervised fine-tuning case study, an AI agent autonomously tuned LoRA rank/alpha, optimizer, and learning rate for google/functiongemma-270m-it on the google/mobile-actions dataset; in a second case study built on Tunix's official GRPO example training Gemma 3 1B for math reasoning on GSM8K, the agent improved total reward by approximately 10% by identifying better LoRA configurations, rollout temperature, KL penalty, and system prompt. The project's code, sample runs, and program.md templates are available in the autofinetune GitHub repository.

Imagine going to sleep after writing a single Markdown specification and waking up to find that an AI agent ran dozens of LLM fine-tuning experiments overnight on your behalf - discovering optimal LoRA ranks, refining learning rate schedules, tuning batch sizes and committing each verified improvement to Git. This is no longer a fantasy. Earlier this year, the autoresearch https://github.com/karpathy/autoresearch project showcased how autonomous LLM agents can iteratively explore pre-training in a self-contained loop. Taking inspiration from this paradigm, we created autofinetune https://github.com/windmaple/autofinetune : applying autonomous research loops to LLM post-training Supervised Fine-Tuning and Reinforcement Learning via GRPO , using Google’s full AI stack— Tunix https://github.com/google/tunix , Gemma, and Cloud TPUs orchestrated with Antigravity CLI https://antigravity.google/product/antigravity-cli and Gemini Flash 3.7. In this post, we’ll explore how the autonomous research loop works for post-training and walk through a couple of real-world LLM finetuning case studies. Traditional post-training involves a repetitive, manual cycle: attn vec einsum As demonstrated in autoresearch , we can now automate this whole process with the power of AI agents: program.md run.py results.tsv . In the first experiment in autofinetune https://github.com/windmaple/autofinetune/tree/main/FunctionGemma-SFT , we took the same SFT setup in our previous blog https://developers.googleblog.com/easy-functiongemma-finetuning-with-tunix-on-google-tpus/ and extended it by creating the autoresearch loop to optimize google/functiongemma-270m-it https://huggingface.co/google/functiongemma-270m-it on the google/mobile-actions https://huggingface.co/datasets/google/mobile-actions dataset. The agent was given boundaries in program.md https://github.com/windmaple/autofinetune/blob/main/FunctionGemma-SFT/program.md : Here is a sample trajectory from sample runs/SFT results.tsv https://github.com/windmaple/autofinetune/blob/main/sample runs/SFT results.tsv demonstrating how the agent hill climbed. As you can see, the agent is able to automatically adjust LoRA rank/alpha, optimizer, learning rate, etc. to keep improving the model’s accuracy in terms of generating correct function calls. Supervised fine-tuning is only a simple test. For our second case study, we took the official GRPO example https://github.com/google/tunix/blob/main/examples/grpo gemma.ipynb from the Tunix repository which trains Gemma 3 1B for math reasoning using GSM8K; the trained model has better numerical accuracy and format accuracy in its answers and set it up for autonomous RL finetuning. Reinforcement learning is subject to hyperparameter sensitivity, instability, and longer execution times - making this task more challenging and time-consuming. Post RL metric , which is simply numerical accuracy + format accurac y you can of course use other metrics, i.e., using different weights . Below is a sample trajectory logged in sample runs/RL results.tsv https://github.com/windmaple/autofinetune/blob/main/sample runs/RL results.tsv , showing the agent’s progress. The agent was able to identify better LoRA configurations, rollout temperature, KL penalty, system prompt, etc. to improve the total reward by ~10%. We hope this project shows you the power of AI agents in the domain of LLM post-training and inspires you to think about how to leverage them to automate your LLM finetuning workflows using Tunix on TPUs. Please check out the code, sample runs, and program.md templates in the autofinetune GitHub repository https://github.com/windmaple/autofinetune , explore the Tunix library https://github.com/google/tunix , and start building your own autonomous post-training lab today