{"slug": "trloom-run-trl-trainers-sft-dpo-grpo-from-one-yaml-config", "title": "Trloom: run TRL trainers (SFT, DPO, GRPO) from one YAML config", "summary": "Developer saqlain2204 released Trloom v0.1.3, an open-source tool that reads a single YAML config and builds the matching Hugging Face TRL trainer for SFT, DPO, or GRPO, mapping the `training` block to SFTConfig, DPOConfig, or GRPOConfig. Trloom is installed via `pip install trloom` and run with `trloom validate grpo.yaml` and `trloom run grpo.yaml`, with a `--modal` flag to execute on a Modal GPU such as an A100. The tool supports LoRA (lora_r 16, lora_alpha 32), reward functions like accuracy_reward, and W&B reporting, and the author is soliciting feedback on the config format and recipe requests.", "body_md": "Hi all, I built Trloom, an open-source tool built on top of TRL. It reads one YAML file and builds the matching TRL trainer, so the `training` block maps to SFTConfig / DPOConfig / GRPOConfig.\n\nHere is GRPO on Qwen2.5-0.5B with LoRA and accuracy_reward:\n\n```\nmethod: grpo\nmodel:\n  model_name_or_path: Qwen/Qwen2.5-0.5B-Instruct\n  dtype: bfloat16\n  use_peft: true\n  lora_r: 16\n  lora_alpha: 32\ndataset:\n  path: HuggingFaceH4/Polaris-Dataset-53K\n  train_split: train\nreward_funcs:\n  - accuracy_reward\ntraining:\n  output_dir: ./outputs/grpo-modal\n  learning_rate: 1.0e-5\n  per_device_train_batch_size: 1\n  gradient_accumulation_steps: 4\n  bf16: true\n  report_to: wandb\nwandb:\n  enabled: true\n  project: trloom-grpo\nmodal:\n  enabled: true\n  gpu: A100\n  secrets: [huggingface, wandb]\n```\n\n`pip install trloom`, then `trloom validate grpo.yaml` and `trloom run grpo.yaml` (add `--modal` to run on a Modal GPU).\n\nRepo: [GitHub - saqlain2204/trloom: YAML-driven end-to-end fine-tuning on Hugging Face TRL — one config for model, dataset, W&B, and Modal GPUs. · GitHub](https://github.com/saqlain2204/trloom)\n\nDocs: [TRLoom](https://saqlain2204.github.io/trloom/)\n\nIt’s early (v0.1.3). Happy to add recipes people ask for, and feedback on the config format is very welcome.", "url": "https://wpnews.pro/news/trloom-run-trl-trainers-sft-dpo-grpo-from-one-yaml-config", "canonical_source": "https://discuss.huggingface.co/t/trloom-run-trl-trainers-sft-dpo-grpo-from-one-yaml-config/190670#post_1", "published_at": "2026-10-16 13:02:05+00:00", "updated_at": "2026-10-16 13:18:45.009313+00:00", "lang": "en", "topics": ["ai-tools", "large-language-models", "developer-tools", "mlops", "ai-infrastructure"], "entities": ["Trloom", "TRL", "Hugging Face", "Qwen2.5-0.5B-Instruct", "saqlain2204", "Modal", "Weights & Biases", "HuggingFaceH4/Polaris-Dataset-53K"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/trloom-run-trl-trainers-sft-dpo-grpo-from-one-yaml-config", "markdown": "https://wpnews.pro/news/trloom-run-trl-trainers-sft-dpo-grpo-from-one-yaml-config.md", "text": "https://wpnews.pro/news/trloom-run-trl-trainers-sft-dpo-grpo-from-one-yaml-config.txt", "jsonld": "https://wpnews.pro/news/trloom-run-trl-trainers-sft-dpo-grpo-from-one-yaml-config.jsonld"}}