Trloom: run TRL trainers (SFT, DPO, GRPO) from one YAML config

Hi all, I built Trloom, an open-source tool built on top of TRL. It reads one YAML file and builds the matching TRL trainer, so the training block maps to SFTConfig / DPOConfig / GRPOConfig.

Here is GRPO on Qwen2.5-0.5B with LoRA and accuracy_reward:

method: grpo
model:
  model_name_or_path: Qwen/Qwen2.5-0.5B-Instruct
  dtype: bfloat16
  use_peft: true
  lora_r: 16
  lora_alpha: 32
dataset:
  path: HuggingFaceH4/Polaris-Dataset-53K
  train_split: train
reward_funcs:
  - accuracy_reward
training:
  output_dir: ./outputs/grpo-modal
  learning_rate: 1.0e-5
  per_device_train_batch_size: 1
  gradient_accumulation_steps: 4
  bf16: true
  report_to: wandb
wandb:
  enabled: true
  project: trloom-grpo
modal:
  enabled: true
  gpu: A100
  secrets: [huggingface, wandb]

pip install trloom, then trloom validate grpo.yaml and trloom run grpo.yaml (add --modal to run on a Modal GPU).

Repo: GitHub - saqlain2204/trloom: YAML-driven end-to-end fine-tuning on Hugging Face TRL — one config for model, dataset, W&B, and Modal GPUs. · GitHub

Docs: TRLoom

It’s early (v0.1.3). Happy to add recipes people ask for, and feedback on the config format is very welcome.