{"slug": "same-effective-batch-different-lora-training-time-a-small-trl-diagnostic", "title": "Same effective batch, different LoRA training time: a small TRL diagnostic", "summary": "A diagnostic experiment by TraceML maintainer found that Qwen3-1.7B LoRA training with TRL on a single T4 ran 17% faster with per-device batch size 4 and no gradient accumulation (238.2s) than with per-device batch size 1 and 4 accumulation steps (287.6s), while using 0.36 GiB more peak reserved memory. The test kept model, data, sequence length, FP16, seed, and 100 optimizer steps fixed, and the results align with Hugging Face's existing guidance on gradient accumulation.", "body_md": "```\nI wanted to check something I usually take for granted: if effective batch is same, does the per-device batch still make much difference to training time?\n\nI ran Qwen3-1.7B with TRL and LoRA on one T4. I kept the model, data, sequence length, FP16, seed and 100 optimizer steps fixed.\n\n| Per-device batch | Accumulation | Effective batch | Runtime | Step time | Peak reserved |\n|---:|---:|---:|---:|---:|---:|\n| 1 | 4 | 4 | 287.6s | 2871ms | 5.45 GiB |\n| 2 | 2 | 4 | 258.8s | 2585ms | 5.58 GiB |\n| 4 | 1 | 4 | 238.2s | 2379ms | 5.81 GiB |\n\nThe 4 × 1 run finished about 17% faster than 1 × 4, while using around 0.36 GiB more reserved memory. Most of the measured difference was in forward and backward.\n\nThis is not a new rule. The [Hugging Face docs](https://huggingface.co/docs/transformers/grad_accumulation) already recommend using gradient accumulation when a larger physical batch does not fit. But I found the size of difference on this T4 useful to see.\n\nI maintain TraceML and used it here to see where the step time was going. I also put the experiment in a Colab, no Hugging Face login needed:\n\n- Colab: https://colab.research.google.com/github/traceopt-ai/traceml/blob/main/notebooks/huggingface_trl_lora_gradient_accumulation.ipynb\n- Notebook source: https://github.com/traceopt-ai/traceml/blob/main/notebooks/huggingface_trl_lora_gradient_accumulation.ipynb\n\nIf anyone tries it on L4, A10 or A100, I would be interested to compare the numbers.\n```", "url": "https://wpnews.pro/news/same-effective-batch-different-lora-training-time-a-small-trl-diagnostic", "canonical_source": "https://discuss.huggingface.co/t/same-effective-batch-different-lora-training-time-a-small-trl-diagnostic/178803#post_1", "published_at": "2026-08-18 12:02:02+00:00", "updated_at": "2026-08-18 12:13:43.746279+00:00", "lang": "en", "topics": ["machine-learning", "developer-tools"], "entities": ["Qwen3-1.7B", "TRL", "LoRA", "T4", "Hugging Face", "TraceML", "Colab"], "alternates": {"html": "https://wpnews.pro/news/same-effective-batch-different-lora-training-time-a-small-trl-diagnostic", "markdown": "https://wpnews.pro/news/same-effective-batch-different-lora-training-time-a-small-trl-diagnostic.md", "text": "https://wpnews.pro/news/same-effective-batch-different-lora-training-time-a-small-trl-diagnostic.txt", "jsonld": "https://wpnews.pro/news/same-effective-batch-different-lora-training-time-a-small-trl-diagnostic.jsonld"}}