05:37
2026-05-27
dev.to
machine-learning
The bf16 grad accumulator that killed our SDXL LoRA training
Photoroom's SDXL LoRA fine-tuning for a product photography model silently corrupted its adapter weights over six days due to a bf16 gradient accumulation issue. The custom training loop, forked from …