08:05
2026-08-17
discuss.huggingface.co
machine-learning
Seeking feedback on token-block context selection for long-sequence QLoRA fine-tuning
An experimental training-time context-selection prototype, SpiralCoreAttention, achieved a mean training-step speedup of 1.675× and reduced peak VRAM by 6.036 GB during QLoRA fine-tuning of Qwen2.5-7B…