# Seeking feedback on token-block context selection for long-sequence QLoRA fine-tuning > Source: > Published: 2026-08-17 08:05:46+00:00 Hi everyone, I’m building SpiralCoreAttention, an experimental training-time context-selection prototype for open-weight LLM fine-tuning. Instead of passing the complete long sequence through every training forward/backward pass, the prototype selects token blocks from the sequence, trains on that selected context, and evaluates the resulting adapter on held-out full-context sequences. I recently ran an internal QLoRA experiment with: Qwen2.5-7B-Instruct 4-bit NF4 QLoRA one RTX PRO 6000 Blackwell GPU 8,192-token sequences 48 training steps per run three seeds 60% selected-context configuration Across the three internal runs, mean training-step speedup was 1.675× and peak VRAM was 6.036 GB lower. Held-out full-context loss did not worsen in these short runs. Important limitations: this is a local-corpus, short-duration, single-GPU research result. It is not a production claim, a task-level quality proof, distributed-training result, or evidence for full pretraining / 70B-scale training. Code, methodology, exact seed-level results, and limitations: [https://github.com/cannural2-cpu/SpiralCoreAttention](https://github.com/cannural2-cpu/SpiralCoreAttention) I would appreciate technical feedback on: Whether this is a reasonable evaluation protocol for selected-context fine-tuning Which task-level quality evaluations I should add next Whether there are established baselines or papers I should compare against Failure modes I should test before attempting a real design-partner validation Thank you.