{"slug": "sdo-structure-aware-data-organization-for-efficient-llm-post-training", "title": "SDO: Structure-Aware Data Organization for Efficient LLM Post-Training", "summary": "Researchers propose SDO (Structure-Aware Data Organization), a plug-and-play framework that organizes mini-batch composition and sample exposure based on representation-space structure to improve efficiency in large language model post-training. SDO uses locality-aware batching via KNN neighborhood traversal within epochs and exposure-balanced scheduling across epochs to reduce redundant updates and improve convergence across SFT, DPO, and GRPO, with the largest gains in the early-to-mid training phase.", "body_md": "arXiv:2607.27273v1 Announce Type: new\nAbstract: Post-training of large language models is expensive, and existing efficiency improvements mainly focus on selecting informative samples or designing training schedules. However, data organization itself is usually treated as a static preprocessing step: embedding-based grouping methods construct fixed partitions before training and cannot adapt to the evolving sample exposure during optimization. As a result, all samples receive similar exposure despite their different optimization needs, leading to redundant updates for some samples while leaving others under-optimized. To address this problem, we propose SDO (Structure-Aware Data Organization), a plug-and-play data organization framework with an exposure-driven feedback mechanism that organizes mini-batch composition and sample exposure according to representation-space structure. SDO operates epoch by epoch on frozen external embeddings, avoiding model warm-up training overhead: within each epoch, locality-aware batching forms coherent mini-batches via KNN neighborhood traversal; across epochs, exposure-balanced scheduling records per-sample participation and reduces the sampling probability of over-exposed samples to preserve long-term coverage. Across SFT, DPO, and GRPO, SDO accelerates convergence, with the largest gains observed in the early-to-mid phase, producing more coherent gradients and more balanced accuracy across question types without permanently excluding training samples.", "url": "https://wpnews.pro/news/sdo-structure-aware-data-organization-for-efficient-llm-post-training", "canonical_source": "https://arxiv.org/abs/2607.27273", "published_at": "2026-07-31 04:00:00+00:00", "updated_at": "2026-07-31 04:34:23.995603+00:00", "lang": "en", "topics": ["large-language-models", "machine-learning", "artificial-intelligence"], "entities": ["SDO"], "alternates": {"html": "https://wpnews.pro/news/sdo-structure-aware-data-organization-for-efficient-llm-post-training", "markdown": "https://wpnews.pro/news/sdo-structure-aware-data-organization-for-efficient-llm-post-training.md", "text": "https://wpnews.pro/news/sdo-structure-aware-data-organization-for-efficient-llm-post-training.txt", "jsonld": "https://wpnews.pro/news/sdo-structure-aware-data-organization-for-efficient-llm-post-training.jsonld"}}