{"slug": "rims-preference-optimization-via-smoothed-multi-pair-aggregation-for-small-scale", "title": "RIMS: Preference Optimization via Smoothed Multi-pair Aggregation for Small-Scale LLM Retrieval-Augmented Generation", "summary": "Researchers propose RIMS, a three-stage preference optimization framework for small-scale language models in retrieval-augmented generation, which uses synthetic chain-of-thought preference data and a differentiable soft aggregation mechanism to improve robustness to noisy evidence. Experiments on four multi-hop QA benchmarks show consistent gains in Exact Match and F1 over state-of-the-art baselines under noisy retrieval conditions.", "body_md": "arXiv:2607.16431v1 Announce Type: new\nAbstract: Small-scale language models (SLMs) are attractive for retrieval-augmented generation (RAG) in resource-constrained settings, but their limited capacity makes them highly sensitive to noisy or spurious retrieved evidence. Existing preference-based methods such as RoseRAG select only the hardest single preference pair via hard argmin/argmax, discarding the remaining signal; others treat multiple pairs as independent binary comparisons, resulting in low data utilization. We propose RIMS, a three-stage preference optimization framework comprising (1) synthetic chain-of-thought preference data generation via rejection sampling using the target SLM itself without relying on proprietary models, (2) a differentiable soft aggregation mechanism that replaces hard selection with a smooth operator, preserving gradient signal from all preference pairs while retaining the discriminative structure of margin-aware selection, and (3) preference optimization with the smoothed objective applied to multiple alignment algorithms. We theoretically show that the smoothed approximation admits a controllable error bound and that smooth aggregation yields provably tighter gradient alignment to the oracle objective than hard selection. Experiments on four multi-hop question answering benchmarks show that our approach outperforms state-of-the-art baselines across multiple SLM backbones, achieving consistent gains in Exact Match and F1 under noisy retrieval conditions. Our implementation is available at https://github.com/tptrix29/RIMS.", "url": "https://wpnews.pro/news/rims-preference-optimization-via-smoothed-multi-pair-aggregation-for-small-scale", "canonical_source": "https://arxiv.org/abs/2607.16431", "published_at": "2026-07-21 04:00:00+00:00", "updated_at": "2026-07-21 04:22:18.374551+00:00", "lang": "en", "topics": ["artificial-intelligence", "large-language-models", "natural-language-processing", "ai-research"], "entities": ["RIMS", "RoseRAG", "arXiv"], "alternates": {"html": "https://wpnews.pro/news/rims-preference-optimization-via-smoothed-multi-pair-aggregation-for-small-scale", "markdown": "https://wpnews.pro/news/rims-preference-optimization-via-smoothed-multi-pair-aggregation-for-small-scale.md", "text": "https://wpnews.pro/news/rims-preference-optimization-via-smoothed-multi-pair-aggregation-for-small-scale.txt", "jsonld": "https://wpnews.pro/news/rims-preference-optimization-via-smoothed-multi-pair-aggregation-for-small-scale.jsonld"}}