Recursive Language Models Meet Uncertainty: The Surprising Effectiveness of Self-Reflective Program Search for Long Context

Researchers introduced Self-Reflective Program Search for Long Context (SRLM), a framework that uses uncertainty-aware self-reflection to improve long-context handling in language models. SRLM leverages self-consistency, reasoning trace length, and verbalized confidence to evaluate context-interaction programs, achieving up to 22% improvement over Recursive Language Models (RLMs) without requiring explicit recursion. The findings challenge the necessity of recursion in RLMs, showing that self-reflective program search yields consistent gains across both short and long contexts.

content type paper /research/ published July 2026 Recursive Language Models Meet Uncertainty: The Surprising Effectiveness of Self-Reflective Program Search for Long Context AuthorsKeivan Alizadeh , Parshin Shojaee , Minsik Cho, Mehrdad Farajtabar Recursive Language Models Meet Uncertainty: The Surprising Effectiveness of Self-Reflective Program Search for Long Context AuthorsKeivan Alizadeh , Parshin Shojaee , Minsik Cho, Mehrdad Farajtabar Long-context handling remains a core challenge for language models: even with extended context windows, models often fail to reliably extract, reason over, and use the information across long contexts. Recent works like Recursive Language Models RLMs have approached this challenge by agentic way of decomposing long contexts into recursive sub-queries through programmatic interaction at inference. While promising, the success of RLMs critically depends on how these trajectories of context-interaction programs are selected, which has remained unexplored. In this paper, we study this problem and introduce Self-Reflective Program Search for Long Context SRLM , a framework that augments programming-based context interaction with uncertainty-aware self-reflection. SRLM leverages three intrinsic signals: self-consistency, reasoning trace length, and verbalized confidence. These serve as complementary indicators of a model’s internal uncertainty, and the model uses them to evaluate and compare candidate context-interaction programs. Extensive experiments across diverse benchmark datasets, context lengths, and backbone models, show that SRLM consistently outperforms state-of-the-art baselines, yielding up to 22% improvement over RLMs under the same time budget. Our findings show that recursion itself is not the primary driver of performance in RLMs, and a simple self-reflective program search can match or surpass RLM without requiring self-query or explicit recursion mechanisms. We find that for context lengths within the model’s context window, RLMs with recursion often degrade performance relative to the base model, whereas SRLM yields consistent and robust gains across both short and long contexts. We also find that RLM is less effective in tasks with semantically intensive nature, where heuristic program search is insufficient and broader contextual understanding is required, while self-reflection in SRLM provides a semantic signal that better steers reasoning in these challenging long-context scenarios. Eliciting In-context Retrieval and Reasoning for Long-Context Language Models August 12, 2025 research area Knowledge Bases and Search /research/?domain=Knowledge%20Bases%20and%20Search , research area Speech and Natural Language Processing /research/?domain=Speech%20and%20Natural%20Language%20Processing conference ACL /research/?event=ACL Recent advancements in long-context language models LCLMs have the potential to transform Retrieval-Augmented Generation RAG by simplifying pipelines. With their extended context windows, LCLMs can process entire knowledge bases and directly handle retrieval and reasoning. This capability is defined as In-Context Retrieval and Reasoning ICR2 . However, existing benchmarks like LOFT often overestimate LCLM performance because they lack… CommVQ: Commutative Vector Quantization for KV Cache Compression July 11, 2025 research area Speech and Natural Language Processing /research/?domain=Speech%20and%20Natural%20Language%20Processing conference ICML /research/?event=ICML Large Language Models LLMs are increasingly used in applications requiring long context lengths, but the key-value KV cache often becomes a memory bottleneck on GPUs as con- text lengths grow. To address this, we propose Commutative Vector Quantization CommVQ to significantly reduce memory usage for long context LLM inference. First, we leverage additive quantization by introducing a lightweight encoder and codebook to compress the KV…