content type paperpublished July 2026 LEAD: Breaking the No-Recovery Bottleneck in Long-Horizon Reasoning
AuthorsDenys Pushkin†, Emmanuel Abbé†
LEAD: Breaking the No-Recovery Bottleneck in Long-Horizon Reasoning
AuthorsDenys Pushkin†, Emmanuel Abbé†
Long-horizon execution in Large Language Models (LLMs) remains unstable even when high-level strategies are provided. Evaluating on controlled algorithmic puzzles, we demonstrate that while decomposition is essential for stability, extreme decomposition creates a “no-recovery bottleneck”. We show that this bottleneck becomes critical due to highly non-uniform error distribution, where consistent errors on a few “hard” steps become irreversible. To address this, we propose Lookahead-Enhanced Atomic Decomposition (LEAD). By incorporating short-horizon future validation and aggregating overlapping rollouts, LEAD provides enough isolation to maintain stability while retaining enough local context to correct errors. This enables the o4-mini model to solve Checkers Jumping up to complexity n = 13, whereas extreme decomposition fails beyond n = 11.
Divide-or-Conquer? Which Part Should You Distill Your LLM?
October 25, 2024research area Speech and Natural Language Processingconference EMNLP Recent methods have demonstrated that Large Language Models (LLMs) can solve reasoning tasks better when they are encouraged to solve subtasks of the main task first. In this paper we devise a similar strategy that breaks down reasoning tasks into a problem decomposition phase and a problem solving phase and show that the strategy is able to outperform a single stage solution. Further, we hypothesize that the decomposition should be easier to…
Joint Learning of Portrait Intrinsic Decomposition and Relighting
July 28, 2021research area Computer Vision Inverse rendering is the problem of decomposing an image into its intrinsic components, i.e. albedo, normal and lighting. To solve this ill-posed problem from single image, state-of-the-art methods in shape from shading mostly resort to supervised training on all the components on either synthetic or real datasets. Here, we propose a new self-supervised training paradigm that 1) reduces the need for full supervision on the decomposition task and…