{"slug": "you-don-t-need-long-context", "title": "You don't need long context", "summary": "A paper submitted to arXiv on 3 November 2026 reports that frontier open-weight language models lose little to no task accuracy across a broad range of long-horizon and long-input tasks when contexts are kept short through frequent compaction, even though later tokens can cost up to 7x more to generate than earlier tokens. The authors also found LLM performance does not collapse on long-horizon software engineering tasks when taking at most one action per context, and on that basis outline semantic state space models (SSSMs). The findings question the need for long-context language models.", "body_md": "# Computer Science > Artificial Intelligence\n\n  [Submitted on 3 Nov 2026]\n\n# Title:You don't need long context\n\n[View PDF](https://arxiv.org/pdf/2611.05306)\n\n[HTML (experimental)](https://arxiv.org/html/2611.05306v1)\n\nAbstract:Later tokens are costlier to generate than earlier tokens. On frontier open weight models, the disparity in cost may rise to as much as 7x. Through frequent compaction, we show across a broad range of long-horizon and long-input tasks that frontier language models sacrifice little to no task accuracy when maintaining short contexts. We further demonstrate that LLM performance does not collapse on long-horizon software engineering tasks even when taking at most one action per context, and based on that we outline the concept of semantic state space models (SSSMs). Our findings question the need for long-context language models.\n    \n\n### Additional Features\n\n### References & Citations\n\nLoading...\n\n# Bibliographic and Citation Tools\n\nBibliographic Explorer \n\n*(*[What is the Explorer?](https://info.arxiv.org/labs/showcase.html#arxiv-bibliographic-explorer))\nConnected Papers \n\n*(*[What is Connected Papers?](https://www.connectedpapers.com/about))\nLitmaps \n\n*(*[What is Litmaps?](https://www.litmaps.co/))\nscite Smart Citations \n\n*(*[What are Smart Citations?](https://www.scite.ai/))\n# Code, Data and Media Associated with this Article\n\nalphaXiv \n\n*(*[What is alphaXiv?](https://alphaxiv.org/))\nCatalyzeX Code Finder for Papers \n\n*(*[What is CatalyzeX?](https://www.catalyzex.com))\nDagsHub \n\n*(*[What is DagsHub?](https://dagshub.com/))\nGotit.pub \n\n*(*[What is GotitPub?](http://gotit.pub/faq))\nHugging Face \n\n*(*[What is Huggingface?](https://huggingface.co/huggingface))\nScienceCast \n\n*(*[What is ScienceCast?](https://sciencecast.org/welcome))\n# Demos\n\n# Recommenders and Search Tools\n\nInfluence Flower \n\n*(*[What are Influence Flowers?](https://influencemap.cmlab.dev/))\nCORE Recommender \n\n*(*[What is CORE?](https://core.ac.uk/services/recommender))\n# arXivLabs: experimental projects with community collaborators\n\narXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.\n\nBoth individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.\n\nHave an idea for a project that will add value for arXiv's community? [**Learn more about arXivLabs**](https://info.arxiv.org/labs/index.html).", "url": "https://wpnews.pro/news/you-don-t-need-long-context", "canonical_source": "https://arxiv.org/abs/2611.05306", "published_at": "2026-11-04 15:12:35+00:00", "updated_at": "2026-11-04 15:20:43.936287+00:00", "lang": "en", "topics": ["large-language-models", "artificial-intelligence", "ai-research", "machine-learning"], "entities": ["arXiv", "semantic state space models", "SSSMs"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/you-don-t-need-long-context", "markdown": "https://wpnews.pro/news/you-don-t-need-long-context.md", "text": "https://wpnews.pro/news/you-don-t-need-long-context.txt", "jsonld": "https://wpnews.pro/news/you-don-t-need-long-context.jsonld"}}