SlopCodeBench: Benchmarking How Coding Agents Degrade over Long-Horizon Tasks

A new arXiv paper, SlopCodeBench (submitted 25 March 2026, revised 7 May 2026), benchmarks 15 coding agents on 36 problems and 196 checkpoints where agents repeatedly extend their own solutions, and finds that no agent fully solves any problem end-to-end, with the best agent passing only 14.8% of checkpoints. Structural erosion rose in 77% of trajectories and verbosity in 75.5%, and agent code was 2.3x more verbose and 2.0x more eroded than 473 open-source Python repositories. Explicit quality guidance cut initial verbosity and erosion by up to a third but did not change degradation rates, showing agents pass checkpoints while producing code that bloats and erodes with each turn.

Computer Science Software Engineering Submitted on 25 Mar 2026 v1 https://arxiv.org/abs/2603.24755v1 , last revised 7 May 2026 this version, v2 Title:SlopCodeBench: Benchmarking How Coding Agents Degrade Over Long-Horizon Iterative Tasks View PDF https://arxiv.org/pdf/2603.24755 HTML experimental https://arxiv.org/html/2603.24755v2 Abstract:Software development is iterative, yet agentic coding benchmarks hide design issues through their single-shot setup. Recent iterative benchmarks attempt to remedy this but heavily constrain an agent's design decision space, making it impossible to faithfully measure how their decisions shape future extensions. We introduce SlopCodeBench, a benchmark of 36 problems and 196 checkpoints where agents repeatedly extend their own solutions. Unlike prior iterative benchmarks, our evolving specifications demand architectural decisions but leave internal structure to the agent. We measure two forms of degradation: structural erosion concentrated complexity and verbosity redundant code . Evaluating 15 coding agents across open and closed models, we find that no agent fully solves any problem end-to-end, and the best agent passes 14.8% of checkpoints. Quality degrades across checkpoints, with structural erosion rising in 77% of trajectories and verbosity in 75.5%. Compared to 473 open-source Python repositories, agent code is 2.3x more verbose and 2.0x more eroded, and the human repositories degrade less often and by smaller margins across their git histories. Explicit quality guidance reduces initial verbosity and erosion by up to a third, without affecting degradation rates. SlopCodeBench provides the first measurement of code degradation under iterative extension, revealing that agents pass checkpoints while producing code that erodes and bloats with each turn. Submission history From: Gabriel Orlanski view email https://arxiv.org/show-email/75a7df29/2603.24755 Wed, 25 Mar 2026 19:26:44 UTC 138 KB \ v1\ https://arxiv.org/abs/2603.24755v1 v2 Thu, 7 May 2026 20:39:58 UTC 219 KB Additional Features Current browse context: cs.SE References & Citations Loading... Bibliographic and Citation Tools Bibliographic Explorer What is the Explorer? https://info.arxiv.org/labs/showcase.html arxiv-bibliographic-explorer Connected Papers What is Connected Papers? https://www.connectedpapers.com/about Litmaps What is Litmaps? https://www.litmaps.co/ scite Smart Citations What are Smart Citations? https://www.scite.ai/ Code, Data and Media Associated with this Article alphaXiv What is alphaXiv? https://alphaxiv.org/ CatalyzeX Code Finder for Papers What is CatalyzeX? https://www.catalyzex.com DagsHub What is DagsHub? https://dagshub.com/ Gotit.pub What is GotitPub? http://gotit.pub/faq Hugging Face What is Huggingface? https://huggingface.co/huggingface ScienceCast What is ScienceCast? https://sciencecast.org/welcome Demos Recommenders and Search Tools Influence Flower What are Influence Flowers? https://influencemap.cmlab.dev/ CORE Recommender What is CORE? https://core.ac.uk/services/recommender arXivLabs: experimental projects with community collaborators arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website. Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them. Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs https://info.arxiv.org/labs/index.html .