{"slug": "can-ai-agents-conduct-open-ended-ai-research", "title": "Can AI agents conduct open-ended AI research?", "summary": "A new study introducing 'shadow evaluations' found that frontier AI agents given six days and thousands of dollars of compute could complete all engineering tasks for two unpublished NeurIPS 2026 submissions without human help, but failed to make substantial progress toward answering the research questions, leading both papers to be unambiguously rejected by the original authors. The researchers identified five recurring failure modes: poor judgment about publishable research standards, uncreative responses to shortcomings, ineffective backtracking, poor resource awareness, and instruction drift.", "body_md": "# Computer Science > Artificial Intelligence\n\n[Submitted on 29 Jul 2026]\n\n# Title:Can AI agents conduct open-ended AI research? Early evidence from two case studies\n\n[View PDF](/pdf/2607.27191)\n\nAbstract:Forecasts of explosive AI progress hinge on AI agents automating AI research. But evidence on whether agents can carry out open-ended AI research is thin. Current evaluations either test agents on narrow, verifiable tasks, which excludes open-ended research, or submit AI-generated papers to blind peer review, which is overstretched, stochastic, and suffers from poor review quality. We introduce a third way to measure progress towards AI R\\&D automation. An agent takes on the central, open-ended research question of a high-quality unpublished paper, and the paper's original authors grade its output. We call these shadow evaluations. We ran shadow evaluations on two unpublished NeurIPS 2026 submissions, giving frontier agents six days and thousands of dollars of compute. The agents completed all of the engineering without human help, yet could not make substantial progress towards answering the research questions. As a result, both papers were unambiguously rejected by the authors. We identify five recurring failure modes: poor judgment about the bar for publishable research, uncreative responses to shortcomings in the research design, ineffective backtracking from dead ends, poor resource awareness, and instruction drift. A robustness check with a second model and scaffold reproduced these failures. We release the expert reviews, survey responses, agent repositories, and logs. Our results provide early evidence that today's agents can do the engineering of AI research, but struggle with critical parts of the research lifecycle.\n\n### Current browse context:\n\ncs.AI\n\n### References & Citations\n\nLoading...\n\n# Bibliographic and Citation Tools\n\nBibliographic Explorer\n\n*(*[What is the Explorer?](https://info.arxiv.org/labs/showcase.html#arxiv-bibliographic-explorer))\nConnected Papers\n\n*(*[What is Connected Papers?](https://www.connectedpapers.com/about))\nLitmaps\n\n*(*[What is Litmaps?](https://www.litmaps.co/))\nscite Smart Citations\n\n*(*[What are Smart Citations?](https://www.scite.ai/))# Code, Data and Media Associated with this Article\n\nalphaXiv\n\n*(*[What is alphaXiv?](https://alphaxiv.org/))\nCatalyzeX Code Finder for Papers\n\n*(*[What is CatalyzeX?](https://www.catalyzex.com))\nDagsHub\n\n*(*[What is DagsHub?](https://dagshub.com/))\nGotit.pub\n\n*(*[What is GotitPub?](http://gotit.pub/faq))\nHugging Face\n\n*(*[What is Huggingface?](https://huggingface.co/huggingface))\nScienceCast\n\n*(*[What is ScienceCast?](https://sciencecast.org/welcome))# Demos\n\n# Recommenders and Search Tools\n\nInfluence Flower\n\n*(*[What are Influence Flowers?](https://influencemap.cmlab.dev/))\nCORE Recommender\n\n*(*[What is CORE?](https://core.ac.uk/services/recommender))# arXivLabs: experimental projects with community collaborators\n\narXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.\n\nBoth individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.\n\nHave an idea for a project that will add value for arXiv's community? [ Learn more about arXivLabs](https://info.arxiv.org/labs/index.html).", "url": "https://wpnews.pro/news/can-ai-agents-conduct-open-ended-ai-research", "canonical_source": "https://arxiv.org/abs/2607.27191", "published_at": "2026-07-30 17:35:04+00:00", "updated_at": "2026-07-30 17:52:25.876328+00:00", "lang": "en", "topics": ["artificial-intelligence", "ai-research", "ai-agents"], "entities": ["NeurIPS 2026"], "alternates": {"html": "https://wpnews.pro/news/can-ai-agents-conduct-open-ended-ai-research", "markdown": "https://wpnews.pro/news/can-ai-agents-conduct-open-ended-ai-research.md", "text": "https://wpnews.pro/news/can-ai-agents-conduct-open-ended-ai-research.txt", "jsonld": "https://wpnews.pro/news/can-ai-agents-conduct-open-ended-ai-research.jsonld"}}