{"slug": "programbench-can-language-models-rebuild-programs-from-scratch", "title": "ProgramBench: Can Language Models Rebuild Programs from Scratch?", "summary": "Meta's Facebook Research released ProgramBench, a benchmark in which AI agents must architect and implement a complete codebase that reproduces a program's behavior given only its compiled binary and documentation. The benchmark is published as arXiv paper 2605.03546 by John Yang, Kilian Lieret, Jeffrey Ma, Parth Thakkar, Dmitrii Pedchenko, Sten Sootla, Emily McMilin, Pengcheng Yin, Rui Hou, Gabriel Synnaeve, Diyi Yang and Ofir Press, and ships as an installable Python package (uvx programbench, uv pip install programbench, or pip install programbench) with a mini-swe-agent baseline runnable via mini-extra programbench.", "body_md": "*Can Language Models Rebuild Programs From Scratch?*\n\nGiven only a compiled binary and its documentation, AI agents must architect and implement a complete codebase that reproduces the original program's behavior.\n\nWe recommend [uv](https://docs.astral.sh/uv/getting-started/installation/) for managing Python environments.\n\n```\n# Run without installing\nuvx programbench --help\n\n# Or install into a project\nuv pip install programbench\n\n# Or with pip\npip install programbench\n```\n\nFor development:\n\n```\ngit clone https://github.com/facebookresearch/programbench.git\ncd programbench\nuv sync  # installs editable + dev dependencies\n```\n\nNote\n\nFor more details, please refer to the [Usage Guide](https://github.com/facebookresearch/ProgramBench/blob/main/docs/README.md).\n\nTo run our baseline:\n\n```\nuvx --from mini-swe-agent mini-extra programbench --help\n# or\npip install mini-swe-agent && mini-extra programbench --help\n```\n\nSee the [mini-swe-agent baseline docs](https://mini-swe-agent.com/latest/usage/programbench/) for details.\n\nIf our work was useful for you, please cite it:\n\n```\n@misc{yang2026programbenchlanguagemodelsrebuild,\n    title={ProgramBench: Can Language Models Rebuild Programs From Scratch?},\n    author={John Yang and Kilian Lieret and Jeffrey Ma and Parth Thakkar and Dmitrii Pedchenko and Sten Sootla and Emily McMilin and Pengcheng Yin and Rui Hou and Gabriel Synnaeve and Diyi Yang and Ofir Press},\n    year={2026},\n    eprint={2605.03546},\n    archivePrefix={arXiv},\n    primaryClass={cs.SE},\n    url={https://arxiv.org/abs/2605.03546},\n}\n```\n\nProgramBench is licensed under the terms of the license found in [LICENSE](https://github.com/facebookresearch/ProgramBench/blob/main/LICENSE).", "url": "https://wpnews.pro/news/programbench-can-language-models-rebuild-programs-from-scratch", "canonical_source": "https://github.com/facebookresearch/programbench", "published_at": "2026-10-30 20:48:40+00:00", "updated_at": "2026-10-30 21:17:03.822170+00:00", "lang": "en", "topics": ["ai-research", "large-language-models", "ai-agents", "developer-tools", "ai-tools"], "entities": ["ProgramBench", "Meta", "Facebook Research", "John Yang", "Kilian Lieret", "mini-swe-agent", "arXiv", "Diyi Yang"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/programbench-can-language-models-rebuild-programs-from-scratch", "markdown": "https://wpnews.pro/news/programbench-can-language-models-rebuild-programs-from-scratch.md", "text": "https://wpnews.pro/news/programbench-can-language-models-rebuild-programs-from-scratch.txt", "jsonld": "https://wpnews.pro/news/programbench-can-language-models-rebuild-programs-from-scratch.jsonld"}}