{"slug": "phyrd-a-living-benchmark-for-reproducing-and-extending-theoretical-physics", "title": "PHYRD: A Living Benchmark for Reproducing and Extending Theoretical Physics Research", "summary": "A new arXiv paper (2610.32575v1) introduces PHYRD, a living benchmark for theoretical-physics reasoning built from 63 Reproduce and 43 Discover problems, with tasks constructed by a method the authors call FormulaForest using reference solutions and subquestion-level rubrics. Across five models, whole-problem accuracy ranged from 49.21% to 68.25% on Reproduce and from 18.60% to 74.42% on closed-book Discover; with tools, four models completed more Discover problems but conceptual errors and missing arguments persisted. The authors also report that FormulaForest data used as synthetic training data raised a Qwen-35B-A3B model's accuracy on OpenAI's FrontierScience benchmark from 61% to 72%.", "body_md": "arXiv:2610.32575v1 Announce Type: new \nAbstract: Large language models and agents have shown significant power in advanced physical research. In this paper, We introduce PHYRD, a living benchmark for theoretical-physics reasoning with 63 Reproduce and 43 Discover problems. Reproduce tests derivation of published results; Discover tests consequences of modified theoretical conditions. FormulaForest constructs the tasks with reference solutions and subquestion-level rubrics. Across five models, whole-problem accuracy ranges from 49.21% to 68.25% on Reproduce and from 18.60% to 74.42% on closed-book Discover. With tools, four models complete more Discover problems, but conceptual errors and missing arguments persist. Besides, our experimental results demonstrate the value of FormulaForest as synthetic data. On FrontierScience, a benchmark developed by OpenAI, it raises the accuracy of a Qwen-35B-A3B model from 61% to 72%.", "url": "https://wpnews.pro/news/phyrd-a-living-benchmark-for-reproducing-and-extending-theoretical-physics", "canonical_source": "https://www.machinebrief.com/news/phyrd-a-living-benchmark-for-reproducing-and-extending-theor-epl5", "published_at": "2026-10-26 04:00:00+00:00", "updated_at": "2026-10-26 05:41:59.962754+00:00", "lang": "en", "topics": ["ai-research", "large-language-models", "ai-agents", "artificial-intelligence"], "entities": ["PHYRD", "FormulaForest", "Qwen-35B-A3B", "FrontierScience", "OpenAI", "arXiv"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/phyrd-a-living-benchmark-for-reproducing-and-extending-theoretical-physics", "markdown": "https://wpnews.pro/news/phyrd-a-living-benchmark-for-reproducing-and-extending-theoretical-physics.md", "text": "https://wpnews.pro/news/phyrd-a-living-benchmark-for-reproducing-and-extending-theoretical-physics.txt", "jsonld": "https://wpnews.pro/news/phyrd-a-living-benchmark-for-reproducing-and-extending-theoretical-physics.jsonld"}}