A new open-source system from Renmin University and Microsoft Research turns AI trial-and-error into structured, cumulative learning
Researchers at Renmin University of China’s Gaoling School of Artificial Intelligence and Microsoft Research released Arbor on June 10, 2026, an open-source framework that outperformed both OpenAI’s Codex and Anthropic’s Claude Code by more than 2.5 times in average relative held-out gains across six autonomous optimization tasks. The framework also achieved the best held-out test results on every single task evaluated.
How Arbor actually works #
Arbor uses Hypothesis-Tree Refinement (HTR), which organizes optimization work into a branching tree structure of hypotheses, experiments, evidence, and insights, where each branch builds on what came before rather than treating each attempt as a standalone experiment.
The architecture splits into two layers. A long-lived coordinator agent handles strategy, deciding which hypotheses are worth pursuing and how to sequence experiments. Short-lived executor agents then run those experiments in controlled environments. When an executor finishes its job and reports back, the coordinator absorbs the findings and refines its approach for the next round.
The benchmark numbers #
Across six autonomous optimization tasks spanning model training and data synthesis, Arbor delivered over 2.5 times the average relative held-out gain compared to both Codex and Claude Code. It also posted the best held-out test results on all evaluated tasks.
On MLE-Bench Lite, a standardized benchmark for machine learning engineering, Arbor running on GPT-5.5 achieved an Any-Medal score of 86.36%. That score measures the percentage of tasks where the system performed well enough to earn at least a bronze-level result.
The BrowseComp accuracy comparison adds another data point: Arbor scored 67.67 versus Claude Code’s 53.33.
The framework is publicly available through its GitHub repository at RUC-NLPIR/Arbor. It ships with a command-line interface runtime and skill sets designed to integrate with other coding agents.
Disclosure: This article was edited by Editorial Team. For more information on how we create and review content, see our