OpenDiscoveryTrace releases 558 AI scientist trajectories across 124 tasks

OpenDiscoveryTrace released a public dataset of 558 complete AI scientific agent trajectories spanning 124 scientific tasks and seven different models, according to the arXiv paper 2609.09203. The dataset captures step-by-step reasoning to enable auditing of AI scientific methodology and diagnosis of failure modes, revealing significant differences in error profiles between models with similar success rates. Practitioners shipping AI scientists can now evaluate the entire reasoning process rather than only final output, potentially reducing costly errors.

arXiv https://arxiv.org/abs/2609.09203 OpenDiscoveryTrace releases 558 AI scientist trajectories across 124 tasks Which summary reads better? Pick one — models revealed after.Both summaries are AI-generated. 558 complete AI scientific agent trajectories are now available in a public dataset, capturing the step-by-step reasoning process of seven different models across 124 scientific tasks. This enables the auditing of AI scientific methodology and diagnosis of failure modes, revealing significant differences in error profiles between models with similar success rates. Practitioners shipping AI scientists can now evaluate not just the final output but the entire reasoning process, potentially reducing costly errors. DeepSeek V3 quota or rate limit — check usage and plan. Original headline: OpenDiscoveryTrace: Process Traces for Evaluating AI Scientist Workflows