# Which AI models stay consistent across repeated runs? > Source: > Published: 2026-11-05 16:12:39+00:00 By [Rafał Łabędzki, PhD](https://deepsense.ai/blog/author/rafal-labedzki/), Senior AI Engineering Manager at deepsense.ai and Assistant Professor at [SGH Warsaw School of Economics](https://www.sgh.waw.pl/knop/en/about-institute-human-capital) I have worked with AI for more than a decade, starting in the video game industry. In recent years, my work has focused on building evaluation and training environments for machine-learning models, alongside studying human-AI collaboration. ## What the EDA Benchmark measures The EDA Benchmark evaluates how language models and agentic systems handle exploratory data analysis tasks. These tasks can require a model to inspect files, clean inconsistent records, combine data sources, detect anomalies and return structured results. The benchmark includes 10 synthetic tasks across different domains. We run each model on every task five times, which gives us 50 trajectories per model. Repeated runs matter because one strong answer does not show whether the system can produce a similar result again. We report two scores. The **mean score** shows average analytical performance. The **reliability-adjusted score** also accounts for variation across runs, so a model receives a lower score when its results are less consistent. The benchmark uses [synthetic data and evaluation methods](https://deepsense.ai/blog/synthetic-data-and-evaluation-methods-for-the-next-generation-of-ai-models?utm_source=Substsack&utm_medium=Post_05_11_2026&utm_campaign=Eval) designed to test both answer quality and repeatability. ## A new group at the top In the October round, claude-opus-5-5 leads with a mean score of 0.65 and a reliability-adjusted score of 0.62. gpt-6-astra follows with 0.63 and 0.55, while gpt-6.1-sol scores 0.59 and 0.46. The mean-score gap between claude-opus-5-5 (0.65) and gpt-6-astra (0.63) is 0.02. After adjustment for run-to-run variation, the gap grows to 0.07 (0.62 vs 0.55), indicating greater relative variation in gpt-6-astra’s results. This difference matters when teams compare models for analytical work, because the mean shows average quality while the adjusted score also reflects how consistently a model performs. ## Why I look beyond the mean The October leaderboard also includes models from the previous measurement, but those models were not tested again for this update. Their scores remain in the chart as reference values, so they provide historical context rather than a direct, same-round comparison with every new model. For example, the previous score for gpt-5.6-sol was a mean of 0.48 and a reliability-adjusted score of 0.46. The new gpt-6-sol scores 0.53 and 0.43. These figures illustrate why mean and repeatability can tell different stories, although they come from different measurement rounds and should not be treated as a controlled head-to-head result. This is one reason I would avoid choosing a model from a single leaderboard number. The mean helps describe average quality, while repeated runs show how much that quality varies. The reliability-adjusted score brings those two signals together, but it still describes performance on this benchmark, not every task a model might face. ## Reading benchmark results in context A benchmark can help teams compare models on a defined set of tasks, and repeated runs can make that comparison more informative. The results still need to be read with the evaluation setup in mind: what tasks were used, how often each model ran, what the score measures and whether every model was tested in the same round. The complete [EDA Benchmark update](https://deepsense.ai/blog/eda-benchmark-leaderboard-october-28th-2026/?utm_source=Substsack&utm_medium=Post_05_11_2026_2&utm_campaign=Eval) includes the full leaderboard and scoring details. The tasks, configuration and evaluation artifacts are available in the [EDA Benchmark GitHub repository](https://github.com/deepsense-ai/eda-benchmark). The public benchmark is one example of evaluation work at deepsense.ai. We also build [custom datasets and evaluation environments](https://deepsense.ai/tech-expertise/llms-rag/custom-synthetic-datasets-for-llm-vlm-evaluation-and-training/?utm_source=Substsack&utm_medium=Post_05_11_2026_3&utm_campaign=Eval) for teams assessing LLMs, VLMs and agentic systems against specific capabilities and goals.