Open models lag closed models by 4 months

Open-weight AI models now trail proprietary state-of-the-art systems by an average of four months in performance, according to Epoch's internal capability metric. The Epoch Capability Index, which aggregates benchmark results across multiple tasks, shows a consistent lag between the release of closed models and the emergence of open models matching their capability. This gap highlights a persistent competitive disadvantage for open-weight AI development.

Epoch's work is free to use, distribute, and reproduce provided the source and authors are credited under the Creative Commons BY license. Learn more about this graph We calculate the average amount of time it takes for the best open-weight models to catch up with state-of-the-art performance according to our internal capability metric, the Epoch Capability Index https://epoch.ai/eci ECI . ECI is a composite measure that captures performance across many benchmarks. Analysis Limitations Explore this data Benchmark results featuring the performance of leading AI models on challenging tasks.