16:10
2026-09-08
dwarkesh.com
artificial-intelligence
Pretraining progress is mostly coming from data
A study by Epoch AI finds that from 2019 to 2025, data improvements contributed 12.0x compute efficiency gains compared to 3.7x from model improvements, meaning 3.24x more gains came from data at a 1eā¦