Rapid LLM development is outrunning the research meant to explain it, study finds

A Stanford study testing 50 widely cited claims about large language models against 20 current models found 44 percent of the findings have already expired and another 34 percent have weakened significantly, with an average claim lifespan of just 15.7 months. Claims about what models cannot do expire fastest, the researchers found. The results indicate that LLM research is being outrun by the pace of model development.