Don't trust large context windows
A new analysis warns that large language model context windows degrade significantly beyond 100k tokens, making advertised sizes of 200k to 2M tokens misleading for practical use. Studies like RULER and Chroma's context …