Large Context Windows Are Not a Solved Problem
A developer argues that large context windows in AI models are not a solved problem, despite recent advances allowing millions of tokens. The 'lost in the middle' phenomenon causes models to underweight content in the mi…