Frame selection is the whole game: notes on making LLMs watch video
A developer building claude-real-video, an MIT-licensed local pipeline for feeding videos to vision LLMs, found that frame selection is the critical factor in video understanding, as LLMs can afford a…