Every model card says "multimodal". Then you hand the model a real video file and discover what that means in practice: ChatGPT reads the subtitle track, Claude doesn't accept video files at all. The model narrates a video it mostly never saw.
I unpack viral videos daily for my own content work, so I couldn't route around this. I built a small tool instead.
claude-real-video
converts a video into three things an LLM can genuinely read:
--adaptive
flag handles slow deformations (a real user bug report: fixed thresholds missed squash/stretch morphs entirely).faster-whisper
is installed it runs in-process and several times faster, with automatic fallback.--text-anchors
flag force-samples frames at subtitle cues so on-screen text never falls between frames.Then you point any LLM at the output folder — Claude, GPT, Gemini, or a local model. No API of mine in the middle, everything runs on your machine.
pip install claude-real-video
crv "video.mp4" -o out
Not real-time — a 90-second video takes about 1–2 minutes all-in on an M-series Mac. Frame sampling can still miss motion between frames; the flags above patch the worst cases, both born from real GitHub issues.
It's MIT, currently at 1,731 stars with ~8k installs last month, which taught me the problem was never just mine:
https://github.com/HUANGCHIHHUNGLeo/claude-real-video