NVIDIA's GPU-accelerated frame interpolation support has been integrated into FFmpeg through a Vulkan API, as announced by the project's developers. For those unfamiliar, FFmpeg is a widely used software suite that powers much of the world's modern multimedia processing, including platforms like YouTube and many other services and applications. This tool now leverages NVIDIA GeForce RTX GPUs to predict and generate the next frame without involving the traditional graphics rendering pipeline. NVIDIA is extending this capability to video encoding, allowing users to increase the frame rate of their videos through frame generation technology. The Optical Flow Accelerator (NVOFA) in NVIDIA GPUs applies the same Optical Flow logic used in graphics rendering to videos, predicting the next frame and enhancing frame rates through AI-generated frames. NVIDIA calls this process Engine-assisted Frame-rate Up Conversion (FRUC).
FRUC is more than a conventional frame-smoothing algorithm, as it uses NVIDIA's dedicated Optical Flow Accelerator to calculate motion between consecutive source frames. NVENC provides the encoder-side frame data, while the optical flow hardware generates motion vectors that describe how objects move between frames. FFmpeg can then use that motion information to synthesize an intermediate frame, effectively increasing the video's frame rate without re-rendering the original content. This is also more efficient than software-based interpolation, which can place significantly higher computational demands on the CPU. The new Vulkan implementation gives FFmpeg direct access to the NVIDIA hardware while avoiding the proprietary software dependency that prevented the earlier 2023 patch from being merged.