NVIDIA Video Codec SDK 13.1: Zero-Copy Transcode, AV1 B-Frames, and Frame-Accurate Seek

NVIDIA released Video Codec SDK 13.1, adding AV1 Hierarchical Reference Mode with up to 31 B-frames, per-macroblock decode statistics for H.264 and HEVC, frame-accurate seek, and application-allocated CUarray for zero-copy transcode. The update also introduces UHQ tuning info combined with iterative encoding and an official Docker-based development environment.

The demand for high-quality video continues to accelerate across industries, powering everything from immersive streaming experiences to remote collaboration, generative AI media tools, and large-scale content delivery. Behind these experiences is a growing need for video pipelines that are faster, more efficient, and capable of handling increasingly complex formats and workloads. NVIDIA Video Codec SDK helps developers meet that challenge by providing access to GPU-accelerated video encoding and decoding through NVIDIA dedicated hardware video engines. NVIDIA Video Codec SDK https://developer.nvidia.com/video-codec-sdk 13.1 is now available. We encourage developers to explore the latest features in their video pipelines, take advantage of the redesigned sample apps, and share feedback via the NVIDIA Developer forums https://forums.developer.nvidia.com/c/visualization-and-design/video-codec-sdk/ . What’s new in SDK 13.1 Encode features : · Hierarchical Reference Mode for AV1 with up to 31 B-frames · UHQ tuning info combined with iterative encoding Decode features : · Per-macroblock decode statistics for H.264 and HEVC · View information during MV-HEVC decode · Seek to a specific frame Transcode features : · Application-allocated CUarray as NVIDIA Video Encoder NVENC input and NVIDIA Video Decoder NVDEC output · Redesigned transcoder samples with a modular, queue-based architecture Other features : · Official Docker-based development environment Encode improvements Why AV1 hierarchical reference mode Using B-frames as references improves encode quality. Hierarchical Reference Mode improves it further by arranging B-frames in a tree-like reference structure: leaf nodes are non-reference B-frames, and the root is the middle B-frame. This structure increases NVENC’s maximum number of B-frames from 7 to 31, enabling the encoder to exploit temporal redundancy and improve overall quality. Use of this mode does not add any performance penalty, but video memory consumption goes higher. SDK 13.1 adds Hierarchical Reference Mode for AV1, supporting 1, 3, 7, 15, and 31 B-frames; H.264 and HEVC will follow in future driver versions. The mode is most effective at seven or more B-frames and has minimal performance impact. For configuration details, see the NVENC Programming Guide https://docs.nvidia.com/video-technologies/video-codec-sdk/13.1/nvenc-video-encoder-api-prog-guide/ . Figure 2, below, shows bitrate savings when encoding 15 B-frames in Constant Quality CQ mode, compared with NVENC’s High Quality HQ tuning in preset p7. Figure 3, below, shows bitrate savings when encoding 15 B-frames in Variable Bitrate VBR . UHQ tuning info with iterative encoding Iterative encoding introduced in Video Codec SDK 12.1 https://developer.nvidia.com/downloads/designworks/video-codec-sdk/secure/12.1/video codec sdk 12.1.14.zip freezes the encoder’s automatic state advancement and lets the user re-encode the same frame with different parameters. NVENC tracks each iteration’s state, and can stop and commit to any one of them. UHQ tuning info introduced in Video Codec SDK 12.2 https://developer.nvidia.com/designworks/video-codec-sdk/secure/12.2/video codec sdk 12.2.72.zip combines lookahead level and temporal filtering to deliver the best quality-vs-performance tradeoffs in latency-tolerant encoding. Temporal filtering reduces noise in natural video by using motion estimation to find matching patches in adjacent frames and applying them to filter the current frame, yielding average coding gains of 4–5% on natural content. The lookahead level feature analyzes future frames and uses coding tree units CTUs and other statistics to allocate bits efficiently for rate control. Four lookahead levels with different performance and quality tradeoffs are currently supported.. Version 13.1 combines UHQ tuning info with iterative encoding, so lookahead level and temporal filtering now work alongside per-iteration re-encode. Decode improvements Per-macroblock decode statistics The NVDECODE API now retrieves detailed per-macroblock decode statistics for each decoded frame of H.264 and H.265 HEVC content. For every 16×16 block, the decoder exposes the luma quantization parameter QP , the coding unit type Intra, Inter, Skip, or PCM , and up to two motion vectors forward and backward , all extracted as a natural byproduct of hardware decode with no additional CPU overhead. These statistics unlock GPU-accelerated video-analytics workflows that previously required CPU-side bitstream parsing. Motion vectors enable scene-change detection, object tracking, and shot-boundary analysis. QP values give a per-block view of encoding quality for adaptive bitrate optimization and quality monitoring. Macroblock types reveal coding structure useful for content classification and compression research. The workflow is straightforward: applications query decoder capabilities via cuvidGetDecoderCaps , enable statistics collection at decoder creation, and retrieve a GPU-resident statistics buffer with each decoded frame from cuvidMapVideoFrame . The user can copy the statistics to host memory or process them directly on the GPU with CUDA kernels for real-time pipelines. The SDK ships a ready-to-use sample, AppDec -dumpstats , that demonstrates the full flow. Frame-accurate seek AI workflows—from inference pipelines for object detection, content moderation, and video summarization, to training-data preparation that samples diverse frames across large datasets—frequently need specific frames rather than sequential decode. Video editing and non-linear post-production have the same requirement. The Video Codec SDK now provides a comprehensive seeking and random-frame-access API through the NvVideoDecoder class, making frame-accurate access as simple as array indexing while fetching only the frames you need. A GOP-aware seek architecture handles the request. For frame N, the SDK locates the nearest IDR frame before the target, seeks the demuxer there, and flushes decoder state via CUVID PKT DISCONTINUITY . From that IDR forward, only frames needed to reach N are processed: the parser flags and skips non-reference frames, and reference frames decode but bypass mapping and post-processing format conversion, scaling, cropping via cuvidMapVideoFrame through PTS-based filtering. Only frame N runs the full decode, mapping, and post-processing pipeline. The NvVideoDecoder class wraps the low-level SeekUtils engine and exposes a clean, operator-based interface. Figure 5, below, shows the internal seek flow when a frame is requested. Key capabilities: Decoder caching for playlists: NvVideoDecoder caches decoder instances by codec, bit depth, and chroma format, and reuses them via LRU eviction to avoid repeated creation overhead Non-seekable streams: Elementary streams, network streams, and pipes are auto-detected. Forward seeking works efficiently; backward seeking is supported with an automatic decoder reset Flexible frame specification: The AppDecVideoDecoder sample supports individual indices 0,10,20 , ranges with step 0:100:10 , time-based access -t 1.5,3.0 , and playlist files for batch processing Open GOP support: Streams with non-IDR I-frames seek to the nearest IDR rather than the nearest keyframe, ensuring all reference frames are available MV-HEVC stereoscopic enhancements The Video Codec SDK now offers better 3D-video support: view-ID and layer metadata from the decoder; multi-GPU encoding at high resolutions; and improved compatibility with third-party software. MV-HEVC decode updates: View-information reporting: The decoder outputs specific layer and reference info such as nuh layer id, letting applications identify and route frames by view for stereo handling Wider bitstream support: The decoder now handles MV-HEVC 3D bitstreams created by third-party encoders, so both left and right views play correctly MV-HEVC encode updates: Simplified metadata for FFmpeg: HEVC 3D display metadata now appears correctly in the bitstream when encoding via FFmpeg, making 3D-video creation easier Split Frame Encoding SFE : Multiple encoders can work on a single frame, boosting speed for high-resolution 3D and XR video that exceeds a single encoder’s capacity Redesigned transcoder pipeline The transcoding samples now favor flexibility, performance, and customization. The updated suite ships four applications: AppTransPerf. Benchmarks the maximum throughput of NVDEC and NVENC AppTrans. 1:1 transcoding with optional bit-depth conversion AppTransOneToN. 1:N transcoding with scaling AppTransZeroCopy new . 1:1 zero-copy pure-transcoding application optimized for the lowest possible latency Modular, queue-based architecture The previous implementation was fast but monolithic: the user had to understand the entire pipeline before modifying it. Version 13.1 redesigns the samples around a strictly modular, queue-based architecture that guarantees concurrency, simplifies customization, and maximizes hardware utilization. The redesign allocates a dedicated CPU thread per pipeline stage, operating as a producer-consumer system where threads communicate through explicitly sized input and output queues. The core AppTrans pipeline Figure 6, below breaks into four isolated execution contexts: a Decode Thread NVDEC handles demux and decode; a Compute Thread CUDA handles processing; an Encode Thread NVENC handles encode; and an Output Thread gathers output and muxes. Each thread isolates one step, producing a decoupled design that handles synchronization natively. Frames pass through the queues sequentially, ensuring safe data flow. Key advantages of the redesign Modularity: Copy and tweak only the pipeline steps you need. Decoupled components keep errors and exceptions isolated within their threads Performance: Full concurrency between pipeline steps is guaranteed. Separating the CPU submission threads ensures the GPU is never starved; once saturated, the hardware engines NVDEC, CUDA, NVENC run their stages in parallel on different frames Customizability: Decoupled queues give you total control. Need encoding only? Remove the decode thread and feed the compute and encode queues directly. Want ultra-low latency over throughput? Reduce queue sizes to minimize buffer wait. Have a custom AI filter? Modify the Compute Thread to dispatch your CUDA kernels in isolation, without blocking the decoder or encoder submission loops Zero-copy transcode with CUarray In a traditional transcoding pipeline as in AppTrans , the decoded frame passes through multiple internal format conversions and copies before reaching the encoder. These are inherent to the standard NvDecoder and NvEncoder APIs. The decoder output converts into an application-accessible surface; the application copies it into the encoder’s input buffer; the encoder converts that input again into its required format. AppTransZeroCopy eliminates this copy chain by having NVDEC and NVENC operate directly on the same GPU memory in a format both engines understand natively. The mechanism has four parts: Shared buffer pool allocation: At startup, the application allocates a pool of CUDA arrays CUarray using cuArray3DCreate with the CUDA ARRAY3D VIDEO ENCODE DECODE flag, which tells the CUDA driver these surfaces will be shared between both video-codec engines. Each CUarray holds one frame luma plus chroma planes in a format both NVDEC and NVENC access natively, bypassing the traditional pipeline’s intermediate conversions. Dual registration: The same CUarray register with both codecs. On the decoder side, they are provided as external output surfaces via SetExternalOutputArrays , telling NVDEC to write decoded frames directly into them. On the encoder side, they register as input resources via NVENC’s nvEncRegisterResource API with resource type NV ENC INPUT RESOURCE TYPE CUDAARRAY , letting the encoder read them directly without input conversion. Pipelined execution: Three threads—decode, encode, and output—connect through concurrent queues with token-based flow control to manage CUarray ownership between decoder and encoder. Stream-ordered synchronization: NVDEC and NVENC share the same CUDA stream, guaranteeing correct ordering between decode writes and encode reads without explicit CPU-GPU synchronization. Key advantages Lower SM utilization: The traditional pipeline uses multiple CUDA copy and conversion kernels on Streaming Multiprocessors to shuttle frames between stages. Zero-copy eliminates these intermediate kernels, freeing SM resources for CUDA preprocessing, inference, or rendering Reduced GPU memory footprint: The traditional pipeline keeps separate buffers at each stage. Zero-copy collapses these into a single shared pool, significantly reducing per-session memory consumption Higher throughput with concurrent sessions: Lower SM utilization plus a smaller memory footprint improve scalability. The GPU can sustain more concurrent transcode sessions before hitting SM saturation or memory exhaustion Figure 8, below, shows lower SM utilization in the new AppTransZeroCopy sample application compared to legacy AppTransPerf sample application. Figure 9, below, shows lower video memory bandwidth utilization in the new AppTransZeroCopy sample application compared to legacy AppTransPerf sample application. Docker development environment Setting up the Video Codec SDK traditionally requires installing the CUDA toolkit, Vulkan SDK, system libraries, and FFmpeg, then building the SDK samples—often across different host distributions and driver versions. Video Codec SDK 13.1 introduces an official Docker-based development environment that packages a consistent, pre-configured stack into a single container. The image builds from an open Dockerfile, so you can reproduce the environment locally or in the cloud and customize it via build arguments. The image is built in two stages: a builder stage compiles the SDK samples and installs the Vulkan SDK and FFmpeg; the runtime stage keeps only what is needed to run and develop. The stack pins CUDA 12.3.2, Vulkan SDK 1.4.304.1, and Ubuntu 22.04 LTS. Use the SDK ZIP build argument to point at Video Codec SDK 13.1.x.zip , and the optional FFMPEG URL argument to supply a custom FFmpeg tarball for example, an LGPL build with NVENC instead of the default BtbN LGPL build. Inside the container, pre-built samples live in /video-codec-sdk/Samples/build/ , including AppDec and AppEncCuda , ready to run with test vectors. FFmpeg LGPL is installed at /opt/ffmpeg for YUV generation, MJPEG and MPEG encoding, and inspection of encoded streams. A test-vector script produces raw YUV in multiple formats, JPEG, MPEG-1/2/4, and — with GPU access — H.264 and HEVC using AppEncCuda , so you can exercise the full pipeline. The container runs as a non-root user and includes a HEALTHCHECK for orchestrators. Prerequisites - an NVIDIA GPU with video encode/decode support - Docker with GPU support enabled - the NVIDIA Container Toolkit, and - the SDK package file Place the SDK zip for example, Video Codec SDK 13.1.x.zip in the Docker build context and build: cd ubuntu22.04 docker build -t nvidia/video-codec-sdk:13.1-ubuntu22.04 \ --build-arg SDK ZIP=Video Codec SDK 13.1.x.zip \ . Launch with GPU access. You can generate test vectors at startup or open a shell and run samples directly: Standard launch docker run --gpus all -it nvidia/video-codec-sdk:13.1-ubuntu22.04 Generate the full test-vector suite during startup ~10-15 min docker run --gpus all -it nvidia/video-codec-sdk:13.1-ubuntu22.04 \ --generate-vectors full Generate only H.264 vectors at 720p docker run --gpus all -it nvidia/video-codec-sdk:13.1-ubuntu22.04 \ --generate-vectors h264 --resolution 1280x720 Generation modes are full , h264 , hevc , vp8 , vp9 , and av1 . Inside the container, the aliases sdk-samples , test-decode , and test-encode jump to the samples directory and run quick tests with the generated vectors. Key advantages Reproducibility: The same CUDA, Vulkan, FFmpeg, and SDK versions run everywhere, reducing “works on my machine” drift Fast onboarding: Clone the repository, add the SDK zip, run docker build and docker run --gpus all ; no host-side SDK or Vulkan install required CI- and cloud-friendly: A single image drives pipelines and cloud workloads wherever the NVIDIA Container Toolkit and GPU support are available The Dockerfile and helper scripts live in the video-codec-sdk-docker repository https://gitlab.com/nvidia/container-images/video-codec-sdk . For detailed build options, environment variables, and troubleshooting, see the repository README https://gitlab.com/nvidia/container-images/video-codec-sdk/-/blob/master/README.md . Get started with Video Codec SDK 13.1 Download the SDK, try the new encode, decode, and transcode features in your pipelines, and share how they work for you. The redesigned samples make it easy to drop the new features into existing workflows or build a custom pipeline from scratch. · Download Video Codec SDK 13.1 https://developer.nvidia.com/video-codec-sdk