{"slug": "building-a-gpu-resident-yolo26-object-detection-pipeline-on-the-amd-radeontm-ai", "title": "Building a GPU-Resident YOLO26 Object Detection Pipeline on the AMD Radeon™ AI PRO R9700 GPU", "summary": "AMD published a guide demonstrating a GPU-resident YOLO26 object detection pipeline on the Radeon AI PRO R9700 GPU using ROCm 7.2. The pipeline keeps video frames in VRAM from decode to bounding boxes by chaining the VCN engine with rocDecode, DLPack, and MIGraphX, reducing CPU load and PCIe traffic.", "body_md": "# Building a GPU-Resident YOLO26 Object Detection Pipeline on the AMD Radeon™ AI PRO R9700 GPU[#](#building-a-gpu-resident-yolo26-object-detection-pipeline-on-the-amd-radeon-ai-pro-r9700-gpu)\n\nModern AMD GPUs include a dedicated hardware block for video processing called the Video Core Next (VCN) engine. By chaining VCN directly into machine learning frameworks, you can build an object detection pipeline where a video frame stays in VRAM from decode to the final bounding boxes. The host sees only the surviving detections.\n\nIn this guide, we build that pipeline on AMD ROCm™ software 7.2, using [Ultralytics YOLO26](https://docs.ultralytics.com/models/yolo26/) as the detector. The complete source code for this tutorial lives in the [companion GitHub repository](https://github.com/ROCm/rocm-examples/tree/amd-staging/AI/MIGraphX/gpu_resident_yolo26_pipeline).\n\nWe integrate three core components:\n\nA ROCm library that decodes compressed video directly on AMD GPUs using the on-chip VCN engines.[rocDecode](https://rocm.docs.amd.com/projects/rocDecode/en/latest/)and its Python bindings[rocPyDecode](https://github.com/ROCm/rocPyDecode):A standard tensor memory layout that allows frameworks to exchange GPU buffers without making redundant memory copies.[DLPack](https://github.com/dmlc/dlpack):The AMD graph inference engine that compiles and runs optimized deep learning models natively on AMD GPUs.[MIGraphX](https://rocm.docs.amd.com/projects/AMDMIGraphX/en/latest/):\n\n## Pipeline Architecture: rocDecode to DLPack to MIGraphX[#](#pipeline-architecture-rocdecode-to-dlpack-to-migraphx)\n\nThere are two common approaches for video pipelines: decoding on the CPU with inference on the GPU, or both on the GPU. This guide takes the second approach. Decoding runs on the VCN block, separate from the compute cores that run PyTorch and MIGraphX. Neither the CPU nor the GPU compute cores are involved in the decoding process itself.\n\nAs shown in the architecture diagram, the pipeline consists of four GPU-resident stages. A video packet is decoded into a device buffer, exposed as a DLPack tensor, preprocessed in PyTorch, and handed to MIGraphX. YOLO26’s end-to-end head emits final detections directly, so PyTorch only undoes the letterbox before returning survivors to the host.\n\nThis approach frees the host CPU for the work that runs after detection: trackers, downstream models, application logic. GPU decoding also reduces PCIe traffic, because only the compressed bitstream crosses the bus instead of every raw frame.\n\nThe `DtoH copy`\n\nshown at the end of the diagram is a demo-only artifact: the output video has to be drawn on the host.\n\n## Prerequisites and Setup[#](#prerequisites-and-setup)\n\n**Hardware:** A ROCm-supported AMD GPU with a VCN-capable video engine, such as AMD Radeon™ AI PRO R9700 or AMD Instinct™ MI300X / MI325X / MI350X / MI355X GPUs. For the codec-by-architecture support matrix, see the[rocDecode formats and architectures reference](https://rocm.docs.amd.com/projects/rocDecode/en/latest/reference/rocDecode-formats-and-architectures.html).**Host software:** A working[ROCm installation](https://rocm.docs.amd.com/projects/install-on-linux/en/latest/)and Docker Engine.**Container:** ROCm 7.2.2 with PyTorch 2.10.0 and Python 3.10 on Ubuntu 22.04 (exact tag below).\n\n**Launch the container.** Use the [rocm/pytorch Docker image](https://hub.docker.com/r/rocm/pytorch) with GPU-access flags. `HIP_VISIBLE_DEVICES=0`\n\npins the container to the first AMD GPU. On a host with multiple DRM render nodes (integrated plus discrete, or several discretes), set the index of the GPU you want to target.\n\n```\ndocker run -it --rm \\\n  --device=/dev/kfd \\\n  --device=/dev/dri \\\n  --network host \\\n  --ipc host \\\n  --group-add video \\\n  --cap-add=SYS_PTRACE \\\n  --security-opt seccomp=unconfined \\\n  -e HIP_VISIBLE_DEVICES=0 \\\n  -v $(pwd):/workspace -w /workspace \\\n  rocm/pytorch:rocm7.2.2_ubuntu22.04_py3.10_pytorch_release_2.10.0\n```\n\n**Install and verify rocDecode.** The rocDecode packages come from the **rocm** repository, already configured in the base image. Their libva backend comes from the AMD **graphics** repository, which we add below:\n\n```\necho \"deb [arch=amd64 signed-by=/etc/apt/keyrings/rocm.gpg] \\\n  https://repo.radeon.com/graphics/latest/ubuntu jammy main\" \\\n  > /etc/apt/sources.list.d/amdgpu-graphics.list\n\napt-get update -y\napt-get install -y \\\n  rocdecode rocdecode-dev rocdecode-host \\\n  rocpydecode rocjpeg rocjpeg-dev libdlpack-dev\npip install onnx rich opencv-python-headless\n\nexport LD_LIBRARY_PATH=/opt/rocm/lib:$LD_LIBRARY_PATH\nexport PYTHONPATH=/opt/rocm/lib:$PYTHONPATH\n\npython3 -c 'import pyRocVideoDecode.decoder, migraphx; print(\"Libraries loaded successfully!\")'\n```\n\n### Compile the model[#](#compile-the-model)\n\nMIGraphX runs the detector from an ONNX graph. Every variant of the Ultralytics [YOLO26 family](https://docs.ultralytics.com/models/yolo26/) shares the same end-to-end head, so the pipeline works with any of them. That head emits a `[1, 300, 6]`\n\ntensor (`[x1, y1, x2, y2, conf, class_id]`\n\nper row) and skips NMS.\n\nUltralytics provides the tooling to export the model to ONNX format; see their [export documentation](https://docs.ultralytics.com/modes/export/) for details. MIGraphX then compiles that ONNX into a GPU-tuned `.mxr`\n\nbinary:\n\n``` python\nimport migraphx\n\nmodel = migraphx.parse_onnx(\"yolo26s.onnx\")\n\n# FP16 quantization speeds up model inference without accuracy drop\nmigraphx.quantize_fp16(model)\n\n# offload_copy=False exposes the output as a named parameter so we can bind\n# a pre-allocated PyTorch tensor to it at inference time (see Step 3).\nmodel.compile(migraphx.get_target(\"gpu\"), offload_copy=False)\n\nmigraphx.save(model, \"model.mxr\")\n```\n\n## Step 1: GPU-native decoding with rocDecode[#](#step-1-gpu-native-decoding-with-rocdecode)\n\nThe input is an MP4 or MKV file with H.264 or H.265 video. This guide uses [ peloton_sample_ai_gen.mp4](https://github.com/ROCm/rocm-examples/blob/amd-staging/AI/MIGraphX/gpu_resident_yolo26_pipeline/data/peloton_sample_ai_gen.mp4), a 15-second 1920×1080 H.264 cycling clip shipped alongside the source code in the\n\n[companion GitHub repository](https://github.com/ROCm/rocm-examples/tree/amd-staging/AI/MIGraphX/gpu_resident_yolo26_pipeline). This sample clip is fully AI-generated and provided for demonstration purposes only; it is upscaled with FFmpeg’s lanczos filter from a 1280×704 source on an AMD Instinct™ MI350X GPU with ROCm 7.2.\n\n*Sample media generated using Qwen-Image-2512 and Wan 2.2 I2V-A14B (Apache 2.0 licensed models).*\n\nTwo rocPyDecode components turn the file into GPU-resident frames:\n\nThe\n\n**demuxer** splits the container into elementary video packets and reports the codec id, which we pass to the decoder.The\n\n**decoder** runs on the VCN engine and produces a decoded frame in VRAM for each packet. rocPyDecode can deliver the frame either as a raw NV12 surface or already converted to RGB on the GPU. We use the second path (`GetFrameRgb`\n\n) so PyTorch sees a ready-to-consume`[H, W, 3]`\n\ntensor. We also pick the*device-copied*output mode: the decoder owns each surface, keeps it in DLPack-wrappable memory, and recycles it on the next call. The lower-overhead*device-internal*mode saves one intra-VRAM copy but requires the caller to manage the surface lifetime; see the[rocDecode documentation](https://rocm.docs.amd.com/projects/rocDecode/en/latest/)for details.\n\n### Set up the demuxer and decoder[#](#set-up-the-demuxer-and-decoder)\n\nBuild the demuxer/decoder pair and pin them to the first HIP device:\n\n``` python\nfrom pyRocVideoDecode.decoder import decoder, GetRocDecCodecID\nfrom pyRocVideoDecode.demuxer import demuxer\nfrom pyRocVideoDecode.types import OUT_SURFACE_MEM_DEV_COPIED\n\ninput_path = \"peloton_sample_ai_gen.mp4\"  # H.264 / H.265 in MP4 or MKV\n\ndemux = demuxer(input_path)\ncodec_id = GetRocDecCodecID(demux.GetCodecId())\n\nviddec = decoder(\n    codec_id,\n    device_id=0,                          # first HIP device\n    mem_type=OUT_SURFACE_MEM_DEV_COPIED,  # own VRAM surface, DLPack-safe\n    b_force_zero_latency=False,           # emit frames in display order\n    crop_rect=None,                       # full frame\n    max_width=0,                          # 0 = use demuxer's width\n    max_height=0,                         # 0 = use demuxer's height\n    clk_rate=1000,                        # timestamp resolution (ms)\n)\n```\n\nWe keep display order because the demo writes the detections back to a video file. For detection-only loops where frame order does not matter, set `b_force_zero_latency=True`\n\ninstead to emit each frame as soon as it is decoded.\n\n### Decode and wrap as a PyTorch tensor[#](#decode-and-wrap-as-a-pytorch-tensor)\n\n`GetFrameRgb(packet, rgb_format)`\n\ndecodes the next packet and converts the result from NV12 to interleaved 8-bit RGB on the GPU in a single step. We use `rgb_format=3`\n\nfor RGB; `rgb_format=1`\n\nwould deliver BGR instead. The converted surface is exposed through `packet.ext_buf[0]`\n\nand we wrap it with [ from_dlpack](https://docs.pytorch.org/docs/stable/generated/torch.from_dlpack.html) as a\n\n`[H, W, 3]`\n\nPyTorch view without copying. The view is valid only until `ReleaseFrame`\n\n, so anything that must outlive the iteration has to be cloned.*Note (rocPyDecode 0.8.0 / rocm/pytorch:rocm7.2.2):* The DLPack capsule for the RGB surface advertises `strides=(W*3, 1, 0)`\n\nfor an `[H, W, 3]`\n\nshape. The helper restores the correct strides `(W*3, 3, 1)`\n\nwith [ torch.Tensor.as_strided](https://docs.pytorch.org/docs/stable/generated/torch.Tensor.as_strided.html) and clamps the height to\n\n`H - 1`\n\nto stay within declared storage. The missing bottom row has no measurable effect on detection: the preprocessing step letterbox-pads the frame to 640×640 anyway.Define the helper once, before the decode loop calls it:\n\n``` php\nimport torch\n\ndef decoded_rgb_view(packet) -> torch.Tensor:\n    raw = torch.from_dlpack(packet.ext_buf[0])\n    H, W = raw.shape[:2]\n    return raw.as_strided((H - 1, W, 3), (W * 3, 3, 1))\n```\n\nWith the helper in scope, the decode loop is a straight call sequence:\n\n```\npacket = demux.DemuxFrame()\nn_frames = viddec.DecodeFrame(packet)\n\nfor _ in range(n_frames):\n    pts = viddec.GetFrameRgb(packet, rgb_format=3)\n    if pts == -1:\n        viddec.ReleaseFrame(packet)\n        continue\n\n    rgb = decoded_rgb_view(packet)  # [H, W, 3] uint8, cuda\n\n    # ... preprocess / inference / draw ...\n\n    viddec.ReleaseFrame(packet)  # return the surface to the decoder pool\n```\n\n## Step 2: GPU-native preprocessing with PyTorch[#](#step-2-gpu-native-preprocessing-with-pytorch)\n\nWith `rgb`\n\nresident on the GPU, we run YOLO preprocessing natively in PyTorch using [torch.nn.functional](https://pytorch.org/docs/stable/nn.functional.html). Reshape, normalization, resize, and letterbox pad each allocate a new device tensor on the active PyTorch stream, the same stream MIGraphX uses in Step 3. None of these steps touch host memory.\n\n``` python\nimport torch.nn.functional as F\n\ndef preprocess_color_layout(rgb_tensor: torch.Tensor) -> torch.Tensor:\n    tensor = rgb_tensor.permute(2, 0, 1).unsqueeze(0)  # HWC -> BCHW\n    return tensor / 255.0                              # [0, 255] -> [0.0, 1.0]\n```\n\nNext, resize and letterbox-pad to the 640×640 model input with a uniform scale. The geometry helper is split out so postprocess can reuse it to undo the letterbox:\n\n``` php\ndef letterbox_geometry(h: int, w: int, target: int = 640) -> tuple:\n    scale = min(target / w, target / h)\n    # floor division matches YOLO's training-time letterbox convention\n    pad_x = (target - int(w * scale)) // 2\n    pad_y = (target - int(h * scale)) // 2\n    return scale, pad_x, pad_y\n\ndef preprocess_spatial(tensor: torch.Tensor, target: int = 640) -> torch.Tensor:\n    h, w = tensor.shape[2], tensor.shape[3]\n    scale, pad_x, pad_y = letterbox_geometry(h, w, target)\n    new_h, new_w = int(h * scale), int(w * scale)\n    tensor = F.interpolate(tensor, size=(new_h, new_w), mode=\"bilinear\", align_corners=False)\n    padding = (pad_x, target - new_w - pad_x, pad_y, target - new_h - pad_y)\n    tensor = F.pad(tensor, padding, value=114.0 / 255.0)  # YOLO letterbox gray\n    return tensor.contiguous()\n```\n\n## Step 3: GPU-native inference with MIGraphX[#](#step-3-gpu-native-inference-with-migraphx)\n\n[MIGraphX](https://rocm.docs.amd.com/projects/AMDMIGraphX/en/latest/) executes the compiled `.mxr`\n\nmodel asynchronously on the active PyTorch stream. Pointing it at PyTorch tensor addresses eliminates framework-level copies, so both input and output stay on the GPU.\n\n### Load the model and discover parameter names[#](#load-the-model-and-discover-parameter-names)\n\nBy default a MIGraphX program owns its output buffer and copies the result to host after each run. Compiling with `offload_copy=False`\n\ninstead exposes the output as a named parameter we can bind to a pre-allocated PyTorch tensor and keep on the GPU:\n\n``` python\nimport migraphx\nimport torch\n\nmodel = migraphx.load(\"model.mxr\")\n\nparam_shapes = model.get_parameter_shapes()\ninput_name = \"images\"\noutput_name = next(name for name in param_shapes if name != input_name)\n\ninput_shape = param_shapes[input_name]\noutput_shape = param_shapes[output_name]\n```\n\n### Pre-allocate the output tensor[#](#pre-allocate-the-output-tensor)\n\nAllocate the output on the GPU once and wrap its device pointer as a MIGraphX argument. Reusing the argument across frames keeps per-frame work to a single pointer bind plus one kernel dispatch:\n\n```\noutput_tensor = torch.empty_strided(\n    output_shape.lens(), output_shape.strides(),\n    dtype=torch.float32, device=\"cuda\",\n)\nmgx_output_arg = migraphx.argument_from_pointer(output_shape, output_tensor.data_ptr())\n```\n\n### Run inference per frame[#](#run-inference-per-frame)\n\nThe per-frame call binds the preprocessed input and submits the graph on the active PyTorch stream. MIGraphX takes the stream handle as `ihipStream_t`\n\n, so its kernels stay ordered with the surrounding PyTorch work.\n\n``` php\ndef run_inference(input_tensor: torch.Tensor) -> torch.Tensor:\n    curr_stream = torch.cuda.current_stream()\n    mgx_buffers = {\n        input_name: migraphx.argument_from_pointer(input_shape, input_tensor.data_ptr()),\n        output_name: mgx_output_arg,\n    }\n    model.run_async(mgx_buffers, curr_stream.cuda_stream, \"ihipStream_t\")\n    return output_tensor\n```\n\n## Step 4: GPU-native postprocessing with PyTorch[#](#step-4-gpu-native-postprocessing-with-pytorch)\n\nMIGraphX returns the 1×300×6 detection tensor in letterboxed coordinates: four box corners, confidence, and class id per row. YOLO26’s end-to-end head does the selection itself, so postprocess collapses to a confidence filter, a coordinate rescale, and a single batched copy to the host.\n\nThe unused slots in this tensor are padded with zero-confidence rows, so a single threshold removes them along with any real low-confidence detections. We use 0.25, the Ultralytics default. We apply the threshold mask to the full tensor at once: filtering each column separately would force three device-to-host syncs to size each output, instead of one for the combined tensor:\n\n``` php\ndef filter_predictions(raw: torch.Tensor, conf_thresh: float) -> torch.Tensor:\n    preds = raw[0]                           # [300, 6]\n    mask  = preds[:, 4] > conf_thresh\n    return preds[mask].clone()               # [N, 6] survivors\n```\n\nThe boxes still live in 640×640 letterboxed space, so undo the uniform scale and the letterbox padding in two batched ops, in place on the survivors tensor:\n\n``` python\ndef transform_coordinates(survivors: torch.Tensor, scale: float, pad_x: int, pad_y: int) -> torch.Tensor:\n    survivors[:, [0, 2]] = (survivors[:, [0, 2]] - pad_x) / scale\n    survivors[:, [1, 3]] = (survivors[:, [1, 3]] - pad_y) / scale\n    return survivors\n```\n\nA single `survivors.cpu().numpy()`\n\ntransfers the result to the host. If the next consumer is another GPU stage (tracker, serializer, downstream model), that final copy is unnecessary too.\n\nThe four steps now compose into a minimal per-frame loop:\n\n```\nwhile True:\n    # Step 1: Decoding: VCN demuxes and decodes the compressed bitstream on-chip\n    packet = demux.DemuxFrame()\n    for _ in range(viddec.DecodeFrame(packet)):  # one packet may yield 0-N frames\n        if viddec.GetFrameRgb(packet, rgb_format=3) == -1:  # frame not ready\n            viddec.ReleaseFrame(packet)\n            continue\n\n        # Step 2: Preprocessing (all on GPU, no host copies)\n        # Fix DLPack strides, reorder HWC -> CHW, normalize to [0, 1],\n        # record letterbox scale/padding, resize to 640x640.\n        rgb                 = decoded_rgb_view(packet)\n        chw                 = preprocess_color_layout(rgb)\n        scale, pad_x, pad_y = letterbox_geometry(chw.shape[2], chw.shape[3])\n        model_input         = preprocess_spatial(chw)\n\n        # Step 3: Inference\n        # MIGraphX runs the pre-compiled .mxr graph on the GPU,\n        # returns a [1, 300, 6] tensor (x1, y1, x2, y2, conf, class_id).\n        raw                 = run_inference(model_input)\n\n        # Step 4: Postprocessing\n        # Drop low-confidence boxes, map coords back to original frame resolution.\n        # Only surviving detections are copied to the host.\n        survivors           = filter_predictions(raw, conf_thresh=0.25)\n        survivors           = transform_coordinates(survivors, scale, pad_x, pad_y)\n        detections          = survivors.cpu().numpy()  # single batched DtoH\n        viddec.ReleaseFrame(packet)\n\n    if packet.bitstream_size <= 0:  # end-of-stream\n        break\n```\n\nThe full runnable sample is in the [companion GitHub repository](https://github.com/ROCm/rocm-examples/tree/amd-staging/AI/MIGraphX/gpu_resident_yolo26_pipeline); its `main.py`\n\nadds command-line parsing and an OpenCV CPU decoding baseline on top of the snippets in this section.\n\n### Adapting to YOLO models that need NMS[#](#adapting-to-yolo-models-that-need-nms)\n\nOlder detectors such as the YOLO11 family use one-to-many label assignment and need NMS to deduplicate predictions. Two GPU-native primitives keep that step on-device: [ torchvision.ops.batched_nms](https://docs.pytorch.org/vision/main/generated/torchvision.ops.batched_nms.html) or\n\n[. Either replaces the filter_predictions step; the rest of the loop is unchanged.](https://docs.ultralytics.com/reference/utils/ops/#ultralytics.utils.ops.non_max_suppression)\n\n`ultralytics.utils.ops.non_max_suppression`\n\n## Running and Verifying on the GPU[#](#running-and-verifying-on-the-gpu)\n\nUse `--decoder rocdecode`\n\nto route decoding through VCN; `--decoder opencv`\n\nis a CPU baseline for the verification step below.\n\n```\npython3 main.py \\\n  --model model.mxr \\\n  --input peloton_sample_ai_gen.mp4 \\\n  --decoder rocdecode \\\n  --output output.mp4\n```\n\nAfter processing, the output video contains the drawn detections:\n\nThe pipeline keeps tensors on the GPU until the final survivor copy; a stray `.cpu()`\n\nor host-side staging would break that silently. **PyTorch asserts** below fail fast if decode, preprocess, or inference drifts to the CPU.\n\n```\nassert rgb.is_cuda,         \"decoder returned a CPU tensor\"\nassert model_input.is_cuda, \"preprocess produced a CPU tensor\"\nassert raw.is_cuda,         \"MIGraphX output is not on the GPU\"\n```\n\nFor a hardware-level view, sample per-engine activity with [ amd-smi](https://rocm.docs.amd.com/projects/amdsmi/en/latest/):\n\n```\n# Print GPU 0 engine usage every second\namd-smi metric --usage -g 0 --watch 1 --watch_time 30\n```\n\nEach GPU exposes up to 4 VCN instances, so `VCN_ACTIVITY`\n\nis an array; entries beyond the physical engine count read `N/A`\n\n. The R9700 used in this guide reports one active engine and three `N/A`\n\nslots; datacenter Instinct™ SKUs expose more (see the [rocDecode codec-and-architecture matrix](https://rocm.docs.amd.com/projects/rocDecode/en/latest/reference/rocDecode-formats-and-architectures.html)). A representative output sample:\n\n```\nGPU: 0\n    USAGE:\n        GFX_ACTIVITY: 40 %\n        UMC_ACTIVITY:  3 %\n        MM_ACTIVITY:   9 %\n        VCN_ACTIVITY: [9 %, N/A, N/A, N/A]\n        ...\n```\n\nTo see how this maps to the hardware over a longer run, we can plot the engine activity side-by-side. The figure below places the two runs panel by panel, tracing each engine’s utilization across the full clip.\n\nWhen using the `rocDecode`\n\npath, the VCN multimedia engine and the GFX compute engine run concurrently. VCN handles the compressed bitstream at a low, steady utilization (mean 10% 1), visible as a continuous low fill in the chart; it is a fixed-function hardware block that decodes H.264/H.265 far faster than real-time for a single stream, so it spends most of its time waiting for the next frame. The GFX compute engine maintains a steady ~21%\n\nload executing the PyTorch preprocessing, MIGraphX inference, and the\n\n[1](#endnote)`GetFrameRgb`\n\ncolor conversion kernel.Falling back to the OpenCV baseline disables the hardware decode path entirely. The VCN engine flatlines at 0% because the host CPU now decompresses the video. The GFX compute engine still runs the neural network pipeline, but its average load falls slightly (to ~19% 1) because OpenCV delivers pre-converted RGB frames, sparing the GPU from the color conversion step.\n\nFor a timeline-level view, [ rocprofv3](https://rocm.docs.amd.com/projects/rocprofiler-sdk/en/latest/) shows per-engine VCN activity side by side with HIP kernel launches.\n\n## Summary[#](#summary)\n\nAcross this guide, we assembled a GPU-resident object detection pipeline in which rocDecode, DLPack, and MIGraphX exchange data through device pointers on ROCm 7.2. A frame is decoded on the VCN engine and stays in VRAM through preprocessing, inference, and postprocessing on a single PyTorch stream, so the host receives only the surviving detections. We verified the result with `amd-smi`\n\n: the VCN decode block and the GFX compute engine share the workload and run concurrently.\n\nThe same template adapts to other ROCm-supported AMD GPUs, from Radeon™ AI PRO workstations to Instinct™ accelerators.\n\nThe reference implementation has two intentional constraints:\n\n**Single-stream and serial by design.** The tutorial uses a`batch=1`\n\nmodel and a single Python thread that dispatches decode and inference serially. To scale up, you would decouple the engines: run decoder threads pushing DLPack tensors to a shared queue, and dedicate a PyTorch stream to pull from the queue, batch the frames, and run inference. You can compile the model with a fixed batch size (e.g.,`batch=4`\n\n) matching your camera count; dynamic shapes are only required if the number of active streams fluctuates at runtime.**Codec coverage is bound to rocDecode.** Only codecs listed in the[rocDecode codec-and-architecture matrix](https://rocm.docs.amd.com/projects/rocDecode/en/latest/reference/rocDecode-formats-and-architectures.html)route through VCN: H.264 and H.265 on most hardware, plus AV1 and VP9 on newer generations. For unsupported formats, remux the video into an MP4 or MKV container first.\n\n## Additional Resources[#](#additional-resources)\n\n[rocJPEG Documentation](https://rocm.docs.amd.com/projects/rocJPEG/en/latest/), the still-image counterpart to rocDecode for single-image pipelines, which plugs into the same DLPack → MIGraphX path. See also[rocJPEG decoding performance on AMD Instinct GPUs](https://rocm.blogs.amd.com/artificial-intelligence/rocjpeg-decoding-performance-blog/README.html).\n\n## Endnote[#](#endnote)\n\n**[1]** Measured on AMD Radeon™ AI PRO R9700, ROCm 7.2.2, `rocm/pytorch:rocm7.2.2_ubuntu22.04_py3.10_pytorch_release_2.10.0`\n\n, `yolo26s`\n\nFP16, 1920×1080 H.264 input at 16 fps, `batch=1`\n\n, 15.2 s source clip (AI-generated, upscaled from 1280×704 to 1920×1080 with FFmpeg `scale=1920:1080:flags=lanczos`\n\n). Method: `amdsmi.amdsmi_get_gpu_activity`\n\nsampled every 100 ms; results are configuration-specific and may vary.\n\n## Disclaimers[#](#disclaimers)\n\nThird-party content is licensed to you directly by the third party that owns the content and is not licensed to you by AMD. ALL LINKED THIRD-PARTY CONTENT IS PROVIDED “AS IS” WITHOUT A WARRANTY OF ANY KIND. USE OF SUCH THIRD-PARTY CONTENT IS DONE AT YOUR SOLE DISCRETION AND UNDER NO CIRCUMSTANCES WILL AMD BE LIABLE TO YOU FOR ANY THIRD-PARTY CONTENT. YOU ASSUME ALL RISK AND ARE SOLELY RESPONSIBLE FOR ANY DAMAGES THAT MAY ARISE FROM YOUR USE OF THIRD-PARTY CONTENT.\n\nUse of certain video codecs (including but not limited to H.264 / AVC, H.265 / HEVC, VP9, and AV1) may require licenses from third parties (such as MPEG LA, Via LA, the Alliance for Open Media, or other patent licensing pools or rights holders). It is the user’s responsibility to obtain any such licenses required for the user’s use of any such codecs. AMD provides no patent rights or licenses for the use of any such codecs through AMD products or technologies.", "url": "https://wpnews.pro/news/building-a-gpu-resident-yolo26-object-detection-pipeline-on-the-amd-radeontm-ai", "canonical_source": "https://rocm.blogs.amd.com/artificial-intelligence/gpu-resident-yolo26/README.html", "published_at": "2026-07-03 00:00:00+00:00", "updated_at": "2026-07-07 01:03:20.066696+00:00", "lang": "en", "topics": ["computer-vision", "machine-learning", "ai-infrastructure", "ai-tools"], "entities": ["AMD", "Radeon AI PRO R9700", "ROCm", "Ultralytics YOLO26", "rocDecode", "DLPack", "MIGraphX", "PyTorch"], "alternates": {"html": "https://wpnews.pro/news/building-a-gpu-resident-yolo26-object-detection-pipeline-on-the-amd-radeontm-ai", "markdown": "https://wpnews.pro/news/building-a-gpu-resident-yolo26-object-detection-pipeline-on-the-amd-radeontm-ai.md", "text": "https://wpnews.pro/news/building-a-gpu-resident-yolo26-object-detection-pipeline-on-the-amd-radeontm-ai.txt", "jsonld": "https://wpnews.pro/news/building-a-gpu-resident-yolo26-object-detection-pipeline-on-the-amd-radeontm-ai.jsonld"}}