{"slug": "run-rf-detr-in-nvidia-deepstream-on-jetson", "title": "Run RF-DETR in NVIDIA DeepStream on Jetson", "summary": "Roboflow published a guide showing how to run RF-DETR inside NVIDIA DeepStream on a Jetson Orin NX, using JetPack 6.2, DeepStream 7.1, and TensorRT 10.3. The post details exporting RF-DETR Nano to ONNX, building a TensorRT engine, and writing a custom parser to interpret the model's output tensors for object detection on RTSP streams.", "body_md": "This post shows how to run [RF-DETR](https://rfdetr.roboflow.com/latest/?ref=blog.roboflow.com) inside [NVIDIA DeepStream](https://developer.nvidia.com/deepstream-sdk?ref=blog.roboflow.com) on a Jetson Orin NX. We cover what DeepStream and RF-DETR are, how to build a TensorRT engine and the parser DeepStream needs to read the model's output, how to run it on RTSP stream, and how to give each class its own box color.\n\nWe used NVIDIA's [deepstream-import-vision-model](https://github.com/NVIDIA/DeepStream/tree/main/skills/deepstream-import-vision-model?ref=blog.roboflow.com) agent skill to do most of the import for us. Every number below was measured on JetPack 6.2 with DeepStream 7.1 and TensorRT 10.3.\n\n## What Is DeepStream?\n\nDeepStream is NVIDIA's video analytics SDK, built on [GStreamer](https://gstreamer.freedesktop.org/?ref=blog.roboflow.com). It gives you hardware-accelerated pipeline parts (decode, batching, TensorRT inference, on-screen display, encode) that keep frames in GPU memory the whole way through. The ones you will touch in this post:\n\n`nvstreammux`\n\n: batches N decoded streams into one GPU buffer`nvinfer`\n\n: runs a TensorRT engine and attaches detection metadata to each frame`nvdsosd`\n\n: draws boxes and labels from that metadata\n\n*Keep in mind the inference engine doesn't know or care what your model's output format is; it hands you raw tensors and leaves the interpretation to you. That interpretation lives in a small parser function you write, and it's where most of these projects succeed or fail.\n\n## What Is RF-DETR?\n\nRF-DETR ([GitHub](https://github.com/roboflow/rf-detr?ref=blog.roboflow.com)) by [Roboflow](https://roboflow.com/?ref=blog.roboflow.com) is a DETR-family detector, released under Apache 2.0. It uses 300 learned queries that look over the image features, and each query predicts one box on its own. So there is no anchor grid and no NMS step afterward. It ships in a few sizes; here I'll use RF-DETR Nano, which takes a 384x384 RGB image and puts out two tensors: 300 boxes (center x, center y, width, height, all scaled 0 to 1) and 300 x 91 class scores run through a sigmoid.\n\n## Getting the Weights and ONNX\n\nThe [rfdetr package](https://pypi.org/project/rfdetr/?ref=blog.roboflow.com) downloads pretrained COCO weights the first time you use it, and exports a ready-to-deploy ONNX file for you:\n\n``` python\npip install rfdetr\n\nfrom rfdetr import RFDETRNano\n\nmodel = RFDETRNano()          # downloads pretrained weights\nmodel.export(                 # writes an ONNX file\n    output_dir=\".\",\n    batch_size=1,             # static batch, baked into the graph\n)\n```\n\nThe export is already simplified and uses a fixed batch size by default, which matters on Jetson. Building the TensorRT engine is one command (about two minutes on the Orin NX):\n\n```\n/usr/src/tensorrt/bin/trtexec \\\n  --onnx=rfdetr-nano.onnx \\\n  --fp16 \\\n  --saveEngine=rfdetr_nano_b1.engine\n```\n\n## Telling DeepStream What the Output Tensors Mean\n\nDeepStream doesn't know the output bytes are boxes, or how they are laid out, so you give it a parser function. DeepStream calls it with the raw output tensors for each frame, and it fills in detection structs (class id, left, top, width, height, confidence).\n\nFor RF-DETR, the post-processing steps are: run the class scores through a sigmoid, pick the best class for each query, and turn the center-form box into a corner-form box in pixels. Here is the core of it:\n\n```\n// called by DeepStream with the raw output tensors of one frame\nextern \"C\" bool NvDsInferParseCustomRfdetr(...)\n{\n    // find the two output layers by name\n    // (\"dets\"/\"labels\" from the rfdetr export)\n    const float *boxes  = layer(\"dets\");    // [300, 4]  cx, cy, w, h in [0,1]\n    const float *logits = layer(\"labels\");  // [300, 91] raw class logits\n\n    for (int q = 0; q < 300; q++) {\n        // best class for this query (skip index 0, the background slot)\n        int   best  = argmax(logits + q * 91);\n        float score = sigmoid(logits[q * 91 + best]);\n        if (score < threshold) continue;\n\n        NvDsInferObjectDetectionInfo obj = {};   // zero-init, always\n        obj.classId = best;\n        obj.detectionConfidence = score;\n        // normalized center-form to pixel corner-form\n        obj.width  = boxes[q*4 + 2] * netW;\n        obj.height = boxes[q*4 + 3] * netH;\n        obj.left   = boxes[q*4 + 0] * netW - obj.width  / 2;\n        obj.top    = boxes[q*4 + 1] * netH - obj.height / 2;\n        objectList.push_back(obj);\n    }\n    return true;\n}\n```\n\nThe parser gets compiled into a small shared library, and the config file tells the system where it is, plus it holds the preprocessing settings. But if you mess any of it up, you won't get any notification that something's wrong - you'll either see no detections or boxes in totally wrong spots.\n\n```\n[property]\nonnx-file=rfdetr-nano.onnx\nmodel-engine-file=rfdetr_nano_b1.engine\nlabelfile-path=labels.txt          # 91 COCO class names, one per line\n# RF-DETR expects plain 0..1 input: rescale only, no ImageNet normalization\nnet-scale-factor=0.00392156862745098   # = 1/255\nmodel-color-format=0               # RGB\nmaintain-aspect-ratio=0            # model squash-resizes to 384x384, no letterbox\ncluster-mode=4                     # DETR family: no NMS\nnetwork-mode=2                     # FP16\ninfer-dims=3;384;384\nnum-detected-classes=91\ncustom-lib-path=libnvdsinfer_rfdetr_parser.so\nparse-bbox-func-name=NvDsInferParseCustomRfdetr\n\n[class-attrs-all]\npre-cluster-threshold=0.4          # confidence cutoff\n```\n\nCheck the model's preprocessing settings instead of assuming ImageNet normalization. This export expects plain 0 to 1 input.\n\n## Running on a Live Camera\n\nOnce the engine, parser, and config are in place, running a live camera takes a single `deepstream-app`\n\nconfig file:\n\n```\n[application]\nenable-perf-measurement=1     # prints per-stream FPS every few seconds\n\n[source0]\nenable=1\ntype=4                        # 4 = RTSP source\nuri=rtsp://user:pass@CAMERA_IP:554/stream1\nlatency=200                   # jitter buffer, ms\n\n[streammux]\nbatch-size=1                  # match the engine's batch size\nwidth=1280                    # frames are scaled to this before inference\nheight=720\nlive-source=1\n\n[primary-gie]\nenable=1\nconfig-file=config_infer_rfdetr.txt   # the nvinfer config above\n\n[sink0]\nenable=1\ntype=1                        # 1 = fakesink (headless); use type=4 to re-stream\nsync=0\ndeepstream-app -c ds_app_rtsp.txt\n```\n\nA single 720p stream runs in real time with plenty of headroom on the Orin NX. If you want to record or re-stream the annotated output, the Orin NX has a hardware H.264 encoder (NVENC), so encoding stays on the GPU (`nvv4l2h264enc`\n\n) instead of loading the CPU.\n\n## Per-Class Colors Need Code, Not Config\n\nThere is no config key for coloring boxes by class. The color lives in per-object metadata, and you change it in a probe (a callback on the pipeline) right before the on-screen display draws the box. That means writing a small app of your own instead of using the stock binary. The [Python bindings](https://github.com/NVIDIA-AI-IOT/deepstream_python_apps?ref=blog.roboflow.com) install as a wheel, with no root needed, and the whole thing is about 20 lines:\n\n``` python\nimport colorsys, pyds\nfrom gi.repository import Gst\n\n# 91 visually distinct hues via golden-ratio spacing\nPALETTE = [colorsys.hsv_to_rgb((i * 0.618) % 1.0, 0.85, 1.0) for i in range(91)]\n\ndef osd_probe(pad, info, _):\n    batch = pyds.gst_buffer_get_nvds_batch_meta(hash(info.get_buffer()))\n    l_frame = batch.frame_meta_list\n    while l_frame:\n        frame = pyds.NvDsFrameMeta.cast(l_frame.data)\n        l_obj = frame.obj_meta_list\n        while l_obj:\n            obj = pyds.NvDsObjectMeta.cast(l_obj.data)\n            r, g, b = PALETTE[obj.class_id % 91]\n            obj.rect_params.border_width = 3\n            obj.rect_params.border_color.set(r, g, b, 1.0)   # box color\n            obj.text_params.set_bg_clr = 1\n            obj.text_params.text_bg_clr.set(r, g, b, 0.55)   # label background\n            l_obj = l_obj.next\n        l_frame = l_frame.next\n    return Gst.PadProbeReturn.OK\n\n# attach to the on-screen-display element's input\nosd.get_static_pad(\"sink\").add_probe(Gst.PadProbeType.BUFFER, osd_probe, 0)\n```\n\nAttach that to the same pipeline, built in Python, and every class gets its own steady color, with the label background matched to the box.\n\nFour streams batched through one RF-DETR Nano engine on a Jetson Orin NX, tiled 2x2 with the colored-box probe above, hardware-encoded and streamed off the device. Every detection here is the same class (car), so they share one color; the probe assigns a distinct hue per class id when the scene has more than one.\n\n## Measured Throughput\n\nNumbers from this setup: RF-DETR Nano, FP16, 384x384, on an Orin NX 16GB in MAXN mode. The TensorRT rows are trtexec on the engine alone; the DeepStream row is the full four-stream pipeline (decode, inference, on-screen display, and H.264 encode), with frames counted on the inference output.\n\n| Metric | Orin NX 16GB (Nano, FP16, 384) |\n|---|---|\n| TensorRT, batch 1 | 132 img/s (7.6 ms latency) |\n| TensorRT, batch 4 | 160 img/s (25 ms/batch) |\n| DeepStream, 4x 720p RTSP at 30 fps | 30 fps/stream sustained (120 aggregate) |\n| GPU utilization, 4-stream pipeline | ~97% |\n| Peak RAM, 4-stream pipeline | 3.6 GB |\n\nBatching from 1 to 4 lifts inference throughput about 20% (132 to 160 img/s), less than the gains you see with CNN detectors, because a detection transformer already keeps the GPU busy at batch 1. And the four live streams hold their full 30 fps with the GPU pinned near 97%: pure inference tops out around 160 img/s, and the rest of the budget goes to decoding four streams, tiling, drawing boxes, and H.264 encoding, all sharing the same GPU. So there's basically no room left to add a fifth 30 fps stream with this model on this device.\n\n## Conclusion\n\nWe've done the full path from pretrained weights to live camera inference on a Jetson: export ONNX with the `rfdetr`\n\npackage, build an FP16 TRT engine, add the parser that tells DeepStream what the tensors mean, point deepstream-app at your camera, and add a Python probe when you want per-class colors. The [DeepStream import skill](https://github.com/NVIDIA/DeepStream/tree/main/skills/deepstream-import-vision-model?ref=blog.roboflow.com) does most of this for you.\n\nRF-DETR is on [GitHub](https://github.com/roboflow/rf-detr?ref=blog.roboflow.com) under Apache 2.0. If you want to train one on your own data, the [Custom Model Training guide](https://docs.roboflow.com/guides/model-training?ref=blog.roboflow.com) covers that.\n\n**Cite this Post**\n\nUse the following entry to cite this post in your research:\n\n[Erik Kokalj](/author/erik/). (Jul 22, 2026).\nRun RF-DETR in NVIDIA DeepStream on Jetson. Roboflow Blog: https://blog.roboflow.com/rf-detr-nvidia-deepstream/", "url": "https://wpnews.pro/news/run-rf-detr-in-nvidia-deepstream-on-jetson", "canonical_source": "https://blog.roboflow.com/rf-detr-nvidia-deepstream/", "published_at": "2026-07-22 17:32:53+00:00", "updated_at": "2026-07-22 17:37:55.605446+00:00", "lang": "en", "topics": ["computer-vision", "ai-infrastructure", "ai-tools", "developer-tools"], "entities": ["Roboflow", "NVIDIA DeepStream", "Jetson Orin NX", "RF-DETR", "TensorRT", "JetPack", "GStreamer", "COCO"], "alternates": {"html": "https://wpnews.pro/news/run-rf-detr-in-nvidia-deepstream-on-jetson", "markdown": "https://wpnews.pro/news/run-rf-detr-in-nvidia-deepstream-on-jetson.md", "text": "https://wpnews.pro/news/run-rf-detr-in-nvidia-deepstream-on-jetson.txt", "jsonld": "https://wpnews.pro/news/run-rf-detr-in-nvidia-deepstream-on-jetson.jsonld"}}