{"slug": "tensorrt-was-fast-my-1080p-rtsp-pipeline-wasn-t", "title": "TensorRT Was Fast. My 1080p RTSP Pipeline Wasn't.", "summary": "A developer troubleshooting a TensorRT-based RTSP video analytics pipeline on an RTX 5070 Ti server found the application processing only 8–10 FPS on a 1920×1080 25 FPS stream despite GPU-accelerated pose inference. By switching decoding to PyNvVideoCodec/NVDEC, keeping frames CUDA-resident through GPU resize, letterbox and normalization, calling TensorRT directly via execute_async_v3(), and transferring only compact detection metadata and GPU-extracted torso ROIs to the CPU, the pipeline reached 24.64 processed FPS measured before the OCR queue.", "body_md": "I was troubleshooting low FPS in a TensorRT-based RTSP video analytics pipeline on a customer's server. The input was a 1920×1080 stream at 25 FPS, running on an NVIDIA GeForce RTX 5070 Ti. The pose model already used TensorRT, but the application was processing only **8–10 FPS**.\n\nInference was accelerated. The video pipeline wasn't.\n\nAfter changing how frames moved through the application, I measured **24.64 processed FPS** against the same 25 FPS source. That measurement was taken **before the OCR queue**; it was not a benchmark of end-to-end OCR or video delivery latency.\n\nI didn't replace the pose model. I changed the data path around it.\n\nThe application did more than pose detection. It decoded RTSP video, detected and tracked people, extracted torso regions for OCR, checked safety-related events, and generated an RTSP output stream.\n\nThe original path was roughly:\n\n```\nRTSP → OpenCV/FFmpeg → CPU frame\n     → GPU inference → CPU-side processing\n     → OCR → output\n```\n\nThe TensorRT engine could execute on the GPU, but full frames still crossed the CPU/GPU boundary too often. Optimizing inference alone wasn't going to solve that.\n\nA 1920×1080 RGB frame contains 6,220,800 bytes, or about **6.2 MB**. One full-frame transfer at 25 FPS amounts to approximately **156 MB/s of image payload** in one direction. This is a size calculation, not a measured PCIe bandwidth figure from the server.\n\nBandwidth wasn't the only concern. Intermediate buffers, format conversions, and synchronization can also add time. I needed to examine the frame's route through the application, not just the model's execution time.\n\nThe revised path was:\n\n```\nRTSP\n  ↓\nPyNvVideoCodec / NVDEC\n  ↓\nCUDA-resident frame\n  ↓\nGPU resize, letterbox, normalization\n  ↓\nDirect TensorRT inference\n  ↓\nGPU postprocessing and coordinate mapping\n  ↓\nSmall metadata transfer for CPU tracking\n  ↓\nGPU torso ROI extraction\n  ↓\nOCR queue\n  ↓\nDownstream processing / RTSP output\n```\n\nI switched the decoding path to **PyNvVideoCodec / NVDEC**. This allowed decoded frames to remain in GPU-accessible memory instead of becoming CPU images that would then need to be uploaded for inference.\n\nNVDEC doesn't make an entire application zero-copy by itself. The formats and buffers used by the next stages still matter. The useful change here was removing an unnecessary full-frame detour through host memory.\n\nResize, letterboxing, and normalization ran on the GPU. That let the pipeline prepare TensorRT input without routinely copying the full-resolution frame back to the CPU.\n\nThese operations aren't necessarily expensive individually. The problem is what happens when each stage creates another buffer or forces a transfer before the next stage can run.\n\nI replaced the higher-level inference path with direct TensorRT execution using `execute_async_v3()`.\n\nThat gave me explicit control over device buffers and execution. It wasn't the function name that made the pipeline faster: memory layout, buffer lifetime, and synchronization still had to be handled correctly. The benefit was being able to avoid work that the earlier path performed around inference.\n\nThe pose model returned person bounding boxes and **17 body keypoints**. Postprocessing and coordinate mapping stayed on the GPU.\n\nTracking still used CPU-side information, but only compact detection metadata needed to cross to the CPU. Sending an entire 1080p frame just to update tracking coordinates would have defeated much of the optimization.\n\nThe pose keypoints defined torso regions for OCR. I extracted those ROIs on the GPU rather than copying the complete image to the CPU just to crop it.\n\nThose ROIs then entered the OCR queue. This boundary matters when interpreting the performance number below.\n\nI didn't try to move every part of the application to CUDA. I kept large pixel buffers on the GPU and moved smaller data when a CPU-side stage actually needed it.\n\nThe cover image is a frame from the tested pipeline, with tracking IDs and FPS counters overlaid.\n\n| Measurement | Value | \n|---|---|\n| Video source | 1920×1080 RTSP | \n| Source frame rate | 25 FPS | \n| GPU | NVIDIA GeForce RTX 5070 Ti | \n| Before optimization | 8–10 processed FPS | \n| After optimization | **24.64 processed FPS** | \n| Output FPS counter | 25.00 FPS | \n| Processing measurement boundary | Before OCR queue | \n\nThe overlay showed:\n\n```\nPROCESSED: 24.64\nOUTPUT:    25.00\nSOURCE:    25\n```\n\nCompared with the original 8–10 FPS range, **24.64 FPS is about 2.5–3.1× the initial processing throughput**.\n\nThe distinction between counters is important. The processed-FPS figure was measured up to the OCR queue. The 25 FPS output counter doesn't establish that OCR, safety checks, encoding, and network delivery all finished within 40 ms. Throughput and end-to-end latency are different measurements.\n\nI also tested this architecture with **four simultaneous camera streams on the same RTX 5070 Ti**. I'm not publishing per-stream throughput or latency figures for that configuration here, so the 24.64 FPS result should be understood as the single-stream measurement.\n\nIf I encountered another RTSP application that wasn't keeping up with the camera, I'd check these boundaries before changing the model:\n\nI'd also record stage timings and queue depth instead of relying on one FPS counter. A pipeline can appear fast at the inference stage while its end-to-end latency continues to grow.\n\nFor API details, NVIDIA documents [PyNvVideoCodec decoding and interoperability](https://docs.nvidia.com/video-technologies/pynvvideocodec/pynvc-api-prog-guide/using_pynvvideocodec_apis.html) and [TensorRT asynchronous Python inference](https://docs.nvidia.com/deeplearning/tensorrt/latest/inference-library/python-api-docs.html). These are references for implementing similar techniques, not code taken from the customer's deployment.\n\nI'd diagram the entire frame path before spending time tuning inference: every buffer allocation, every CPU-to-GPU or GPU-to-CPU transfer, and every synchronization point.\n\nIn this project, that was where the meaningful improvement came from. A fast TensorRT engine didn't help much while the rest of the pipeline kept moving full-size frames around.\n\nI shared more context and the original benchmark in [my LinkedIn post about this RTSP/TensorRT optimization](https://www.linkedin.com/feed/update/urn:li:activity:7494790587561406464/). If you've run into a similar bottleneck, I'd be interested to hear whether it was decoding, preprocessing, memory movement, or something else.\n\n*Implementation note: I configured and tested this pipeline on a customer's server. The performance figures come from that work. AI assistance was used to draft and edit the English text.*", "url": "https://wpnews.pro/news/tensorrt-was-fast-my-1080p-rtsp-pipeline-wasn-t", "canonical_source": "https://dev.to/colaweb/tensorrt-was-fast-my-1080p-rtsp-pipeline-wasnt-11df", "published_at": "2026-10-30 10:48:25+00:00", "updated_at": "2026-10-30 10:51:54.079919+00:00", "lang": "en", "topics": ["computer-vision", "ai-infrastructure", "ai-tools", "mlops"], "entities": ["TensorRT", "NVIDIA", "PyNvVideoCodec", "NVDEC", "CUDA", "OpenCV", "FFmpeg", "GeForce RTX 5070 Ti"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/tensorrt-was-fast-my-1080p-rtsp-pipeline-wasn-t", "markdown": "https://wpnews.pro/news/tensorrt-was-fast-my-1080p-rtsp-pipeline-wasn-t.md", "text": "https://wpnews.pro/news/tensorrt-was-fast-my-1080p-rtsp-pipeline-wasn-t.txt", "jsonld": "https://wpnews.pro/news/tensorrt-was-fast-my-1080p-rtsp-pipeline-wasn-t.jsonld"}}