Analyze Video Feeds for Process Monitoring with RF-DETR

Roboflow released a tutorial on using RF-DETR, ByteTrack, and custom counting logic within Roboflow Workflows to build a real-time video process monitoring pipeline for manufacturing, logistics, and traffic analysis. The system detects objects, assigns persistent tracking IDs, and counts crossings over a defined line, enabling automated extraction of operational data such as object counts, throughput, and speed from raw camera feeds.

Video process monitoring automates the extraction of actionable operational data such as object counts, throughput, and speed, from raw camera feeds without requiring manual observation. By combining RF-DETR for object detection, ByteTrack for persistent tracking, and custom counting logic within Roboflow Workflows, you can easily build and deploy a real-time analytics pipeline for manufacturing, logistics, and more. Analyzing video feeds for process monitoring means using computer vision to turn raw camera footage into operational data: counts, throughput, speed, and alerts, with no person watching a screen. In this tutorial, you'll build a working process monitoring pipeline in Roboflow Workflows https://roboflow.com/workflows/build?ref=blog.roboflow.com , combining RF-DETR https://rfdetr.roboflow.com/latest/?ref=blog.roboflow.com for detection, ByteTrack https://inference.roboflow.com/workflows/blocks/byte track tracker/?ref=blog.roboflow.com for tracking, and a line counter to measure flow. We'll demo it on a highway CCTV feed, a process where the stakes are high: road crashes cause around 1.19 million deaths annually https://www.who.int/news-room/fact-sheets/detail/road-traffic-injuries?ref=blog.roboflow.com and cost most countries about 3% of GDP, per WHO. But the same pipeline monitors parts on a conveyor, packages in a warehouse https://roboflow.com/industries/warehousing?ref=blog.roboflow.com , or people in a work zone. By the end, you'll have a workflow that detects objects, assigns persistent tracking IDs, and counts each one crossing a defined line. How to Analyze Video Feeds for Process Monitoring with Roboflow A CCTV feed is processed frame by frame. RF-DETR detects vehicles, ByteTrack tracks them with unique IDs, and the Line Counter increments when a vehicle crosses the line. RF-DETR: Detects vehicles and returns labels, boxes, and confidence. ByteTrack: Tracks vehicles and prevents double counting. Line Counter: Counts each vehicle crossing once. This setup works best for fixed cameras with consistent vehicle movement. Heavy occlusion or multiple angles require a specialized multi-camera tracking system. Dataset The dataset https://universe.roboflow.com/fsmvu/traffic-detection-zthfz?ref=blog.roboflow.com for this tutorial is available on , a collection of over 1 billion images, 1 million open-source datasets, and 250,000 fine-tuned models. It contains images captured from fixed elevated cameras at intersections and highways, annotated across six classes: car, bus, bicycle, motorcycle, motorbike, and person. https://universe.roboflow.com/?ref=blog.roboflow.com Roboflow Universe Fork the dataset into your own Roboflow workspace to get started. The fork copies the images and annotations into your account so you can generate a versioned dataset and train directly from it. The dataset comes with a train, validation, and test split already defined. The train split is used for model training, the validation split for tuning, and the test split is held out for evaluation. Train RF-DETR Go to the Versions tab in your forked project and generate a new dataset version. Once the version is generated, click Custom Train and select RF-DETR Small. Training runs through Roboflow's hosted pipeline with no local GPU or setup required. Roboflow will show a summary of the training configuration before starting, including the model size, dataset version, split counts, and estimated training time. Click Start Training . Once training completes, copy the model URL and proceed to the Build the Workflow section. Build the Workflow Here's the workflow we'll build. https://app.roboflow.com/workflows/embed/eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ3b3JrZmxvd0lkIjoiZzg1MjlFNERxMkUyM05WVW1MNHIiLCJ3b3Jrc3BhY2VJZCI6Im5JRk5DOGRjbU5OOXZ4d29ybWpoWTdCNjdQZTIiLCJ1c2VySWQiOiJuSUZOQzhkY21OTjl2eHdvcm1qaFk3QjY3UGUyIiwiaWF0IjoxNzgzODc3MzM4fQ.m6he q47VJyu3TYu-iZrvlylTeaK3n1rTGVk C7DvGg?ref=blog.roboflow.com What each block does Object Detection Model: Detects vehicles in each frame and returns a bounding box, class label, and confidence score. ByteTrack: Tracks vehicles across frames with a unique ID to prevent double counting. Velocity: Estimates vehicle speed in km/h using pixel movement and a pixels-per-meter calibration. Line Counter: Increments the count when a tracked vehicle crosses the defined line. Class Line Count: Keeps a running total for each vehicle class. Line Counter Visualization: Displays the counting line and live counts on the video. Bounding Box Visualization: Draws a box around each detected vehicle. Label Visualization: Shows the tracking ID above each vehicle. Speed Label Visualization: Displays the estimated speed on each vehicle. Roboflow Vision Events: Logs detections, counts, and speed data for every processed frame. Step 1: Create a new Workflow Open the Workflows tab in your Roboflow workspace and click Create Workflow . Select Custom Workflow . Roboflow automatically adds an Image Input block and an Outputs block to the canvas. The Image Input block is the entry point for every frame of the video. All subsequent blocks connect to it directly or through the chain of preceding blocks. Step 2: Add the vehicle detector as an Object Detection Model block Add an Object Detection Model block named vehicle detector . Connect Image to inputs.image and add your model URL. Set: - Confidence: Custom 0.4 - Classes: car, bus, motorcycle, motorbike - IoU: 0.3 - Max Detections: 300 Only detections above 0.4 confidence are passed to ByteTrack in the next step. Step 3: Add a ByteTrack Tracker block Add a ByteTrack Tracker block named byte tracker . Connect Image to inputs.image and Detections to vehicle detector.predictions . Set: - IoU Threshold: 0.1 - Consecutive Frames: 2 - Lost Track Buffer: 30 - Activation Threshold: 0.7 - High Confidence Threshold: 0.6 Minimum Consecutive Frames is set to 2, which filters out false positives before the tracker assigns an ID. Lost Track Buffer is set to 30, keeping tracks alive through brief occlusions. Step 4: Add a Velocity block Add a Velocity block named speed estimation . Connect Image to inputs.image and Detections to byte tracker.tracked detections . Set: - Smoothing Alpha: 0.5 - Pixels Per Meter: 8 Pixels Per Meter is set to 8, converting pixel movement into real-world distance. Calibrate this value against your scene for accurate speed estimates. Smoothing Alpha is set to 0.5, which reduces speed fluctuations between frames. Step 5: Add a Line Counter block Add a line counter block. Connect image to inputs.image and detections to speed estimation.velocity detections . Line coordinates: 0,360 , 1280,360 Set Triggering Anchor to CENTER . Line is placed at the frame center. Adjust the y-coordinate as needed. Counts increase once per vehicle crossing. Step 6: Add a Class Line Count block Add total count Custom Python block. Connect image to inputs.image and detections to Line Counter . Outputs: display text , counts , total . Click Edit Code to open the code editor. The block needs three inputs, three outputs, and the Python code below. Paste the following: python def run self, image, detections in, detections out, class names : video id = "default" try: meta = getattr image, "video metadata", None if isinstance meta, dict : video id = str meta.get "video identifier" or meta.get "id" or "default" elif meta is not None: video id = str getattr meta, "video identifier", None or getattr meta, "id", None or "default" except Exception: pass classes = c.strip for c in class names.split "," if isinstance class names, str else str c for c in class names or "car", "truck", "bus", "motorcycle" if not hasattr self, "state" : self.state = {} state = self.state.setdefault video id, {"seen": set , "counts": {c: 0 for c in classes}} for c in classes: state "counts" .setdefault c, 0 def norm name : v = str name or "" .strip return "motorcycle" if v == "motorbike" else v def update dets : if dets is None or len dets == 0: return names = list dets.data.get "class name", if hasattr dets, "data" else tids = getattr dets, "tracker id", None cids = getattr dets, "class id", None for i in range len dets : cls = norm names i if i < len names else cids i if cids is not None and i < len cids else "unknown" if cls not in state "counts" : continue tid = int tids i if tids is not None and i < len tids else None key = f"{cls}:{tid}" if tid is not None and tid = 0 else f"{cls}:untracked:{i}" if key not in state "seen" : state "seen" .add key state "counts" cls += 1 update detections in update detections out counts = {c: int state "counts" .get c, 0 for c in classes} total = sum counts.values text = "Vehicles crossed by class:\n" + "\n".join f"{c}: {counts c }" for c in classes return {"display text": text, "counts": counts, "total": total} Tracks class counts using IDs to avoid duplicates and maps motorbike to motorcycle . Step 7: Add a Bounding Box Visualization block Add a bounding box visualization block. Connect image to line counter visualization.image and predictions to speed estimation.velocity detections . This draws a colored box around every detected vehicle on the output frame. Step 8: Add a Label Visualization block Add tracker id labels block. Connect image to bounding box visualization.image and predictions to byte tracker.tracked detections . Set text to Tracker ID. Settings: Copy Image enabled, palette DEFAULT, size 10, colors FF0000, 00FF00, 0000FF. This writes the tracking ID above each bounding box so individual vehicles can be followed visually across frames. Step 9: Add a Speed Label Visualization block Add a Custom Python block named speed label visualization . Connect Image to tracker id labels.image and Predictions to speed estimation.velocity detections . The block has one output: image . Open Edit Code . Set: Block Type: Speed Label Visualization Description: Draws vehicle speed labels using Velocity data. Paste the following: python def run self, image, predictions : arr = image.numpy image.copy try: n = len predictions except Exception: n = 0 if n == 0: return {"image": WorkflowImageData.copy and replace origin image data=image, numpy image=arr } boxes = getattr predictions, "xyxy", speeds = getattr predictions, "data", {} or {} .get "velocity", None h, w = arr.shape :2 font, scale, thickness = cv2.FONT HERSHEY SIMPLEX, 0.55, 2 for i, box in enumerate boxes : if speeds is None or i = len speeds : continue try: label = f"{float speeds i 3.6:.0f} km/h" except Exception: continue x1, y1 = int round box 0 , int round box 1 x = max 0, min w - 1, x1 y = max 40, min h - 5, y1 + 46 tw, th , bl = cv2.getTextSize label, font, scale, thickness cv2.rectangle arr, x, max 0, y - th - bl - 6 , min w-1, x + tw + 8 , min h-1, y + bl + 4 , 0, 0, 0 , -1 cv2.putText arr, label, x + 4, y , font, scale, 255, 255, 255 , thickness, cv2.LINE AA return {"image": WorkflowImageData.copy and replace origin image data=image, numpy image=arr } The block reads velocity data and displays each vehicle's speed in km/h at the top of its bounding box on the output frame. Step 10: Add a Roboflow Vision Events block Add a vision events block. Connect image, output image, and predictions to their respective inputs. Set: - Event Type: Inventory Count - Use Case: Traffic Vehicle Counting - Location: traffic camera - Item Count: total count.total - Item Type: vehicle by class - Cooldown: 1 This block logs each processed frame with its detections, per-class counts, and speed data. It does not affect the workflow output. Step 11: Configure Outputs Add the following outputs to the Outputs block: output image from speed label visualization.image tracked predictions from speed estimation.velocity detections count in from line counter.count in count out from line counter.count out class vehicle counts from total count.counts vision events status from vision events.message total vehicle count from total count.total speed predictions from speed estimation.velocity detections Once all blocks are connected, the full pipeline looks like this: Click Save and then Publish to make the workflow ready to run against your test video. Analyze Video Feeds for Process Monitoring Results Test video: Highway CCTV footage The workflow processes frames in real time. Vehicles are detected and tracked with ByteTrack IDs. The counting line increments when a vehicle crosses it, while speed estimates appear above each bounding box. The JSON output contains each vehicle’s bounding box, confidence score, tracker ID, and class label. Each detection confirms vehicle classification and consistent tracking IDs across frames. Production Deployment Deploy the workflow via Roboflow Inference https://inference.roboflow.com/?ref=blog.roboflow.com to run it against live CCTV feeds or recorded highway footage. Roboflow Inference supports deployment on edge devices, cloud servers, and local machines. Every frame logged by Vision Events becomes a structured data point containing the vehicle class, count, speed estimate, timestamp, and image. Over time, these records reveal traffic patterns, such as peak truck activity or recurring congestion areas. As more real-world detections are collected, the dataset grows automatically and can be used to retrain the model, improving performance on unseen vehicle types and camera angles. Scaling from one camera to many only requires adding new inputs while keeping the same workflow. Where Video Process Monitoring Is Used The detect-track-count pattern in this tutorial is the core primitive of real-time video analytics. Swapping the model and the counting logic adapts it to most industrial processes. Manufacturing: Point the workflow at a production line to count parts, measure throughput, and flag anomalies like stalled conveyors or missing components. The line counter becomes a cycle counter; Vision Events becomes your production log. See how teams apply this in computer vision for manufacturing https://roboflow.com/industries/manufacturing?ref=blog.roboflow.com and industrial manufacturing https://roboflow.com/industries/industrial-manufacturing?ref=blog.roboflow.com . Logistics and warehousing: The same pipeline counts packages crossing a sortation line, tracks pallets between zones, and monitors dock activity from existing CCTV or RTSP streams. Explore vision AI for logistics https://roboflow.com/industries/logistics?ref=blog.roboflow.com and warehousing https://roboflow.com/industries/warehousing?ref=blog.roboflow.com . Safety monitoring: Replace the vehicle detector with a PPE or person model to alert on missing helmets, restricted-zone entries, or near-misses between forklifts and workers. Here's a full walkthrough of PPE detection with RF-DETR https://blog.roboflow.com/ppe-detection/ . In each case the workflow stays the same; only the trained model, the line placement, and the event logic change. Analyze Video Feeds for Process Monitoring with Roboflow Agent Instead of assembling this workflow block by block, you can describe your video analysis task to Roboflow Agent in plain English, such as "count vehicles crossing a line and estimate their speed," and it builds, configures, and tests the pipeline for you. The same approach works for any video feed, whether you're monitoring a production line, a loading dock, or a highway: describe the process you want to monitor, and the Agent assembles the detection, tracking, and counting logic. What is video process monitoring? Video process monitoring uses computer vision models to analyze camera feeds and extract operational data, such as object counts, throughput, speed, and anomalies, in real time. It replaces manual screen-watching with structured, queryable events. Can this workflow run on live CCTV or RTSP streams? Yes. Roboflow Inference accepts RTSP streams and webcam feeds as video inputs, so the same workflow you test on recorded footage runs against live cameras without changes. What hardware do I need? Training runs on Roboflow's hosted infrastructure, so no local GPU is required. For deployment, workflows run in the cloud, on a local server, or on edge devices like an NVIDIA Jetson. A GPU improves frame rates on high-resolution streams but isn't required for lower-throughput feeds. Do I need to train a custom model? Not always. Pre-trained models on Roboflow Universe cover common objects like vehicles and people. Train a custom RF-DETR model when your process involves domain-specific objects, unusual camera angles, or classes the base models miss. Get Started This workflow processes a highway CCTV feed by detecting vehicles with RF-DETR, tracking them with ByteTrack, estimating their speed, and counting each vehicle as it crosses a defined line. The pipeline structure stays the same regardless of what you add. Expanding coverage to new vehicle types only requires labeled data and retraining. Roboflow Workflows, RF-DETR, and Roboflow Inference provide the tools needed to scale this from a tutorial into a production traffic monitoring system. Further Reading Cite this Post Use the following entry to cite this post in your research: Mostafa Ibrahim /author/mostafa/ . Jul 14, 2026 . Analyze Video Feeds for Process Monitoring with RF-DETR. Roboflow Blog: https://blog.roboflow.com/analyze-video-feeds-for-process-monitoring/