Build Agentic Computer Vision with Roboflow Workflows Roboflow published a guide describing agentic computer vision, an architecture in which a vision model's output feeds a reasoning step that decides and executes an action instead of stopping at a bounding box or label. The guide outlines a four-step loop — perceive, reason, act, verify — combining detectors such as RF-DETR for perception, a vision language model for judgment, and tool calls for action, and cites ReAct, Toolformer, and Reflexion as research foundations for interleaving reasoning and actions. Agentic computer vision is a system where a vision model's output feeds a reasoning step that decides what to do next and takes an action, instead of stopping at a box or a label. In practice, that means a detector such as RF-DETR for perception, a vision language model for judgment, and tool calls or integrations for action, chained in a loop that can check its own result, using Roboflow Workflows. Computer vision has traditionally been built around prediction. An image enters a model and the system returns an object class, bounding box, segmentation mask, keypoint, count, or another structured result. That model output is useful, but it does not necessarily answer the operational question. For example: - A warehouse camera may detect a forklift - A manufacturing camera may identify a damaged carton - A security camera may find a person in a restricted area A conventional system then needs application code around the model to decide whether the observation matters and what should happen next. Agentic computer vision extends this architecture. Instead of treating perception as the endpoint, it treats perception as the first step in a system that can interpret what it sees, use additional context, select an allowed action, observe what happened, and continue from there. The result is not simply a computer vision model. It is a closed-loop computer vision application. What Is Agentic Computer Vision? Agentic computer vision is a computer vision architecture in which visual perception feeds a reasoning layer that can use context and tools to decide what action to take, execute that action, and verify the result. The idea closely follows the broader development of AI agents. Research such as ReAct https://research.google/blog/react-synergizing-reasoning-and-acting-in-language-models/?ref=blog.roboflow.com showed how reasoning and actions can be interleaved rather than handled as isolated stages, while Toolformer https://arxiv.org/abs/2302.04761?ref=blog.roboflow.com demonstrated the value of language models invoking external tools through APIs. Reflexion https://doi.org/10.52202/075280-0377?ref=blog.roboflow.com explored another important part of the pattern, using feedback from an environment to improve the next decision. For vision applications, the environment is physical rather than purely textual. Cameras https://ai1.roboflow.com/?ref=blog.roboflow.com provide observations, vision models https://playground.roboflow.com/models?ref=blog.roboflow.com convert those observations into structured evidence, multimodal models interpret situations, and integrations connect the resulting decision to software or industrial systems. A useful way to think about agentic computer vision is as a four-step loop: 1. Perceive: A detector, segmenter, classifier, OCR model, tracker, or other vision model converts pixels into structured observations such as person detected , forklift at x=420 , defect present , or three pallets counted . 2. Reason: A vision language model or other reasoning model combines the visual evidence with context such as time, location, policy, previous observations, production state, or external data and determines what the situation means. 3. Act: The system invokes an approved tool for example, send a Slack notification, call a webhook, create a maintenance ticket, write an event to a database, update an MES, send a value to an OPC UA server, or request human review. 4. Verify: The system checks whether the action succeeded or whether the visual situation changed as expected. If not, it can retry, choose another allowed path, or escalate. The last step is important because an agent is different from a one-shot multimodal prompt. Consider a camera monitoring a pedestrian area. A traditional object detector might return: forklift - confidence: 0.97 The output is accurate but incomplete from an operational perspective. An agentic system could instead establish that the forklift overlaps a pedestrian-only zone, retrieve the current shift or area policy, determine whether the condition requires escalation, send a message containing the relevant image or video clip to the supervisor responsible for that shift, record the incident, and later verify whether the forklift left the restricted area. That is the difference between seeing an object and using visual evidence to complete a task . The Four Components of an Agentic Vision System The perceive, reason, act, and verify loop can be implemented as four practical system components. 1. Perception: turn pixels into structured facts The first component is the perception layer. For most production systems, this should still be a specialized computer vision model such as an object detector, instance segmentation model, classifier, keypoint model, or OCR model. For object detection, a model such as RF-DETR https://rfdetr.roboflow.com/latest/?ref=blog.roboflow.com can turn an image into structured predictions containing classes, confidence scores, bounding boxes, and coordinates. For example: { "class": "forklift", "confidence": 0.97, "x": 614, "y": 355, "width": 281, "height": 214 } The important architectural decision is that the reasoning model does not necessarily need to inspect every full-resolution frame. A specialist model can first answer narrow questions extremely efficiently: - Is a person present? - Is a trailer present? - Is there a defect? - Did an object enter a polygon? - How many boxes crossed a line? - Has an object remained in a region for more than 30 seconds? Only observations that satisfy those conditions need to move to the more expensive reasoning layer. This architecture is especially useful for video. A 30 FPS stream produces 30 opportunities for inference every second. Asking a VLM to independently reason about every frame is unnecessary for many applications. Instead: The perception layer acts as a visual filter. RF-DETR was designed around the accuracy-latency trade-off required by this kind of workload. The RF-DETR provides a family of lightweight specialist detection transformers that can be fine-tuned to generate specialist detectors. Hence it provide better deployment trade-offs than relying on heavyweight vision-language models for every domain-specific detection problem. There is another reason to separate perception from reasoning. Current VLMs are powerful but should not automatically be treated as perfect localization engines. Research continues to identify visual hallucination and localization failure modes in multimodal models. HallusionBench , published at CVPR, demonstrated that visual reasoning models can produce answers influenced by language priors rather than the evidence in an image. More recent CVPR work such as ORIC similarly finds that unusual object-context combinations can degrade object recognition in large vision-language models. That's why a strong production pattern is to use a specialist model for evidence and then a VLM for ambiguous interpretation, rather than using a VLM for every visual operation. The detector provides the agent's eyes. The reasoning model decides what the evidence means. 2. Reasoning: interpret the scene in context Perception tells the application what is visible. Reasoning determines why it matters. A vision language model can receive some combination of the image, cropped regions, detector output, temporal state, textual policies, sensor values, or data retrieved from external applications. Models from families such as Gemini https://blog.roboflow.com/gemini-computer-vision/ , Qwen-VL https://playground.roboflow.com/models/qwen/qwen-vl?ref=blog.roboflow.com , and GPT can perform tasks that are difficult to represent using fixed detection classes alone. For example, a detector might identify: person forklift pallet door A reasoning prompt can ask: A forklift and a person have been detected inside the loading area. Using the supplied crop and zone information, determine whether: 1. normal loading is occurring, 2. the pedestrian is safely separated from the forklift, 3. the scene is ambiguous and requires human review. Return only structured JSON. The VLM is not replacing the detector in this architecture. It is receiving a much smaller and better-defined reasoning problem. That distinction matters. A model that must first search an entire frame for every relevant object, remember which objects matter, determine their relationships, interpret a policy, and produce a decision has more opportunities to fail than a model receiving already-localized evidence and a constrained task. A production architecture should therefore not assume that a persuasive natural-language answer is necessarily a correct visual judgment. Evaluate the reasoning model on your own task. Vision Playground https://playground.roboflow.com/?ref=blog.roboflow.com and Vision Evals https://playground.roboflow.com/evals?ref=blog.roboflow.com are useful starting points because models can be compared across standardized visual tasks rather than selected solely from general language-model benchmarks. The current evaluation covers object detection, counting, identification, OCR, data extraction, and reasoning using the same samples and ground truth. The production choice may not be the model with the highest overall benchmark score. A warehouse https://roboflow.com/industries/warehousing?ref=blog.roboflow.com application may prioritize reasoning accuracy and cost. A document workflow may prioritize OCR and structured extraction. A near-real-time monitoring application may value response speed more heavily. The reasoning layer should therefore be benchmarked against your actual visual decisions, not just a generic leaderboard. 3. Action: give the system controlled tools A reasoning model becomes operationally useful when its decision can cause something to happen. This is the action layer. An action can be entirely digital such as: - send slack alert - create ticket - post webhook - write database record - send email - request human review Or it can interact with an industrial system: - write opcua tag - write plc value - publish mqtt event Roboflow Workflows supports integration patterns including webhooks, Slack notifications https://blog.roboflow.com/slack-notification-workflows/ , email, SQL Server, MQTT https://roboflow.com/build-a-workflow/roboflow-object-detection-model-to-mqtt-publisher?ref=blog.roboflow.com , OPC UA, and PLC https://blog.roboflow.com/computer-vision-plc-integration/ -oriented blocks. The precise integrations available depend on deployment and plan. For example, the output of the reasoning layer could become: { "status": "blocked", "severity": "medium", "reason": "pallet obstructing loading path", "action": "create ticket" } The system can then map create ticket to a specific approved integration. This is preferable to giving a model unrestricted control over arbitrary APIs. The agent should choose from a small set of actions that the application explicitly exposes. A good action contract might allow: NO ACTION NOTIFY SUPERVISOR CREATE MAINTENANCE TICKET REQUEST HUMAN REVIEW but not arbitrary commands. This principle becomes even more important when computer vision interacts with physical equipment. Roboflow supports industrial integrations such as an OPC UA Writer Sink https://blog.roboflow.com/integrate-roboflow-with-ignition/ , PLC Writer, and Modbus-oriented workflow integrations. An OPC UA workflow can, for example, convert a vision result into a Boolean, count, status, or defect code and publish that value to an OPC UA server used by PLCs, SCADA applications, or dashboards. The reasoning model, however, should not become the machine's safety controller. For industrial systems https://roboflow.com/industries/industrial-manufacturing?ref=blog.roboflow.com , an agent can recommend or publish a bounded state such as: inspection result = REVIEW while deterministic PLC logic retains responsibility for machine timing, interlocks, watchdogs, safety conditions, and physical actuation. Agentic reasoning should expand what the vision system understands, not remove the safeguards already built into the control system. 4. Memory and verification: know what happened before and what happened next A single frame has no memory, a real operational process does. Suppose a camera sees a trailer at a dock door. One frame cannot tell you whether the trailer arrived one second ago or has remained idle for 45 minutes. Similarly, detecting a person inside a region does not tell you whether that person just entered, has been there continuously, or has crossed the same area repeatedly. This is where tracking and state become part of the agent. Roboflow Workflows includes tracking and temporal-processing components that can maintain object identity and compute events across frames. The available block ecosystem includes Byte Track https://trackers.roboflow.com/latest/trackers/bytetrack/?ref=blog.roboflow.com , BoT-SORT https://trackers.roboflow.com/latest/trackers/botsort/?ref=blog.roboflow.com , Time in Zone https://docs.roboflow.com/workflows/blocks/blocks/video-processing/timein-zone?ref=blog.roboflow.com , counters, caches, filters, and other stateful processing blocks. A tracked observation might evolve like this: Frame 120: truck 17 enters dock zone 4 Frame 1,920: truck 17 still present time in zone = 60 seconds Frame 10,920: truck 17 still present time in zone = 360 seconds Only after the temporal condition is satisfied does the reasoning layer need to run. Memory can also include non-visual state: { "camera": "dock 04", "track id": 17, "first seen": "14:03:11", "time in zone seconds": 367, "previous state": "loading", "previous action": "none", "shift": "afternoon" } The second part of this component is verification. After the agent acts, ask a simple question: Did the intended result actually occur? Verification might be digital. A ticket API returns a ticket ID. Slack returns a successful response. An OPC UA write reports success. A database confirms that the event was inserted. Or verification can be visual. The system sends an alert because an access route is obstructed. Thirty seconds later, it inspects the next observation. If the obstruction remains, it escalates. If it disappears, it closes the event. That closes the loop: Research on agent architectures has repeatedly shown the value of feedback rather than assuming the first generated action is final. Reflexion, for example, formalizes the use of environmental feedback and memory to guide later decisions. For production vision systems, the implementation can be much simpler than a research agent. Verification may be nothing more than checking an API status code, querying an acknowledgement flag, or looking at the next relevant camera frame. Why Agentic Computer Vision Is Possible Now Agentic computer vision combines these components i.e. object detection, tracking, workflow automation, multimodal reasoning, and APIs into one deployable system. Vision language models can reason about visual context Modern VLMs can answer open-ended questions about images, extract structured information, compare visual evidence, interpret text inside scenes, and reason about relationships that would otherwise require many narrowly trained classes. Roboflow's current Vision Evals compares dozens of models across six grounded visual tasks and records accuracy, token usage, estimated cost, and speed. This makes model selection increasingly measurable rather than anecdotal. At the same time, research on hallucinations makes an important limitation clear i.e. VLM output must still be evaluated and constrained. The existence of strong reasoning does not eliminate the need for reliable perception, structured output, and verification. This is exactly why the hybrid architecture works well. Use a specialist model to establish visual facts, then let the VLM reason about the part that is difficult to hard-code. Detectors are fast enough to gate VLM reasoning The second enabling technology is fast local perception. A specialist detector can run continuously while the slower reasoning model runs only when needed. RF-DETR demonstrated single-image detection latency in the low-millisecond range on the specified T4/TensorRT configuration, allowing perception to operate at a fundamentally different cadence from VLM reasoning. This creates an architecture such as: This gating step is also what makes the economics more practical. VLM APIs are generally priced per input and output token or equivalent model usage. If a camera generates 108,000 frames per hour at 30 FPS, sending every frame to a remote reasoning model is usually unnecessary. If the detector and temporal logic reduce that stream to a handful of meaningful events, the VLM is paying attention only when semantic reasoning is useful. The detector therefore acts not only as a perception layer, but also as a cost and latency gate. Workflows can orchestrate the system without custom glue code The third change is orchestration. Building this system required developers to separately implement video capture, model serving, tracking, event logic, VLM APIs, JSON parsing, retry behavior, webhooks, notification services, and deployment. Roboflow Workflows provides a visual environment for chaining these operations into a multi-step computer vision application. A Workflow can then be deployed in the cloud or on compatible local hardware using Roboflow Inference https://inference.roboflow.com/?ref=blog.roboflow.com and can process images, videos, and live streams. The available Workflow block ecosystem includes object detection, tracking, conditions, dynamic crops, VLMs, JSON parsing, Slack and email notification, webhooks, databases, MQTT, and industrial integrations. The architecture can therefore be represented directly: This makes agentic vision useful for applications where the visual observation itself is straightforward but interpreting its operational significance is not. Examples include safety-event triage, logistics https://roboflow.com/industries/logistics?ref=blog.roboflow.com exception monitoring, maintenance assessment, retail shelf exceptions https://blog.roboflow.com/retail-store-object-detection/ , visual inspection escalation, site monitoring, asset condition reporting, document workflows, and situations where an operator currently has to look at an image before deciding what business process should happen next. Building An Agentic Computer Vision System in Roboflow Workflows Roboflow Workflows https://roboflow.com/workflows/build?ref=blog.roboflow.com provides the orchestration layer needed to connect perception, reasoning, structured outputs, control flow, actions, and deployment into one computer vision application. The Roboflow Vision Agents https://blog.roboflow.com/vision-agents/ tutorial demonstrates the general vision-agent pattern using a specialist perception model, conditional gating, Gemini-based reasoning, JSON parsing, and an automated notification. To demonstrate this architecture, we will build a dock-door monitoring system https://roboflow.com/ai/dock-safety?ref=blog.roboflow.com that watches a video feed, detects trucks and people, tracks how long each truck remains near the loading bay, and triggers an AI assessment when a truck exceeds a configurable dwell-time threshold. The completed system uses two Roboflow Workflows: 1. Dock Door Monitor https://app.roboflow.com/workflows/embed/eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ3b3JrZmxvd0lkIjoiNjZqY0VjSjRVaEl0dkJ1eDV3engiLCJ3b3Jrc3BhY2VJZCI6InZjQmw1Y0x3bUtQallLTGNRemV1VkE4UlRhNjIiLCJ1c2VySWQiOiJ2Y0JsNWNMd21LUGpZS0xjUXpldVZBOFJUYTYyIiwiaWF0IjoxNzg5ODA1MDY5fQ.PNMpnUu4GRlDiyJby8aHLE2uiAOoijHwFIanj9ERHU0?ref=blog.roboflow.com processes the video, detects and tracks trucks, measures dwell time, and emits a single trigger frame. 2. Dock Door Frame Analyzer https://app.roboflow.com/workflows/embed/eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ3b3JrZmxvd0lkIjoiY0g0QklpZklQSkhhZXpGaXpLdmYiLCJ3b3Jrc3BhY2VJZCI6InZjQmw1Y0x3bUtQallLTGNRemV1VkE4UlRhNjIiLCJ1c2VySWQiOiJ2Y0JsNWNMd21LUGpZS0xjUXpldVZBOFJUYTYyIiwiaWF0IjoxNzg5ODA1MTExfQ.iDt5wUN-IgB23E2J9Q6ol6 bTVqQB-DnWqmicMdsDmQ?ref=blog.roboflow.com sends that trigger frame to Gemini and returns a structured assessment of whether the dock is active, idle, or blocked. A small Python application connects the two Workflows: Workflow 1: Detect, track, and decide when an event is worth investigating The first Workflow, Dock Door Monitor https://app.roboflow.com/workflows/embed/eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ3b3JrZmxvd0lkIjoiNjZqY0VjSjRVaEl0dkJ1eDV3engiLCJ3b3Jrc3BhY2VJZCI6InZjQmw1Y0x3bUtQallLTGNRemV1VkE4UlRhNjIiLCJ1c2VySWQiOiJ2Y0JsNWNMd21LUGpZS0xjUXpldVZBOFJUYTYyIiwiaWF0IjoxNzg5ODA3MTUwfQ.nb uKkjmKoArgSvO7TqLyFW896UK2EuknnahudytM88?ref=blog.roboflow.com , is responsible for the fast perception and memory parts of the agent. It starts with two inputs: - an image , which receives each video frame, - and a dwell seconds parameter. The second input makes the dwell threshold configurable at runtime, so the same published Workflow can be tested with a one-second threshold or deployed with a much longer production threshold without rebuilding the graph. RF-DETR Object Detection The first processing block is an Object Detection Model using rfdetr-small . For this example, the model is configured with a confidence threshold of 0.4 and restricted to the classes: truck person RF-DETR therefore answers the first question in the system: What objects are visible in this frame? Filtering to trucks and people also prevents unrelated detections from being passed through the rest of the Workflow. The pipeline begins as: ByteTrack Tracker Object detection works frame by frame. It can detect a truck in two consecutive frames, but by itself it does not know that both detections represent the same truck. The ByteTrack Tracker solves this by assigning a persistent tracker id to objects as they move through the video. This gives the Workflow memory across frames. The example uses a minimum of two consecutive frames before establishing a track, a lost-track buffer of 30 frames, and detection thresholds of 0.4 . The buffer allows an object's identity to survive short detection gaps caused by blur or temporary occlusion. This persistent identity is what makes dwell-time measurement possible. Time in Zone Next, the tracked detections are passed to Time in Zone. A polygon is drawn around the dock-door region. When the center of a tracked object's bounding box enters that polygon, the block begins measuring how long it remains there. For the example camera, the polygon is: 267, 173 , 48, 464 , 337, 552 , 506, 250 These coordinates are camera-specific and should be redrawn for a different dock view. The block uses the bounding-box CENTER as the triggering anchor and resets the timer after an object leaves the zone. The agent can now answer a second question: How long has this particular truck been at the dock? The sequence has become: Detections Filter The Detections Filter https://docs.roboflow.com/workflows/blocks/blocks/logic-and-branching/detections-filter?ref=blog.roboflow.com determines when a truck becomes operationally interesting. It retains a detection only when: class name == "truck" AND time in zone = dwell seconds The runtime dwell seconds value may arrive as a string, so the comparison uses a ToNumber operation to cast it to a float before comparing it with time in zone . This block therefore answers: Has a truck remained at the door long enough to require attention? Instead of asking Gemini to inspect every truck immediately, the deterministic pipeline waits until the dwell condition has actually been met. Property Definition: count qualifying trucks The filtered detections are then passed to a Property Definition https://docs.roboflow.com/workflows/blocks/blocks/advanced-blocks/property-definition?ref=blog.roboflow.com block using the SequenceLength operation. This produces: qualifying truck count For example: 0 → no truck is currently over the dwell threshold 1 → one truck is over the threshold 2 → two trucks satisfy the condition SequenceLength is important here because the task is to count detections, rather than extract one of their properties. Delta Filter and Continue If Without another control step, the Workflow would continue triggering on every subsequent frame while the same truck remained over the threshold. That would result in repeated Gemini requests. A Delta Filter https://docs.roboflow.com/workflows/blocks/blocks/logic-and-branching/delta-filter?ref=blog.roboflow.com watches qualifying truck count and allows the next branch to execute only when the value changes. A Continue If https://docs.roboflow.com/workflows/blocks/blocks/logic-and-branching/continue-if?ref=blog.roboflow.com block then checks whether: qualifying truck count 0 Together, these blocks convert a continuous condition into an event. For example: This is an important part of the agentic architecture. The perception system does not simply report detections continuously; it identifies when a meaningful change has occurred and only then allows the reasoning or action stage to continue. Select the qualifying truck Once the event branch is triggered, a Detections Transformation https://docs.roboflow.com/workflows/blocks/blocks/transformations/detections-transformation?ref=blog.roboflow.com block selects the first truck that satisfies the dwell-time condition. This selected detection identifies the specific truck that caused the event and is passed to the trigger-frame visualization. There is no separate truck crop in this version of the Workflow. Instead, the complete dock-door frame is preserved so that the later reasoning stage can see both the truck and its surrounding environment. Create the trigger frame A Bounding Box Visualization https://inference.roboflow.com/workflows/blocks/bounding box visualization?ref=blog.roboflow.com block creates the event image called: trigger frame It uses the annotated dwell-time frame as the image and highlights the selected qualifying truck. The full trigger frame is particularly useful for VLM reasoning because the operational cause of a delay may not be inside the truck's bounding box. For example, the surrounding scene may contain a person, forklift, pallet, equipment, closed access point, or another condition affecting dock activity. The event path is therefore: This trigger frame becomes the visual evidence sent to the second Workflow for semantic analysis. Continuous video visualization The trigger frame is intentionally sparse: it is produced only when the event branch executes. The Workflow therefore has a separate visualization path that remains populated during normal video processing. A Bounding Box Visualization draws the objects returned by Time in Zone, and a Label Visualization https://docs.roboflow.com/workflows/blocks/blocks/visualize-predictions/label-visualization?ref=blog.roboflow.com displays their Time In Zone values. Using the tracker ID as the color axis also makes it easier to visually follow the same object across frames. This produces the continuously available: output image The two visualization paths therefore serve different purposes: output image is the normal monitored video, while trigger frame is generated only when a qualifying dwell event occurs. Workflow outputs The updated Dock Door Monitor workflow exposes: - Continuous video with detections and dwell-time labels - ByteTrack tracked detections - Time in Zone detections - Trucks satisfying the dwell-time rule - Number of trucks currently satisfying the rule - Context-rich event frame highlighting the qualifying truck - Error status returned by the optional Slack action - Message produced by the optional Slack action The distinction between continuous and event-driven outputs is important. output image remains available on every processed frame, whereas trigger frame is populated only when the Delta Filter and Continue If branch execute. For this reason, output image should remain the primary video output. The sparse trigger frame is used as evidence for the reasoning Workflow. Optional Slack action The Workflow also contains a Slack Notification https://docs.roboflow.com/workflows/blocks/blocks/notifications/slack-notification?ref=blog.roboflow.com block connected to the event path. A notification could contain information such as: Dock-door alert: 1 truck exceeded the configured 60-second dwell threshold. Review the trigger frame in the monitoring application. The message can use qualifying truck count and the configured dwell seconds value, with a cooldown to prevent excessive notifications. When Slack integration is required in production, the block can be enabled after providing the appropriate Slack credentials and destination channel. The first Workflow now represents the perceive → remember → decide → trigger action part of the agent: This design keeps continuous perception fast and deterministic. Gemini/ VLM is not called for every frame. The first Workflow detects and tracks activity, maintains temporal state, applies the dwell-time rule, and produces a single context-rich trigger frame only when an event becomes important enough to investigate. Workflow 2: Let Gemini interpret the event The second Workflow, Dock Door Frame Analyzer https://app.roboflow.com/workflows/embed/eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ3b3JrZmxvd0lkIjoiY0g0QklpZklQSkhhZXpGaXpLdmYiLCJ3b3Jrc3BhY2VJZCI6InZjQmw1Y0x3bUtQallLTGNRemV1VkE4UlRhNjIiLCJ1c2VySWQiOiJ2Y0JsNWNMd21LUGpZS0xjUXpldVZBOFJUYTYyIiwiaWF0IjoxNzg5ODM2NTgzfQ.c67TkN822sYo4B iuW-ZF32BQSRDxNBWod8Rtkf7IWI?ref=blog.roboflow.com , is deliberately much smaller. It processes only the event image produced by the first Workflow. This Workflow receives one image input and passes it to a Google Gemini https://docs.roboflow.com/workflows/blocks/blocks/run-a-model/google-gemini?ref=blog.roboflow.com block configured for visual question answering using gemini-2.5-flash . The prompt asks Gemini to determine whether the trailer is: loaded idle blocked and return only: { "status": "loaded|idle|blocked", "blocked by": "short description or none", "reason": "one concise sentence" } The instructions further define the meanings: - loaded means visible loading or unloading activity is occurring. - idle means the trailer is docked but there is no visible work or obstruction. - blocked means a visible person, vehicle, object, equipment, closed access point, or another condition appears to prevent work. The explicit schema is important because the output is intended for software, not just human reading. JSON Parser Gemini's text response is passed directly to a JSON Parser https://docs.roboflow.com/workflows/blocks/blocks/advanced-blocks/json-parser?ref=blog.roboflow.com . The parser extracts: status blocked by reason and exposes them as: dock status blocked by assessment reason The Workflow also returns gemini raw output and json parse error . These are useful when validating the application because they make malformed or incomplete VLM responses visible instead of silently hiding them. The final result can therefore look like: { "dock status": "blocked", "blocked by": "person standing between trailer and dock", "assessment reason": "A worker appears to obstruct access to the loading area.", "json parse error": false } This illustrates the reasoning layer clearly. Connecting the two Workflows A small Python application connects Dock Door Monitor https://app.roboflow.com/workflows/embed/eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ3b3JrZmxvd0lkIjoiNjZqY0VjSjRVaEl0dkJ1eDV3engiLCJ3b3Jrc3BhY2VJZCI6InZjQmw1Y0x3bUtQallLTGNRemV1VkE4UlRhNjIiLCJ1c2VySWQiOiJ2Y0JsNWNMd21LUGpZS0xjUXpldVZBOFJUYTYyIiwiaWF0IjoxNzg5OTEzMTExfQ.h2F5PYW2iA5OVwyej4WbVW7C7gvHc2b2TJD2UD5GlTA?ref=blog.roboflow.com and Dock Door Frame Analyzer https://app.roboflow.com/workflows/embed/eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ3b3JrZmxvd0lkIjoiY0g0QklpZklQSkhhZXpGaXpLdmYiLCJ3b3Jrc3BhY2VJZCI6InZjQmw1Y0x3bUtQallLTGNRemV1VkE4UlRhNjIiLCJ1c2VySWQiOiJ2Y0JsNWNMd21LUGpZS0xjUXpldVZBOFJUYTYyIiwiaWF0IjoxNzg5OTEzMTM1fQ.bEGYLTuX-IP tuDsr8MkTf6jCeC-LAO9luro4uZv xA?ref=blog.roboflow.com . The first Workflow runs continuously over the video using WebRTC. It performs RF-DETR detection, ByteTrack tracking, dwell-time measurement, and event gating. The application displays the continuous output image stream and listens for the sparse trigger frame output. When trigger frame is produced, Python sends that single image to Dock Door Frame Analyzer. Gemini interprets the scene, the JSON Parser converts the result into structured fields such as dock status , blocked by , and assessment reason , and the application prints the result, stores it as JSON, and overlays the latest assessment on the video. The Gemini request runs in a background thread so the slower reasoning step does not block the continuous video-processing callback. This preserves the separation between fast, stateful perception and event-driven reasoning. . https://github.com/tim3in/cv-examples/blob/main/dock monitor.py?ref=blog.roboflow.com dock monitor.py Outputs generated by the application Live annotated video: Displays the continuous output image from Dock Door Monitor, with tracked detections, dwell-time information, and the latest Gemini assessment overlaid on the video. Trigger frame: Saves trigger frame