Planogram Compliance Detection with Roboflow

Roboflow released a tutorial on building a planogram compliance detection workflow using its RF-DETR model and Google Gemini. The system automates shelf auditing by detecting empty zones and flagging brand mixing violations, addressing the 10% weekly compliance drift that can reduce in-store sales by up to 20%.

Automate planogram compliance with Roboflow Workflows. Learn to use RF-DETR and Gemini to detect empty shelves and flag brand mixing violations. Poor shelf execution is more expensive than most retailers realize. According to Infilect https://www.infilect.com/library/all-about-planogram-compliance?ref=blog.roboflow.com , planograms fall out of compliance at a rate of roughly 10% per week, and improper product placement alone can reduce in-store sales by , according to OmniShelf. As products get restocked incorrectly or placed in the wrong brand zone, shelves drift from their intended layout, and sales quietly follow. https://www.omnishelf.io/blogs/5-reasons-planogram-compliance-fails-in-retail-stores-and-how-to-fix-it?ref=blog.roboflow.com as much as 20% Planogram compliance ensures products are correctly placed, stocked in proper quantities, and aligned with their assigned brand zones. Traditionally, this is done manually by staff inspecting shelves in-store, which doesn’t scale across many locations or frequent shifts. In this tutorial, you will build a planogram compliance workflow in Roboflow Workflows https://roboflow.com/workflows?ref=blog.roboflow.com using an model and two https://blog.roboflow.com/rf-detr/ RF-DETR analysis steps. The model detects empty shelf zones with bounding boxes, while Gemini identifies products and checks for issues like brand mixing, uneven facings, and misplacement without needing a fixed planogram. Rather than comparing against a predefined layout document, the system audits general shelf organization and flags visible violations in real time. https://ai.google.dev/gemini-api/docs?ref=blog.roboflow.com Gemini Here's the workflow we'll build. https://app.roboflow.com/workflows/embed/eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ3b3JrZmxvd0lkIjoiWXFoTGNRRkY5cWhWWXNDSE9oMDkiLCJ3b3Jrc3BhY2VJZCI6Im5JRk5DOGRjbU5OOXZ4d29ybWpoWTdCNjdQZTIiLCJ1c2VySWQiOiJuSUZOQzhkY21OTjl2eHdvcm1qaFk3QjY3UGUyIiwiaWF0IjoxNzgyMTE5NTkwfQ.kRtAQG5x t37a-uJ5GMmgGxYtfjtMC5C7uXOlM In9c?ref=blog.roboflow.com Planogram Compliance Detection with Robfolow Start with Roboflow Universe https://universe.roboflow.com/?ref=blog.roboflow.com . Home to more than 250,000 open-source datasets, it is often the fastest way to find the data needed for a computer vision project. 1. Build your dataset For this tutorial, search for the Supermarket Empty Shelf dataset https://universe.roboflow.com/final-project-e7zqd/super-market-3uol8?ref=blog.roboflow.com . It contains real supermarket shelf images annotated for empty shelf zones across different store layouts and lighting conditions. Click Fork Dataset to copy it into your own workspace. With the dataset forked, you are ready to train the empty shelf detection model that will power the localization branch of the workflow. 2. Train the Model Open the forked dataset and create a new version with default preprocessing and no augmentations. Generate it, go to the Train tab, select RF-DETR https://blog.roboflow.com/rf-detr/ , and start training. Roboflow runs training automatically. Training takes about 30-45 minutes, depending on early stopping. Once finished, check the results page for metrics. For this single-class dataset, you should see a high mAP. Copy the model ID from the results page. You will need it when configuring the Object Detection block. 3. Build the Workflow Block List Google Gemini identify shelf products : Identifies shelf products and returns structured JSON. JSON Parser parse product list : Extracts the product list. Custom Python format compliance input : Prepares the audit payload. Object Detection Model detect empty shelves : Detects empty shelf zones. Custom Python relabel empty zones : Renames detections for display. Custom Python count empty zones : Counts empty shelf zones. Bounding Box Visualization draw empty zones : Draws boxes around empty zones. Label Visualization label empty zones : Label empty zones. Google Gemini generate compliance report : Generates the shelf compliance audit. JSON Parser parse compliance report : Extracts audit results. Custom Python inject empty violations : Adds empty shelf violations deterministically. Roboflow Vision Events log compliance event : Logs each audit run. Step 1: Create the Workflow Open Roboflow Workflows https://app.roboflow.com/workflows?ref=blog.roboflow.com and create a new workflow. The image input is created by default and serves as the single entry point for the entire pipeline. All three branches, product identification, empty shelf detection, and compliance auditing, receive their data from this block. Step 2: Add the Product Identifier Add a Google Gemini block named identify shelf products. Connect it to the input image, set Task Type to Visual Question Answering and Model Version to Gemini 2.5 Flash, then paste the prompt. You are a retail shelf analyst. Analyze this shelf image and identify every visible product. For each product return product name, brand, row top/middle/bottom , position index left to right starting at 1 , and facing count. Return ONLY valid JSON with no markdown and no code fences using this schema: {"products": {"product name": "string", "brand": "string", "row": "string", "position index": integer, "facing count": integer} } The prompt instructs Gemini to return only valid JSON with a fixed schema, which keeps the output clean for the parser in the next step. First stage. All downstream steps depend on its JSON output. Step 3: Add the Product List Parser Add a JSON Parser block named parse product list, connected to identify shelf products, extracting products. Sends a clean product list to the next block. This block is what allows the compliance auditor later in the pipeline to reason over a structured list rather than a raw text string. Step 4: Add the Empty Shelf Detector Add the Object Detection Model block detect empty shelves. Connect the input image, set model ID. Detects empty shelf zones. With the detector in place, the next three steps handle relabeling, counting, and visualizing the detections before they feed into the compliance pipeline. Step 5: Relabel Empty Zones Add a Custom Python block named relabel empty zones. Connect it to the predictions output of detect empty shelves. Define one input, predictions any , and one output, relabeled predictions any . python def run self, predictions : if predictions is None: return {"relabeled predictions": } if isinstance predictions, list : for pred in predictions: pred "class" = "Empty" return {"relabeled predictions": predictions} return {"relabeled predictions": predictions} This block renames every detection class to "Empty" so the label visualization block displays the correct text on each bounding box. With detections relabeled, the next step is to count them before passing the number into the compliance pipeline. Step 6: Count Empty Zones Add a Custom Python block and name it count empty zones. Connect it to the predictions output of detect empty shelves. Define one input, predictions any , and one output, empty count integer . python def run self, predictions : count = 0 try: if predictions is None: count = 0 elif isinstance predictions, list : count = len predictions elif isinstance predictions, dict : count = len predictions.get "predictions", elif hasattr predictions, "predictions" : count = len predictions.predictions except Exception: count = 0 return {"empty count": int count } Rather than passing the full predictions object downstream and risking format mismatches, this block extracts a plain integer that the compliance pipeline can use directly. With the count ready, the next two steps handle the visual output for the empty zones. Step 7: Draw Empty Zone Boxes Add Bounding Box Visualization block draw empty zones. Connect predictions from detect empty shelves, image from inputs.image. Set color red. Draws red boxes on empty zones. With the boxes drawn, the next step is to add the Empty label to each one. Step 8: Label Empty Zones Add a Label Visualization block named label empty zones. Connect predictions from detect empty shelves and image from draw empty zones.image. Set label text to "Empty" and font scale to 1.2. With the empty zone visualization complete, the next step is to prepare the compliance audit payload for Gemini. Step 9: Add the Compliance Input Formatter Add a Custom Python block named format compliance input. Connect product list from parse product list.products and empty count from count empty zones.empty count. Define a single output, formatted input string . python import json def run self, product list, empty count : payload = { "detected products": product list, "empty shelf zones detected": int empty count if empty count else 0, "audit role": "retail planogram compliance auditor", "checks": "brand blocking violations", "inconsistent facing counts", "unexpected visual gaps", "products in illogical positions" , "output schema": { "compliance status": "pass or fail", "violations": {"type": "brand mixing|inconsistent facings|gap|wrong position", "product": "string", "message": "string"} , "summary": "one-line summary" }, "instructions": "Return valid JSON only. No markdown, no code fences.", "Set compliance status to exactly 'pass' or 'fail'.", "Only include clearly visible violations.", "Keep the summary to one line." } return {"formatted input": json.dumps payload, ensure ascii=False } This block combines the product list, empty zone count, audit role, required checks, and output rules into a single structured payload, giving Gemini all the necessary context in one input. With the payload ready, the next step sends it to the compliance auditor. Step 10: Add the Compliance Auditor Add a Google Gemini block named generate compliance report, connected to the input image. Set Task Type to Open Prompt, Model to Gemini 2.5 Flash https://ai.google.dev/gemini-api/docs/models/gemini-2.5-flash?ref=blog.roboflow.com , Thinking Level to low, Max Tokens to 4096, and paste the prompt below: You are a retail planogram compliance auditor. Analyze this shelf image and the visible product arrangement. Evaluate whether this shelf follows good planogram practices. Check for brand blocking violations including mixed brands in the same zone, inconsistent facing counts, unexpected gaps, and products in illogical positions. Return structured JSON only, no markdown, no commentary, no code fences. Schema: {"compliance status":"pass|fail","violations": {"type":"brand mixing|inconsistent facings|gap|wrong position","product":"string","message":"short description"} , "summary":"one-line summary"}. Set compliance status to exactly "pass" or "fail". The prompt keeps the output schema strict so the parser in the next step can extract the fields cleanly every time. This block is the compliance brain of the pipeline. It evaluates the shelf against standard planogram practices and returns a structured audit report. Step 11: Add the Compliance Report Parser Add JSON Parser block parse compliance report, connected to generate compliance report, extracting compliance status, violations, and summary. Outputs clean fields for the next blocks. With the compliance report parsed, the next step is to merge the empty zone violations into the final list. Step 12: Inject Empty Violations Add a Custom Python block named inject empty violations. Connect violations from parse compliance report.violations, compliance status from parse compliance report.compliance status, and empty count from count empty zones.empty count. Define two outputs: final violations list and final status string . python def run self, violations, compliance status, empty count : count = int empty count if empty count else 0 final violations = violations if isinstance violations, list else if count 0: final violations = { "type": "empty shelf", "product": "Multiple shelf zones", "message": f"{count} empty shelf zones detected across the shelf." } + final violations final status = "fail" if count 0 or compliance status == "fail" else "pass" return { "final violations": final violations, "final status": final status } Adds empty shelf violations from detector output. If RF-DETR finds empty zones, they are always included. With the final violations list ready, the next step formats it for display on the image Step 13: Add Vision Events Add a Roboflow Vision Events https://docs.roboflow.com/deploy/vision-events?ref=blog.roboflow.com block named log compliance event. Set the input image to inputs.image, Output Image to label empty zones.image, Event Type to Quality Check, Use Case to Planogram Compliance, and Result to inject empty violations.final status. With Vision Events configured, the final step is to wire up the outputs Step 14: Configure Outputs Open the Outputs block and add: output image from label empty zones.image, compliance status from inject empty violations.final status, violations from inject empty violations.final violations, and summary from parse compliance report.summary. With all outputs configured, the workflow is complete and ready to test. Image splits into detection and audit; outputs merge; Vision Events logs run. Results Test 1: FAIL The first test used a personal care shelf stocking Aveeno, Dial, Equate, and Olay body wash products. The workflow returned a FAIL with three violations. RF-DETR detected 7 empty shelf zones across multiple rows, marking each one with a red bounding box. Gemini flagged two brand mixing violations with Equate bottles in the Dial section and one placement issue with mismatched Dial soap pack sizes. Test 2: PASS The second test used a single-brand nutrition shelf with PediaPro 3-5 Years across two rows. The workflow returned a PASS with no violations. Gemini found no issues; the single-brand shelf is compliant. Planogram Compliance Detection Production Deployment The workflow runs as a REST endpoint out of the box. Pass a shelf image, and it returns output image, compliance status, violations, and summary as structured JSON ready to pipe into any audit tool, dashboard, or alerting system. A fail routes the violations list to a store manager or brand rep. A pass is logged, and the run moves on. Vision Events https://docs.roboflow.com/deploy/vision-events?ref=blog.roboflow.com stores every run automatically with the input image, output image, and result for review. For lower latency, deploy on an edge device using Roboflow Inference https://inference.roboflow.com/?ref=blog.roboflow.com to avoid sending images over the network on every run. Empty zone detections and brand mixing violations are separate in the output. Route empty zones to a restocking queue and brand violations to a compliance team. Build It Faster with the Roboflow Agent If you'd rather not add each block by hand, use Roboflow Agent https://app.roboflow.com/solutions/chat/new?ref=blog.roboflow.com . Instead of configuring blocks one at a time, you describe the pipeline you want in plain text and the Agent builds it for you. Here's an example: Planogram Compliance Detection Conclusion This workflow pairs a trained RF-DETR model with two Gemini calls to handle the two main planogram failure types. The detector localizes empty shelf zones with precise bounding boxes. Gemini handles brand mixing and wrong-zone placements without a hardcoded spec or store-specific configuration. The same pipeline works across product categories. Point it at a personal care shelf, a beverage aisle, or a nutrition section, and it runs without modification. Further Reading Cite this Post Use the following entry to cite this post in your research: Mostafa Ibrahim /author/mostafa/ . Jun 22, 2026 . Planogram Compliance Detection with Roboflow. Roboflow Blog: https://blog.roboflow.com/planogram-compliance-detection/