Finding Edge Cases in a Computer Vision Dataset

Roboflow published a tutorial showing how to surface computer vision edge cases by running a trained model against unlabeled production images in a Roboflow Workflow that flags any image with no detections or a confidence below 0.60 and has Google's Gemini explain what made each flagged image hard. The tutorial's baseline model, trained on the Fabric Defect Detection dataset with a 70/20/10 train/validation/test split using Roboflow 3.0, reached 65.7% mAP@50, 81.0% precision, and 72.2% recall, then was tested against the separate Fabric Defects 5 Class dataset to expose distribution gaps. Flagged images are labeled in Roboflow Annotate and used to retrain, with results exported to a CSV for review.

Find your model's edge cases with a Roboflow Workflow that runs the model on unlabeled production images, flags anything with no detections or a confidence below 0.60, and has Gemini explain what made each flagged image hard. Label the flagged images in Roboflow Annotate and retrain. A model https://playground.roboflow.com/models?ref=blog.roboflow.com can pass validation with strong metrics and still fail once it hits production. Validation sets are built from the same distribution the model was trained on, so they typically don't capture the lighting changes, occlusions, or rare defect https://roboflow.com/solutions/defect-detection?ref=blog.roboflow.com types a camera on a real line https://ai1.roboflow.com/?ref=blog.roboflow.com eventually sees. Those gaps accumulate as missed or misclassified images. So today we'll take a look at how to find computer vision edge cases in production. This tutorial builds a Roboflow workflow that runs a trained model against unlabeled production images, flags the ones it struggles with, and asks a VLM to explain why. The results export to a CSV for review, and the loop closes with labeling and retraining. What Counts as an Edge Case? An edge case is any image that falls outside what a model reliably handles. In computer vision, that generally means: rare conditions the training data barely covered; hard negatives that look deceptively similar to what the model was trained to detect; and distribution gaps, differences in lighting, texture, or camera setup between training and production data. These matter more than overall accuracy suggests, since averages hide the small slice of real-world variation a model consistently fails on. One quick clarification: "edge case" here has nothing to do with edge devices, the Jetsons and Raspberry Pis a model might run on. Methods for Finding Edge Cases - Mining low-confidence and disagreement frames: flag predictions below a confidence threshold or images with no detections at all, what this tutorial's Workflow does directly. - Comparing train vs production distributions: run a model trained on one dataset against a separate, unseen dataset to surface a real distribution gap, demonstrated here by using two different fabric datasets. - Reviewing the confusion matrix off-diagonal: use Roboflow's Model Evaluation https://docs.roboflow.com/models/evaluate/evaluate-trained-models?ref=blog.roboflow.com to see which classes get confused with each other or missed entirely. - Clustering embeddings to find sparse regions: group images by visual similarity and look for small, isolated clusters that suggest underrepresented conditions. - Using a VLM to describe outliers: ask a vision-language model to explain what makes a flagged image hard, what Gemini does in this tutorial's Workflow. How to Find Edge Cases in a Computer Vision Dataset Let's get started. Fork the Fabric Defect Detection dataset https://universe.roboflow.com/muhammad-saad/fabric-defect-detection-8ndhb?ref=blog.roboflow.com to use as your training set. Then fork the Fabric Defects 5 Class dataset https://universe.roboflow.com/jaswant-oyc4f/fabric-defects-5-class-b2mwz?ref=blog.roboflow.com to use as unlabeled production images. The difference between these two images is the whole premise of this tutorial. The training set's hole sits on a plain, high-contrast background, easy to learn and easy to detect. The production set's holes are smaller and buried in a busy, multi-color pattern, the kind of gap that shows up later as low confidence or missed detections. Train the Baseline Model Generate a new dataset version from the training set using a 70/20/10 train/validation/test split. Click Custom Train and choose whichever model type fits your needs. For this tutorial, Roboflow 3.0 was selected. Once training finishes, review the model metrics. This baseline run reached 65.7% mAP@50, 81.0% precision, 72.2% recall, and 68.9% F1, numbers that look reasonable on paper but leave real room for missed and misclassified defects. The confusion matrix tells a more specific story. Hole predictions are clean: three correct with no false negatives, but four background regions got incorrectly predicted as holes. Knot has zero misclassifications between classes but four missed detections entirely, and Stain shows the same pattern with two missed detections. In other words, when the model does identify a class, it tends to get it right, but it's missing a meaningful share of knots and stains altogether. Build the Edge Case Mining Workflow In the steps that follow, we'll build a Workflow that runs a trained model on unlabeled production images, flags the ones it struggles with, and asks Gemini to explain why. Here's the workflow we'll build https://app.roboflow.com/workflows/embed/eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ3b3JrZmxvd0lkIjoiR3kwbU11TVNqUFNCaE84MnRkUTQiLCJ3b3Jrc3BhY2VJZCI6ImRXZ2VaNElWM0JOaDdYM1VQUE41TVMyaXEyeTIiLCJ1c2VySWQiOiJkV2dlWjRJVjNCTmg3WDNVUFBONU1TMmlxMnkyIiwiaWF0IjoxNzg4OTc5MzYwfQ.8WPzVWw2qNj8NYXzIMD4KB4TGoX 8dAxKQbYiizL6uk?ref=blog.roboflow.com . Here's what each block does: - Fabric Defect Model: Runs the trained model on the image and returns defect predictions. - Detection Count: Counts how many defects were found. Zero is treated as a potential edge case. - Minimum Confidence: Returns the lowest confidence score among the detections. - Edge Case Flag: Flags the image as an edge case if there are no detections or the minimum confidence falls below a threshold. - If Edge Case: Only lets flagged images continue to the Gemini branch, saving unnecessary VLM calls. - Edge Case Description: Asks Gemini to describe what might have made the image hard for the model. - Draw Boxes: Draws bounding boxes around the model's predictions. - Draw Labels: Adds class names and confidence scores to each box. - VLM Description or Blank: Uses Gemini's description when available, or an empty string otherwise. - Batch Export Fields: Normalizes the output types for the CSV export script. - Outputs: Returns the annotated image, edge case flag, detection count, minimum confidence, and VLM description. Step 1: Add the Fabric Defect Model Add a Model block and connect its image input to inputs.image. Select your trained Fabric Defect model. Step 2: Add Detection Count Add a Property Definition block named detection count . Connect its input to the Fabric Defect Model's predictions, and set the operation to count the number of detections. Step 3: Add Minimum Confidence Add a Property Definition block named minimum confidence , connected to the Fabric Defect Model's predictions, set to extract the lowest confidence value. With no detections, it returns 1.0 as a neutral default, since Detection Count already handles that case separately. Step 4: Add the Edge Case Flag Add an Expression block named edge case flag . Set the condition to output true when detection count equals 0 or minimum confidence is below 0.60, and false otherwise. Step 5: Add If Edge Case Add a Continue If block named if edge case . Set the condition to check whether edge case flag is true , and set the Gemini block as the next step. This way, the workflow only runs the VLM on images actually flagged as edge cases, saving unnecessary calls. Step 6: Add Edge Case Description with Gemini Add a Google Gemini block named edge case description , using Gemini 3.5 Flash Lite https://playground.roboflow.com/models/google/gemini-3-5-flash-lite?ref=blog.roboflow.com . Connect its image input to the original image, and write a prompt asking Gemini to review the flagged image and describe in 2 to 4 sentences what might have made it hard for the model, such as blur, poor lighting, unusual texture, subtle defects, occlusion, or a distribution shift from the training data. Tell it not to invent defects it can't actually see. Step 7: Add Draw Boxes Add a Bounding Box Visualization block named draw boxes . Connect its predictions input to the Fabric Defect Model's output. This draws a box around each detected defect on the image. Step 8: Add Draw Labels Add a Label Visualization block named draw labels . Connect its image input to draw boxes.image and its predictions input to the Fabric Defect Model's output. This adds the predicted class and confidence score to each box, and its result becomes the annotated output image . Step 9: Add VLM Description or Blank Add a First Non Empty Or Default block named vlm description or blank . Set it to use edge case description's output when available, falling back to an empty string when Gemini didn't run because the image wasn't flagged. Step 10: Add Batch Export Fields Add a Custom Python Block named batch export fields . Connect edge case flag and vlm description or blank as inputs, and have it return is edge case as a boolean and vlm description as a string. This keeps both fields in a consistent type for the CSV export script to read later. python def run self, is edge case, vlm description : flag = bool is edge case description = '' if vlm description is None else str vlm description return {'is edge case': flag, 'vlm description': description} This code makes sure the flag is a true boolean and the description is never None , just an empty string when Gemini didn't run. Without this step, small type mismatches between blocks could turn into formatting problems in the final CSV. Step 11: Configure Outputs Connect the outputs as shown below. This returns everything the CSV export script needs from every run of the workflow. Use Roboflow Agent You don't have to build this block by block. Roboflow Agent can take a plain language description of the pipeline you want and configure it automatically. Run the Workflow Against Production Images With the Workflow built, the next step is running it against a real batch of unlabeled production images. Two scripts handle this: one pulls a random sample from the production dataset, and the other runs the Workflow across that sample and writes the results to a CSV. Install the dependencies both scripts need, then export your API key as an environment variable: pip install -U inference-sdk roboflow requests pillow export ROBOFLOW API KEY="YOUR ROBOFLOW API KEY" Pull a random batch with download random 100.py python import os, random, shutil, requests from io import BytesIO from pathlib import Path from PIL import Image from roboflow import Roboflow KEY = os.environ "ROBOFLOW API KEY" ALLOWED = {"hole", "stain", "knot"} COUNT = 100 project = Roboflow api key=KEY .workspace "test-1eiqw" .project "fabric-defects-5-class-b2mwz-mgtfe" images = sum project.search all in dataset=True, limit=250, fields= "id", "url", "annotations" , eligible = for image in images: annotations = image.get "annotations" or {} classes = set annotations.get "classes" or {} .keys if classes and classes <= ALLOWED: eligible.append image if len eligible < COUNT: raise RuntimeError f"Only {len eligible } matching images found." selected = random.Random 42 .sample eligible, COUNT output = Path "fabric random 100" shutil.rmtree output, ignore errors=True output.mkdir for number, image in enumerate selected, 1 : response = requests.get image "url" , timeout=60 response.raise for status with Image.open BytesIO response.content as source: source.convert "RGB" .save output / f"{number:03} {image 'id' }.jpg", "JPEG", quality=95, print f"Downloaded {number}/{COUNT}" shutil.make archive str output , "zip", output print f"Created {output}.zip" Replace YOUR WORKSPACE NAME and YOUR PROJECT NAME with your own. This pulls every production image whose classes fall within hole, stain, and knot, samples 100 with a fixed seed for reproducibility, and saves them as JPEGs into a fabric random 100 folder, zipped for convenience. Move the images into a folder named input images , since that's what export edge cases.py reads from. Run the Workflow with export edge cases.py python import csv import os from pathlib import Path from inference sdk import InferenceHTTPClient, InferenceConfiguration INPUT DIR = Path "fabric random 100" OUTPUT CSV = Path "fabric edge cases.csv" API KEY = os.getenv "ROBOFLOW API KEY", "YOUR ROBOFLOW API KEY" client = InferenceHTTPClient api url="https://serverless.roboflow.com", api key=API KEY, .configure InferenceConfiguration api key transport="header" extensions = {".jpg", ".jpeg", ".png", ".webp"} images = sorted path for path in INPUT DIR.iterdir if path.suffix.lower in extensions if API KEY == "YOUR ROBOFLOW API KEY": raise RuntimeError "Set ROBOFLOW API KEY or replace YOUR ROBOFLOW API KEY." if not images: raise RuntimeError f"No images found in {INPUT DIR.resolve }" with OUTPUT CSV.open "w", newline="", encoding="utf-8" as file: writer = csv.DictWriter file, fieldnames= "image", "is edge case", "vlm description" , writer.writeheader for number, image path in enumerate images, 1 : print f" {number}/{len images } Processing {image path.name}" try: result = client.run workflow workspace name="test-1eiqw", workflow id="fabric-defect-edge-case-mining", images={"image": str image path }, use cache=True, output = result 0 if isinstance result, list else result writer.writerow { "image": image path.name, "is edge case": output.get "is edge case", False , "vlm description": output.get "vlm description", "" , } except Exception as error: print f" Error: {error}" writer.writerow { "image": image path.name, "is edge case": True, "vlm description": f"Workflow error: {error}", } file.flush print f"Finished. CSV saved to: {OUTPUT CSV.resolve }" Replace YOUR WORKSPACE NAME here too. This sends every image in input images through your Workflow and writes is edge case and vlm description to a row in fabric edge cases.csv . Failed calls still get logged as edge cases with the error message instead of being skipped. The resulting CSV Each row corresponds to one image, with is edge case showing whether the Workflow flagged it and vlm description holding Gemini's explanation for flagged rows, left blank for the rest. This file is what you'll filter and hand off for labeling in the next step. Review and Label the Flagged Edge Cases Open fabric edge cases.csv and filter for rows where is edge case is true . Each row's image column gives you the filename to look up inside your input images folder, so you can pull the actual flagged images out and set the rest aside. Upload these flagged images to your training project as a new batch. They arrive unannotated, ready for review. Open the batch in Roboflow Annotate and label each image manually, drawing boxes around any hole, knot, or stain you find. This is the step that turns a hard example into something the model can actually learn from. Once every image in the batch is labeled, add them to your dataset. These images now become part of the pool you'll train on in the next step. Checklist: Edge-Case Dimensions to Watch For A few dimensions worth watching for while reviewing flagged images: - Lighting: shadows, glare, or conditions the training set didn't capture - Occlusion: the object partially blocked or overlapping something else - Part or product variants: colors, patterns, or versions not seen in training - Camera drift: a shifted angle, distance, or resolution from training Retrain and Measure the Lift With the flagged edge cases labeled and added to your dataset, generate a new version. Roboflow will include the newly labeled images alongside your original training data in the split. Click Custom Train and select Roboflow 3.0 again, keeping the model type consistent with the baseline for a fair comparison. Once training finishes, review the new metrics. Every metric moved up from the baseline: mAP@50 climbed from 65.7% to 71.4%, precision from 81.0% to 86.5%, recall from 72.2% to 77.3%, and F1 from 68.9% to 81.5%. That's a real lift, and it came entirely from adding a small batch of the images the baseline model struggled with most. The confusion matrix shows where that lift came from: correct Hole detections rose from 3 to 16, Knot from 4 to 19, and Stain from 4 to 28, with far fewer missed detections across the board. Beyond a one-time retrain, Model Monitoring https://docs.roboflow.com/deploy/model-monitoring?ref=blog.roboflow.com can track confidence and class-level performance continuously once a model is live, with alerts that flag drift before it requires someone to manually run a batch like this to find it. Conclusion This tutorial built a loop for finding and fixing what a model gets wrong: mine unlabeled images for edge cases, review the flagged ones, label them in Roboflow Annotate, retrain, and measure the lift with Model Evaluation. Validation metrics only tell you how a model performs on data resembling what it trained on, real images rarely stay that well-behaved, and that gap is where edge cases live. Closing it isn't a one-time fix, it's a loop you run again every time the model meets something new. Further reading: Cite this Post Use the following entry to cite this post in your research: Mostafa Ibrahim https://blog.roboflow.com/author/mostafa/ . Sep 2, 2026 . Finding Edge Cases in a Computer Vision Dataset. Roboflow Blog: https://blog.roboflow.com/finding-edge-ases-in-a-computer-vision-dataset/