Surface Defect Detection on Machined Metal Medical Parts

wpnews.pro

To automate metal surface defect detection, train an RF-DETR Small model on a Roboflow Universe metal surface dataset (66.5% mAP@50, 77.6% precision) and deploy it in a Workflow that boxes each flagged region. A Gemini 2.5 Pro block then reads only those detections and writes a Defect or Defect-Free inspection report onto the image, grounded in the detector output so it cannot invent new findings, which suits precision medical parts like bone screws and fixation plates.

Machined metal components are used throughout the medical device industry, from surgical instruments and orthopedic implants to bone screws, fixation plates, and dental implants. These parts often require strict quality standards because even small surface defects can affect product quality, downstream manufacturing processes, and regulatory compliance.

Surface defects can be introduced during machining, deburring, polishing, handling, or cleaning operations. Identifying these defects consistently through manual inspection can be challenging, particularly when manufacturing large volumes of precision components.

In this tutorial, we will build an automated surface defect inspection system using Roboflow. We will train an RF-DETR model to detect visible defect regions on machined metal parts, deploy the model in

, and use

Roboflow workflowsto generate inspection observations based on the detected defects.

Gemini 2.5 Pro## Surface Defect Detection on Machined Metal Medical Parts

The system can help identify:

Visible surface defect regions on machined metal parts
Parts that appear defect-free
AI-generated inspection observations based on detected defect locations

The final output is an annotated image containing defect detections alongside AI-generated inspection observations. Here's the workflow we'll build.

Step 1: Prepare the Dataset

For this tutorial, we use the Metal Surface Defect Detection dataset from

. The dataset contains metal surface images captured under industrial inspection conditions and annotated with multiple defect classes representing different types of surface abnormalities.

__Roboflow Universe__RF-DETR is trained using the original annotations, allowing the model to learn the visual characteristics and locations of the various defect types present in the dataset.

The dataset includes metal surfaces captured under different lighting conditions, orientations, and surface textures. Defect regions vary in size, shape, and appearance, helping the model learn to identify a broad range of visible surface defects.

Here are examples of the detection targets:

To begin, fork the dataset into your Roboflow workspace. Then navigate to the Train tab and select Custom Training. From the available architectures, choose RF-DETR and set the model size to Small.

Once the model architecture is selected, generate a new dataset version before starting training. Configure a 70/15/15 split for training, validation, and testing.

Enable the following preprocessing steps:

Auto-orientation
Resize (512×512)

These preprocessing steps help normalize images captured under different inspection conditions while providing a consistent input size for RF-DETR training.

Step 2: Train the RF-DETR Model

Once training begins, RF-DETR learns to localize visible defect regions directly from the annotated examples in the dataset. Unlike a traditional classification model that only predicts whether a surface contains a defect, RF-DETR identifies the location of defect regions using bounding boxes.

This localization capability is valuable for inspection workflows because it provides visual evidence of where potential defects occur. Rather than returning a single pass/fail prediction, the model highlights the specific regions that may require further review by inspectors or quality engineers.

After training completes, Roboflow provides evaluation metrics that help measure how accurately the model detects defect regions. These metrics allow us to determine whether the model is ready to be deployed in a workflow and used for downstream analysis with Gemini 2.5 Pro.

Step 3: Evaluate Metrics

Once training completes, review the model's detection performance on the test set. Our RF-DETR Small model achieved the following results on the surface defect detection dataset:

The model achieved 66.5% mAP@50, along with 77.6% precision, 61.6% recall, and a 67.3% F1 score. These metrics indicate that the model can identify a variety of surface defect regions across the test dataset while maintaining a relatively low false-positive rate.

The higher precision score suggests that most detected defect regions correspond to actual surface defects, reducing unnecessary inspections. The recall score indicates that some defects may still be missed, which is expected given the variety of defect types and appearances present in the dataset.

These metrics evaluate the surface defect detection model only. In the next steps, we will deploy the model in Roboflow workflows and use Gemini 2.5 Pro to generate inspection observations and classify parts as either defect or defect-free based on the detection results.

Step 4: Deploy to Workflows

After reviewing the model metrics, deploy the RF-DETR model in Roboflow Workflows. The workflow starts with one input image of a metal part. The model detects visible surface defect regions, and a Bounding Box Visualization block renders the detections on the image.

The annotated image is then passed to Gemini 2.5 Pro. Gemini does not perform its own defect detection. Instead, it reviews the RF-DETR annotations and generates a concise inspection report. A custom Python block formats the report, and a Text Display block overlays it onto the annotated image.

The final output is a single image containing both the defect detections and the inspection summary.

The workflow uses the following sequence:

To create the workflow, open the trained model deployment options and click Try Workflows. Use the Detect and Visualize template as the starting point.

Step 5: Configure the Gemini 2.5 Pro VLM Block

Add a Google Gemini block after the Bounding Box Visualization block. Configure Gemini to receive the annotated image, not the original image. This ensures Gemini reviews the same detection evidence that appears in the final output.

Use these settings:

Image: $steps.detection_visualization.imageModel: Gemini 2.5 ProTask Type: Open Prompt

Use this prompt:

You are reviewing the output of a surface defect detection system for machined metal medical parts.

The colored annotations and bounding boxes were generated by an RF-DETR object detection model.

Do not perform independent defect detection.

Do not search the image for additional defects.

Only analyze the defect regions that have already been identified by the detection model and highlighted with annotations.

Based solely on the provided detections:

* Determine whether the part should be classified as Defect or Defect-Free.
* If one or more defect annotations are present, classify the part as Defect.
* If no defect annotations are present, classify the part as Defect-Free.

Provide a concise inspection report using the following format:
Classification:
Defect or Defect-Free
Observations:
Briefly summarize the annotated defect regions and their approximate locations.

Do not mention defects that are not annotated.
Return only the inspection report.

This keeps the workflow grounded in the object detector output. Gemini adds a readable inspection context, but it does not invent new defect findings or override the RF-DETR detections.

Step 6: Format and Overlay the Inspection Report

Next, add a custom Python block after the Gemini block. This block cleans the Gemini response so it displays cleanly on the final image.

Use this code:

def run(self, gemini_output):
    text = str(gemini_output or "")
    text = text.replace("\r\n", "\n").replace("\r", "\n")
    lines = []
    previous_blank = False
    for raw_line in text.split("\n"):
        line = " ".join(raw_line.strip().split())
        if not line:
            if not previous_blank and lines:
                lines.append("")
            previous_blank = True
            continue
        lines.append(line)
        previous_blank = False
    cleaned = "\n".join(lines).strip()
    cleaned = cleaned.replace("Classification: ", "Classification:\n", 1) if cleaned.startswith("Classification: ") else cleaned
    cleaned = cleaned.replace(" Observations:", "\n\nObservations:")
    return {"formatted_report": cleaned}

The block preserves Gemini’s inspection report, removes extra whitespace, and keeps the Classification and Observations fields readable.

Finally, add a Text Display block. Use the bounding box visualization image as the base image and the formatted report as the overlay text.

Step 7: Test the Workflow

To test the workflow, click Run in the top-right corner of Workflows and upload an image of a machined metal part. The workflow will process the image through RF-DETR, generate defect detections, and produce an inspection report based on the detected regions.

The final output is a single annotated image containing:

RF-DETR defect detections
A Defect or Defect-Free classification
Gemini-generated inspection observations

Since the workflow relies on visual detections from a single image, the results should be treated as an inspection aid rather than a final quality decision. Image quality, lighting conditions, surface finish, and camera angle can all affect detection performance. Clear images with well-defined surface features generally produce the most reliable results.

Production Considerations for Medical Device Manufacturing #

This tutorial demonstrates a proof-of-concept workflow using a general metal surface defect dataset. A production deployment would be trained on images collected directly from the target manufacturing process and component types.

For medical device manufacturers, this may include surgical instruments, orthopedic implants, fixation plates, bone screws, or other precision-machined metal components. Training data should capture the range of surface finishes, lighting conditions, and defect patterns encountered during normal production.

The workflow can be integrated into an inspection station using industrial cameras positioned after machining, polishing, cleaning, or finishing operations. RF-DETR can automatically identify potential defect regions, while Gemini generates structured inspection observations based on those detections.

Inspection records, images, and defect reports can then be stored in quality management and traceability systems for auditing, review, and process improvement. Human inspectors can focus their attention on flagged parts rather than reviewing every component manually.

Build It Faster with the Roboflow Agent #

If you'd rather not add each block by hand, use Roboflow Agent. Instead of configuring blocks one at a time, you describe the pipeline you want in plain text and the Agent builds it for you. Here's an example: