# How to Distill Gemini Into an Edge Model

> Source: <https://roboflow.com/blog/how-to-distill-gemini-into-an-edge-model>
> Published: 2026-11-16 15:58:46+00:00

To distill Gemini into an edge model, have Gemini label your images, review and correct those labels, then train a small detector that runs next to the camera and does the routine inspections. In Roboflow, auto-label with Gemini through a class description, the Auto Label agent or an Open Prompt Workflow. Then train RF-DETR Small on the reviewed dataset, run it with Inference on your edge device, and route uncertain cases back to Gemini or a human reviewer.

Checking an assembly for missing screws takes more than counting detections. A model might miss one screw and mistake a reflection for another, returning the expected count while overlooking an incomplete product. A useful inspection system needs to locate the visible screws at each mounting position and flag images that need further review.

With Roboflow, you can use Gemini to help build the training dataset for that system, then train a compact RF-DETR (opens in a new tab) model to detect screws near the camera. In this approach to model distillation, Gemini generates candidate bounding boxes, you review and correct the annotations, and RF-DETR learns the specific detection task. At deployment, the smaller model handles routine detection, while routing logic sends selected difficult images to Gemini or a person for review.

In this guide, you’ll explore three ways to prompt Gemini for screw annotations: class descriptions in Roboflow Auto Label, conversational feedback with the Auto Label agent, and an Open Prompt Workflow that returns boxes for inspection in Python. You’ll then review the labels, train RF-DETR Small, test it on your edge hardware, and configure a fallback for uncertain results.

Prompting Gemini for boxes in a consistent format

Before generating labels, decide exactly what one object means. In our reference image, each screw annotation covers the full visible fastener region, including its outer exposed end and inner metal feature. One screw receives one box. This definition must stay the same across the dataset.

Paste the description below into Describe what it looks like (optional).

Gemini searches using that description, and Auto Label saves each resulting annotation as screw.

```
A vertically oriented metal screw at an upper or lower mounting position
on the product's central structure. Draw one box around the full visible
fastener region, including the outer exposed end and inner metal feature.
Keep the visible components of one screw inside one box.
Exclude empty positions, plastic ribs, circular plastic features, and
reflections. Label only screws that are visible. Do not add a box just
because a screw is expected at that position.
```

The expected count of four is an application rule. It should not force the teacher to produce four boxes when a screw is missing.

Option B: Use the Auto Label agent

You can also describe the labeling task through Chat with Auto Label agent. The agent builds a labeling Workflow from the model and classes and shows a preview. Use the conversation to refine what it should label. The saved annotations still need a class name, which remains screw in this example. See labeling with the Auto Label agent (opens in a new tab).

With Gemini 4 Carbon selected, open Chat with Auto Label agent and enter the following request.

```
Use Gemini 4 Carbon to label the screws in these assembly-station images.
Use one class named screw. Draw one bounding box around each full visible
fastener region, including its outer exposed end and inner metal feature.
Keep the components of one screw together in one box.

The complete assembly has four screw positions, but label only screws
that are visible. Do not add boxes for empty positions. Exclude plastic
ribs, circular plastic features, and reflections. Show a preview first.
```

Check the preview and request specific corrections. You can use the following correction when that error appears.

```
The lower-left screw is missing a box. Include its full visible fastener
region. Keep the class name screw and show the updated preview.
```

You can also click an incorrect label or use Flag missing object. Once the preview matches the annotation rule, choose Use this Workflow, confirm the output-class mapping to screw, and select Label batch. Review the job estimate before starting. Continue with the label review in Section 2.

This conversation configures the Auto Label Workflow. Gemini 4 Carbon remains the requested annotation model. Check that the saved Workflow uses it. Either setup option produces the same kind of training data for RF-DETR Small. The training examples contain reviewed screw bounding boxes.

Option C: Use Open Prompt in a Roboflow Workflow

For this option, you can choose any Gemini model available in the Roboflow Google Gemini block (opens in a new tab) that supports image input and Open Prompt. You can also try different available models with the same prompt and images to compare their bounding boxes.

Understand Gemini's normalized box format

For the Auto Label process above, Roboflow manages the saved annotations. Use the description field or the Auto Label agent conversation above. Neither option requires you to enter a JSON-format prompt or convert coordinates yourself.

If you inspect raw Gemini detection output, Google's official object detection documentation (opens in a new tab) specifies box_2d in [ymin, xmin, ymax, xmax] order, normalized to 0–1000. Normalization expresses a position relative to the image dimensions. A value of 500 is halfway across the relevant dimension, regardless of image size.

Use the following conversion for an image of width W and height H.

```
xmin_pixels = xmin × W / 1000
xmax_pixels = xmax × W / 1000
ymin_pixels = ymin × H / 1000
ymax_pixels = ymax × H / 1000
```

For example, [100, 200, 300, 250] on a 1000-pixel-wide, 600-pixel-high image gives pixel bounds xmin=200, ymin=60, xmax=250, ymax=180. This is a coordinate example, not a measured screw prediction.

Build the Workflow

Prompting Gemini for boxes in a consistent format gives the visualization code a clear schema to check. Gemini models can still format their answers differently even when they receive the same instructions. This Open Prompt test uses a consistent box_2d schema and a model-independent parser. The prompt requests the screw label and [ymin, xmin, ymax, xmax] coordinates normalized to 0–1000. The script accepts known JSON formatting differences, but checks the coordinates before drawing them. This follows Gemini’s object-detection coordinate convention (opens in a new tab). The parser uses the requested coordinate order and scale. Numeric checks alone cannot prove that Gemini followed those instructions, so inspect the drawn boxes.

Build this small test separately from Auto Label. Use an image input, a text input for the prompt, one Google Gemini block, and two outputs. Follow Build a Workflow (opens in a new tab).

Create a Workflow with an image input named image and a text parameter input named prompt. Keep these names exactly as written so the Python code can pass both values.

Add the Google Gemini block (opens in a new tab) and name it gemini_boxes. Connect images to $inputs.image. Select Open Prompt (unconstrained). For the block’s prompt property, select $inputs.prompt as an input reference rather than entering fixed text. Select any available Gemini model that can analyze the image. Use settings supported by that model and leave model-specific options at their defaults if unsure.

For an editor test, paste the exact instructions below into the Workflow’s prompt input. The Python script uses the identical text in its PROMPT variable.

Create two Workflow outputs. Name the first boxes_json and select $steps.gemini_boxes.output. Name the second prompt_echo and select $inputs.prompt. The first returns Gemini’s raw answer and the second echoes the received prompt. Python checks prompt_echo == PROMPT before parsing. This verifies the Workflow input, but you must still confirm that the Gemini block’s prompt property is actually connected to $inputs.prompt.

```
Detect visible screws at the upper and lower mounting positions on the
product's central structure. Use one box for each full visible fastener
region, including its outer exposed end and inner metal feature.
Exclude empty positions, plastic ribs, circular plastic features,
and reflections. Do not add a screw because it is expected there.

Return only a JSON list without Markdown fences or extra explanation.
Use the key "label" with the value "screw" for every object.
Use the key "box_2d" with [ymin, xmin, ymax, xmax] coordinates.
Use integers normalized to 0–1000 relative to the full image.
Return [] if no screw is visible.
```

Test with an assembly image and supply the example text in the Workflow’s prompt input. Confirm that both outputs, boxes_json and prompt_echo, appear in the test response. Save the Workflow and use Deploy (opens in a new tab) to copy its workspace slug, Workflow ID, and API URL for Python.

Open Prompt returns raw text, so it may contain a JSON array, a Markdown JSON code block, or an object with a boxes field. The same model may return a string nested under boxes_json in a Workflow response. The script below removes recognized wrappers, parses one JSON result, and validates each box without depending on a model name. It also accepts recognized Gemini text-part wrappers and small amounts of text around one JSON value. The parser rejects malformed boxes and ambiguous output fields. Arrays must use box_2d. Alternate box fields must name their coordinates explicitly.

Visualize the returned screw boxes

Install the Inference SDK, Pillow, and NumPy in your Python environment.

Shell

```
python -m pip install inference-sdk
```

Set ROBOFLOW_API_KEY in the same terminal using the commands in Section 4. Save the following code as visualize_gemini_boxes.py and put an input image beside it as inspection.jpg. Replace the workspace slug and Workflow ID. The example uses the hosted API, but you should use the API URL shown by Deploy. Make sure that gemini_boxes.prompt uses $inputs.prompt, and that the Workflow returns prompt_echo from $inputs.prompt. Otherwise, the script will not have evidence that the intended prompt was received.

Python

```
"""Run a Roboflow Gemini Open Prompt Workflow and draw validated screw boxes.

The parser depends on a documented JSON schema, not on a Gemini model name.
The Workflow needs two outputs: boxes_json and prompt_echo.
"""

import json
import math
import os
import re
from pathlib import Path

from PIL import Image, ImageDraw, ImageOps

# Configure the Gemini block's prompt as $inputs.prompt (not hardcoded text).
# Use these same instructions for the editor's prompt input during testing.
PROMPT = """Detect visible screws at the upper and lower mounting positions on the
product's central structure. Use one box for each full visible fastener
region, including its outer exposed end and inner metal feature.
Exclude empty positions, plastic ribs, circular plastic features,
and reflections. Do not add a screw because it is expected there.

Return only a JSON list without Markdown fences or extra explanation.
Use the key "label" with the value "screw" for every object.
Use the key "box_2d" with [ymin, xmin, ymax, xmax] coordinates.
Use integers normalized to 0–1000 relative to the full image.
Return [] if no screw is visible."""

MAX_TEXT = 1_000_000
MAX_DEPTH = 15
MAX_BOXES = 1000
WRAPPER_KEYS = (
    "boxes_json", "boxes", "detections", "objects", "predictions",
    "output", "output_text", "response", "result", "results",
    "data", "text", "message", "content", "candidates", "parts",
)

def _parse_text(text):
    """Accept JSON, one JSON fence, or prose surrounding one JSON value."""
    source = text.strip().lstrip("\ufeff").strip()
    if not source or len(source) > MAX_TEXT:
        raise ValueError("Gemini response is empty or too long")
    try:
        return json.loads(source)
    except json.JSONDecodeError:
        if source.startswith(("[", "{")):
            raise ValueError("Malformed JSON or multiple JSON values")

    fenced = list(re.finditer(r"```(?:json)?\s*([\s\S]*?)\s*```", source, flags=re.I))
    if fenced:
        if len(fenced) != 1:
            raise ValueError("Expected exactly one JSON code block")
        block = fenced[0]
        outside = source[:block.start()] + source[block.end():]
        if any(char in outside for char in "[]{}`"):
            raise ValueError("Ambiguous content outside the JSON code block")
        return json.loads(block.group(1).strip())

    # Parse from the first opening bracket. Never salvage an inner object
    # from a malformed outer array or silently choose among multiple results.
    openings = [p for p in (source.find("["), source.find("{")) if p >= 0]
    if not openings:
        raise ValueError("Gemini did not return a JSON array or object")
    position = min(openings)
    value, consumed = json.JSONDecoder().raw_decode(source[position:])
    if any(char in source[position + consumed:] for char in "[]{}"):
        raise ValueError("Multiple or malformed JSON values")
    return value

def _check_coordinate_metadata(item):
    """Reject declared pixel, 0-1, or XYXY coordinates rather than misplot."""
    for key in ("coordinate_system", "coordinate_units", "units", "coordinate_format", "box_format", "coordinate_order"):
        if key not in item:
            continue
        hint = str(item[key]).strip().lower().replace(" ", "")
        scale_01 = bool(re.search(r"(?<!\d)0(?:-|\.\.)1(?!\d)", hint))
        if ("pixel" in hint or "xyxy" in hint or "xmin,ymin" in hint
                or scale_01 or "[0,1]" in hint):
            raise ValueError(f"Unsupported coordinate format specified in {key}: {item[key]!r}")

def _unwrap(value, depth=0):
    """Remove common Roboflow, Gemini, and Open Prompt response wrappers."""
    if depth >= MAX_DEPTH:
        raise ValueError("Gemini response has too many nested wrappers")

    if isinstance(value, str):
        return _unwrap(_parse_text(value), depth + 1)

    if isinstance(value, list):
        if not value:
            return []
        if all(isinstance(item, dict) and (
            "box_2d" in item or "bbox_2d" in item or "bounding_box" in item
        ) for item in value):
            return value
        if len(value) == 1:
            return _unwrap(value[0], depth + 1)
        raise ValueError("Expected detections or one Workflow response, not multiple responses")

    if isinstance(value, dict):
        _check_coordinate_metadata(value)
        if value.get("error_status") is True:
            raise ValueError("The Gemini block reported an output-decoding error")
        if any(k in value for k in ("box_2d", "bbox_2d", "bounding_box")):
            return [value]
        if "parts" in value and isinstance(value["parts"], list):
            parts = value["parts"]
            if all(isinstance(p, dict) and isinstance(p.get("text"), str) for p in parts):
                return _unwrap("".join(p["text"] for p in parts), depth + 1)
        keys = [key for key in WRAPPER_KEYS if key in value]
        if len(keys) > 1:
            raise ValueError("Multiple output fields; cannot choose a result safely")
        if keys:
            return _unwrap(value[keys[0]], depth + 1)
        raise ValueError("No boxes_json, boxes, or recognizable Gemini output field")

    raise ValueError(f"Unsupported Gemini response type: {type(value).__name__}")

def _number(value, label):
    if isinstance(value, bool):
        raise ValueError(f"{label} must be a number")
    # Gemini occasionally returns numeric JSON strings. Accept only simple
    # numeric strings, not text, units, or expressions.
    if isinstance(value, str) and re.fullmatch(r"\s*[+-]?(?:\d+(?:\.\d*)?|\.\d+)\s*", value):
        value = float(value)
    if not isinstance(value, (int, float)) or not math.isfinite(value):
        raise ValueError(f"{label} must be a finite number")
    if not 0 <= value <= 1000:
        raise ValueError(f"{label} must be normalized between 0 and 1000")
    return value

def _coordinates(item, index):
    keys = [key for key in ("box_2d", "bbox_2d", "bounding_box") if key in item]
    if len(keys) != 1:
        raise ValueError(f"Detection {index} needs exactly one coordinate field")
    coords = item[keys[0]]
    if keys[0] != "box_2d" and not isinstance(coords, dict):
        raise ValueError("Alternate box fields must use explicitly named coordinates")
    if isinstance(coords, dict):
        # Named coordinates make the order unambiguous.
        aliases = (
            ("ymin", "xmin", "ymax", "xmax"),
            ("y_min", "x_min", "y_max", "x_max"),
        )
        for names in aliases:
            if all(name in coords for name in names):
                coords = [coords[name] for name in names]
                break
    if not isinstance(coords, (list, tuple)) or len(coords) != 4:
        raise ValueError(
            f"Detection {index} needs box_2d [ymin, xmin, ymax, xmax] "
            "or a bounding_box object with explicitly named coordinates"
        )
    vals = [_number(v, f"Detection {index} coordinate {i}") for i, v in enumerate(coords, 1)]
    ymin, xmin, ymax, xmax = vals
    if ymin >= ymax or xmin >= xmax:
        raise ValueError(f"Detection {index} has a reversed or empty box")
    return vals

def decode_boxes(response):
    """Normalize valid Gemini box JSON to [{label, box_2d}].

    Assumes the prompt's [ymin, xmin, ymax, xmax] ordering and 0..1000 units.
    Unrecognized schemas fail clearly rather than drawing incorrect boxes.
    """
    boxes = _unwrap(response)
    if len(boxes) > MAX_BOXES:
        raise ValueError("Unexpectedly large number of detections")
    normalized = []
    for index, item in enumerate(boxes, 1):
        if not isinstance(item, dict):
            raise ValueError(f"Detection {index} is not a JSON object")
        label = next((item.get(k) for k in ("label", "class", "class_name") if k in item), None)
        if not isinstance(label, str) or label.strip().casefold() != "screw":
            raise ValueError(f"Detection {index} has missing or unexpected label {label!r}")
        _check_coordinate_metadata(item)
        normalized.append({"label": "screw", "box_2d": _coordinates(item, index)})
    return normalized

def visualize(image, boxes, output_path):
    image = image.copy()
    width, height = image.size
    draw = ImageDraw.Draw(image)
    for item in boxes:
        ymin, xmin, ymax, xmax = item["box_2d"]
        left = min(width - 1, max(0, math.floor(xmin * width / 1000)))
        right = min(width - 1, max(left, math.ceil(xmax * width / 1000) - 1))
        top = min(height - 1, max(0, math.floor(ymin * height / 1000)))
        bottom = min(height - 1, max(top, math.ceil(ymax * height / 1000) - 1))
        draw.rectangle((left, top, right, bottom), outline="#6706CE", width=3)
        caption_top = max(0, top - 18)
        draw.rectangle((left, caption_top, left + 45, caption_top + 17), fill="#6706CE")
        draw.text((left + 3, caption_top + 2), "screw", fill="white")
    image.save(output_path)

def verify_workflow_response(results, prompt):
    """Check the same text reached the Workflow's named prompt input."""
    if not isinstance(results, list) or len(results) != 1 or not isinstance(results[0], dict):
        raise ValueError("Expected one Roboflow Workflow result for one image")
    result = results[0]
    if "prompt_echo" not in result:
        raise ValueError("Add a Workflow output named prompt_echo selecting $inputs.prompt")
    if result["prompt_echo"] != prompt:
        raise ValueError("Prompt mismatch. Check the Workflow's prompt input and its output binding")
    if "boxes_json" not in result:
        raise ValueError("Add a Workflow output named boxes_json selecting $steps.gemini_boxes.output")
    return result["boxes_json"]

def main():
    import numpy as np
    from inference_sdk import InferenceConfiguration, InferenceHTTPClient

    if not PROMPT.strip():
        raise ValueError("PROMPT must not be empty")
    image = ImageOps.exif_transpose(Image.open("inspection.jpg")).convert("RGB")
    client = InferenceHTTPClient(
        api_url="https://serverless.roboflow.com",
        api_key=os.environ["ROBOFLOW_API_KEY"],
    ).configure(InferenceConfiguration(api_key_transport="header"))

    results = client.run_workflow(
        workspace_name="YOUR_WORKSPACE",
        workflow_id="YOUR_WORKFLOW_ID",
        images={"image": np.asarray(image)[:, :, ::-1].copy()},
        parameters={"prompt": PROMPT},
        use_cache=False,
    )

    # Save the complete response and supplied prompt even if parsing fails.
    Path("gemini_workflow_response.json").write_text(
        json.dumps(results, ensure_ascii=False, indent=2, default=str), encoding="utf-8"
    )
    Path("gemini_prompt.txt").write_text(PROMPT, encoding="utf-8")
    raw = verify_workflow_response(results, PROMPT)
    Path("gemini_boxes.txt").write_text(
        raw if isinstance(raw, str) else json.dumps(raw, ensure_ascii=False, indent=2),
        encoding="utf-8",
    )
    boxes = decode_boxes(raw)
    Path("gemini_boxes_normalized.json").write_text(
        json.dumps(boxes, indent=2), encoding="utf-8"
    )
    visualize(image, boxes, "gemini_boxes_preview.png")
    print(f"Saved gemini_boxes_preview.png with {len(boxes)} boxes for review")
    if not boxes:
        print("Gemini returned no boxes. This does not prove that screws are absent.")

if __name__ == "__main__":
    main()
```

Run the script in the same terminal.

Shell

```
python visualize_gemini_boxes.py
```

Open gemini_boxes_preview.png to review each mounting position. The script saves the full result as gemini_workflow_response.json, the raw Gemini answer as gemini_boxes.txt, the submitted text as gemini_prompt.txt, and the checked detections as gemini_boxes_normalized.json. It stops if the prompt echo does not match or the JSON is malformed. use_cache=False bypasses the normal Workflow definition cache for these tests. Save Workflow edits before running the script again.

The parser handles direct JSON arrays, a single Markdown JSON fence, recognized nested wrappers, empty lists, and numeric strings. It checks labels, finite values, coordinate bounds, and box order. This is model-independent JSON parsing, not a guarantee that every Gemini model will detect the same objects or follow the schema. It can reject declared pixel or 0-1 coordinates, but cannot reliably identify an undeclared change in scale or order. Always compare the preview with the original image. An empty list does not prove that screws are absent.

This check helps separate formatting mistakes from localization mistakes before teacher labels are used for student training. If you do not need Open Prompt for this test, the current Google Gemini Workflow block also provides an Object Detection task that decodes model responses into predictions. This article keeps Open Prompt to show how a consistent prompting and JSON parsing work across different model selections. The test saves local inspection files. It does not add annotations to your dataset. Continue with either Auto Label option and review the labels before training. Using this workflow you may test how different models detect and label the image. I tested it using four Gemini models such as Gemini 2.5 Pro, Gemini 3.1 Pro, Gemini 3.5 Flash-Lite and Gemini 4 Carbon. The results are following:

Among these I found, Gemini 4 Carbon working best for my use case. For your use case there may be other cheaper model. You may also try other Gemini models available for the Google Gemini Block. This example workflow may further be enhanced to annotate the dataset.

Know the teacher's limits

A teacher can return a plausible object name and still place its box incorrectly. Consider the face-landmark comparison. Gemini 2.5 Flash Lite confuses side labels and places some regions away from the corresponding features. Its LEFT_EYE box sits above the visible eye, while RIGHT_CHEEK_CENTER appears high on the forehead. Gemini 3.1 Pro places the regions more closely around the facial features in this example.

This is a qualitative, single-image example. It does not establish a general accuracy ranking or measure Gemini 4 Carbon on the assembly task. The lesson for manufacturing is to check location as carefully as class. A screw label on a reflection, a missed mounting position, or two boxes on one fastener can produce the wrong inspection result.

Pro's higher token cost also matters when generating many labels. Google's published standard paid API rates, in USD per million tokens, provide a direct cost comparison.

Model

Input

Output, including thinking tokens

Gemini 2.5 Flash-Lite

$0.10 for text, image, and video

$0.40

Gemini 3.1 Pro Preview, prompts up to 200,000 tokens

$2.00

$12.00

Review the teacher on your own task

Run the Auto Label preview on complete and incomplete products. Use Change to inspect other images and Run again after revising the description. Gemini box labels do not provide an adjustable confidence threshold in this interface, so inspect their locations and coverage directly. See evaluating Auto Label results (opens in a new tab).

Check every visible screw, including the upper and lower mounting positions. Look for missed targets, duplicate boxes, incorrect extents, and false detections on the surrounding structure. A total of four boxes can hide a missed screw and an extra false detection.

Once the preview is satisfactory, choose Label batch, review the estimate, and run the job. Correct its labels in Roboflow Annotate. Add missed objects, remove false boxes, and resize boxes to match the reference rule. Add the checked images to the dataset.

Optionally use a cheaper dedicated annotator for common classes

A manufacturing dataset may contain common objects such as cartons, pallets, or people alongside product-specific components. If an existing dedicated detector recognizes the required common classes reliably and at lower measured cost, use it for those labels. Reserve Gemini for targets that need a custom visual description, such as the complete screw regions in this assembly. This is an option to compare, not a guaranteed saving.

A dedicated model is useful only if its learned classes and box definitions match the dataset. A face detector cannot become a screw detector by renaming its output. Compare candidate annotators on a small human-labeled sample and include correction effort in the cost comparison.

In Roboflow Auto Label, you can select a trained model from the same dataset or a saved Workflow compatible with the project type. Trained models use their learned classes rather than the description field. A Workflow requires output-class mapping before its labels are applied.

For this tutorial, Gemini 4 Carbon is the selected teacher for the screw regions on the product. Test its preview against the reference annotations before proceeding. The dedicated-annotator route is optional. If a suitable detector later becomes available, compare it with Gemini using the same images and annotation rule. Review either source before training, and record which annotator produced each batch so repeated errors can be traced.

Train RF-DETR and deploy on-device

The teacher has now supplied task-specific training examples. RF-DETR Small learns to predict the screw boxes directly from images. The expected count remains separate application logic.

Prepare the training and evaluation data

Split images by product or capture session to keep near-duplicates out of different splits. Use corrected teacher labels for training and independent human annotations for validation and testing. Include missing screws at different mounting positions. Keep the final test set separate from prompt changes and threshold selection.

Choose a single architecture, select RF-DETR, and choose Small.

Select the reviewed dataset version, or choose Current Dataset and check its splits.

Open Customize Training Recipe. Start with the Public checkpoint, review preprocessing and augmentation, and retain the supported defaults for a first run.

Check the estimated duration and credit cost, then select Start Training. Training cost depends on job duration.

Inspect precision, recall, and mAP in Training Results. Use Model Evaluation (opens in a new tab) to inspect per-class errors and threshold behavior. Automatic evaluations are available for versioned models trained or uploaded by paid users. Its confusion matrix includes validation and test images, so retain an untouched final holdout if you use those results for tuning. Then evaluate the assembly inspection task with the following checks.

Inspect validation predictions for missed screws, duplicate boxes, and detections on plastic features.

Choose the detector cutoff and fallback rules using validation images. A detector cutoff removes low-score predictions; a separate review threshold decides which returned predictions need more attention.

Freeze those settings and evaluate the held-out test images against human annotations. Use the same intersection-over-union matching rule when comparing teacher and student boxes.

Check the number and location of screws per product. Measure how often incomplete products are accepted and complete products are rejected or sent for review.

Measure latency and completed images per second on the deployment device after warm-up. Include image capture and downstream processing when timing the complete inspection system.

Four detections alone do not establish that every mounting position contains a screw. For a fixed camera, test a rule that requires one valid detection in each expected region. This is an application rule to validate with your images.

For the Docker installation, prepare Docker and the GPU support required by your device. Then install the Inference CLI and the Inference SDK (opens in a new tab).

Shell

```
pip install inference-cli inference-sdk
inference server start
```

The CLI selects the container for the machine and starts the local server on port 9001.

Set your Roboflow API key in the same terminal where you will run the script. Use this command in Windows Anaconda Prompt.

Shell

```
set ROBOFLOW_API_KEY=YOUR_ROBOFLOW_API_KEY
```

Use this command on Linux or macOS.

Shell

```
export ROBOFLOW_API_KEY="YOUR_ROBOFLOW_API_KEY"
```

Save the following as run_edge.py. Replace the model ID with the RF-DETR Small model ID you copied. Replace 0.20 with your validation-selected detector cutoff, and place a test image at inspection.jpg.

Python

``` python
import json
import os
from inference_sdk import InferenceHTTPClient, InferenceConfiguration

client = InferenceHTTPClient(
    api_url="http://localhost:9001",
    api_key=os.environ["ROBOFLOW_API_KEY"],
).configure(InferenceConfiguration(
    api_key_transport="header",
    confidence_threshold=0.20,  # Replace with your validated cutoff.
))

result = client.infer(
    "inspection.jpg",
    model_id="YOUR_RFDETR_SMALL_MODEL_ID",
)
predictions = result["predictions"]
screws = [p for p in predictions if p["class"] == "screw"]
expected_count = 4
screw_count = len(screws)

print(json.dumps({
    "expected_count": expected_count,
    "screw_count": screw_count,
    "count_matches": screw_count == expected_count,
    "predictions": screws,
}, indent=2))
```

This example assumes a clear image of one product in the fixed inspection view. It counts the returned screw predictions after the configured detector cutoff. count_matches is a count comparison, not a final assembly approval. Use the review rules in Section 5 before accepting the product. Missing or malformed responses must be handled as errors, not as a successful zero-screw inspection.

Run the script.

Shell

```
python run_edge.py
```

The SDK sends the image to the local server, which runs the student. The first request downloads and initializes the model. Wait for this to finish before measuring normal request latency. The SDK configuration guide (opens in a new tab) covers the confidence setting and header authentication, which requires Inference Server (opens in a new tab).

Test a sequence of representative images after warm-up. Record median and 95th-percentile latency, then measure sustained completed images per second. For the full camera application, include capture, queues, and downstream processing in the timing.

Additionally you can use benchmark_edge.py for measuring sequential local image requests. Put representative images in a folder, then run the following command.

Shell

```
python code/benchmark_edge.py test-images --model YOUR_COPIED_RFDETR_SMALL_MODEL_ID --confidence 0.20 --output edge-benchmark.json
```

Replace the model ID and set --confidence to the same cutoff used for evaluation. The output includes successful and failed request counts, median and 95th-percentile latency, and completed images per second. This benchmark measures requests to the server; it does not include camera capture or machine actions.

Keep Gemini as the fallback for hard cases

Distillation and routing work together. Distillation gives us a smaller model for the repeated task. Routing decides when that model's result needs a more expensive review.

This extends How to Route Images Between Cheap and Expensive AI Models. That tutorial runs RF-DETR first, validates its predictions, and uses routing signals to choose a downstream model or human review. Here, RF-DETR Small can finish routine screw inspections locally, while selected hard cases go to Gemini. We adapt the routing signals to screw count, detection confidence, and mounting position instead of copying the wood-defect classes or thresholds.

Student result

Route

Four valid screw detections satisfy the tested confidence and mounting-position rules

Record the local screw-presence result.

Unexpected count, low-confidence detection, or questionable box location in a usable image

Send the original image and detection hints to Gemini.

Invalid prediction data or an unusable image

Capture another image or request human review.

Gemini returns an invalid answer, remains uncertain, or disagrees with the evidence

Keep the inspection unresolved for human review.

Connect the fallback in Roboflow Workflows

Use the Routing Images tutorial for the branching, response parsing, and validation pattern.

Add a Google Gemini (opens in a new tab) block. Supply the original image, the expected mounting positions, and the student's detections as hints. Use structured answering for the review fields and validate the returned answer before acting on it.

In the fallback Gemini block, select Structured Output Generation (structured-answering), put the following text in prompt, and define the expected review fields in output_structure. Include the student detection hints and their coordinate convention in the prompt. Use the following review prompt.

```
Inspect the four screw mounting positions in this product image.
The detector boxes are hints and may contain errors.
Use image-left and image-right for all position names.
For upper_left, upper_right, lower_left, and lower_right, report
present, missing, or uncertain and describe the visible evidence. Do not infer a screw from the expected count.
If glare, blur, or occlusion prevents a decision, report uncertain.
```

The Auto Label teacher and this runtime reviewer are separate components. Selecting Gemini 4 Carbon during annotation does not create the fallback Workflow. Configure and evaluate the deployment-stage Gemini model separately.

Follow Deploy a Workflow (opens in a new tab) to run the detector and routing logic through local Inference. Gemini calls still require network access. If review cannot complete, retain an unresolved result for later or human review.

Finally, collect hard images for another labeling cycle. The Roboflow Dataset Upload block (opens in a new tab) can save images with Persist predictions set to false. Review their labels before adding them to training. This connects routing back to distillation. Difficult production cases help improve the next student model.

Conclusion

At a manufacturing inspection station, a small detector can handle the repeated task of locating screws while selected uncertain images receive further review. Consistent teacher annotations, independent evaluation, and clear fallback rules make this approach practical to test on the assembly line. Gemini helps build and improve the training data. RF-DETR small or nano runs the routine detection task near the camera.

Start with a small batch in Roboflow, check the teacher's boxes, and train an RF-DETR baseline. Use the routing pattern to keep hard cases on a review path and feed corrected examples into the next training cycle.
