Run inference in Codex with the Roboflow plugin: find a trained model, run detection or segmentation on your images, and visualize the results.
Running inference means applying a trained model to an image to generate predictions. Codex is OpenAI’s coding agent: you describe a task in plain language, and it writes and runs the code for you. With the Roboflow plugin installed, Codex can find a trained model in your Roboflow workspace or on Roboflow Universe, run inference, and visualize the results, all from natural-language prompts.
With the Roboflow plugin in Codex, you can select a trained model, run inference, and visualize the results using natural-language prompts.
In this tutorial, we will start with a simple example using an existing object detection model from a Roboflow workspace. We will then explore two RF-DETR examples with more detailed visualizations and analytics.
Tutorial and model source
Application
Model
Output
- Manufacturing inspection Own Roboflow workspace
Ceramic tile defect inspection
RF-DETR Small
Prediction table and annotated image
- Manufacturing inspection Roboflow Universe
Steel sheet surface defect detection
RF-DETR Object Detection
Bounding boxes and defect analytics
- Warehouse inspection Roboflow Universe
Pallet segmentation
RF-DETR Instance Segmentation
Instance masks and area analytics
Before starting, open Codex with the Roboflow plugin installed and connected. Keep an original test image ready for each example.
How to Install and Connect the Roboflow Plugin in Codex
The Roboflow plugin connects Codex to your Roboflow account, so Codex can access your projects, trained models, and public models on Roboflow Universe. Setup takes a few minutes.
Step 1: Add the Roboflow plugin
There are two ways to install the Roboflow plugin:
Codex App
Codex CLI
Open the codex app and on the left pane click the Plugins icon then in search plugins text field type "Roboflow". You will see a Roboflow plugin as a result. Click on it.
Alternatively to install it via Codex CLI, open your terminal and run codex. Inside Codex, enter /plugins to open the plugin browser. Find the Roboflow plugin by typing "Roboflow" and hit enter to install it. If it is installed but disabled, highlight it and press Space to enable it. Complete any connection prompts, then start a new Codex session before using the plugin.
Step 2: Sign in to Roboflow
The first time the plugin calls Roboflow, Codex asks you to sign in with your Roboflow account. Approve the requested permissions, and you’re connected. You don’t need to create or paste an API key.
If you’re running Codex somewhere a browser sign-in isn’t possible, such as a headless server, you can connect with a Roboflow API key instead. The plugin’s GitHub repository explains how to set this up. Never paste a private API key into the chat.
Step 3: Confirm the connection
Run a quick test prompt:
@Roboflow List the projects in my workspace.
If Codex returns your projects, the plugin is ready. Keep an original test image ready for each example below.
How to Run Inference from Codex with Roboflow
We will begin with a ceramic tile defect inspection model already available in my Roboflow workspace.
Tutorial 1: Run Inference Using an Existing Object Detection Project
This example follows three steps:
find a project with a trained model,
run inference, and
display the annotated image.
Step 1: Find your projects and trained models
Ask the plugin to list object detection projects that already have trained models. Organizing the results by project makes it easier to compare the available models and select an exact model ID.
@Roboflow List the object detection projects in my workspace that have at least one trained model.
Present the results directly in this chat using clear Markdown formatting.
Start with a short summary showing:
- Workspace name
- Number of matching projects
- Total number of trained models found
Then organize the results into a separate section for each project:
### [Project name — clickable project link]
- Project ID: `exact project ID`
- Project type: Object Detection
- Project image count: number of images
Under each project, show its trained models in a compact table:
| Model name | Exact model ID | Model architecture |
|------------|----------------|--------------------|
For model architecture, include the model family and size where available, such as RF-DETR Small or YOLOv8s.
Include all trained models under their corresponding project. Distinguish the project image count from dataset-version image counts.
Use exact IDs returned by Roboflow. Mark unavailable information as “Not available.”
Keep everything readable within the chat; do not create a separate file or dashboard.
The response groups trained models under their corresponding projects, with project details and clickable links. You should see something like this:
Select a model suitable for your test image. For this example, we will use the RF-DETR small model ceramic-tile-defects-v7pui-1-rfdetr-small-t1 from the ceramic-tile-defects project. When using your own project, copy the exact model ID returned by the plugin.
Step 2: Attach an image and run inference
Attach the original image you want to inspect and then specify the selected model ID in your prompt as follows.
@Roboflow Use ceramic-tile-defects-v7pui-1-rfdetr-small-t1 to run inference on the attached image with a confidence threshold of 0.40.
Show the detected classes and total detection count. Include a table with each detection's number, confidence score, and bounding box center coordinates in pixels.
Also report the confidence threshold and model inference time, if available.
I have attached the following input image with the prompt,
and received the following response:
The response reported a 40% confidence threshold and 0.760 seconds model inference time with total 19 detections for different defect classes.
These values describe this particular run. Predictions and inference time can differ for another image or model.
Step 3: Display the annotated image
The prediction table tells you what the model detected. To inspect where those detections appear, ask Codex to draw them on the original image.
@Roboflow Show the annotated image here using the predictions from the previous inference run.
Draw the returned bounding boxes on the original image. Use a different color for each detection and number the boxes to match the prediction table.
Add a compact dark legend in the top-right corner showing each detection's number, class name, and confidence score. Include the model ID below the legend.
Render and display the finished image directly in the conversation using the existing predictions.
You will see the following output image:
You have now selected an existing trained model, run inference, and inspected its predictions visually using prompts.
Tutorial 2: Run RF-DETR Object Detection for Manufacturing
Next, we will inspect manufactured steel sheet surfaces for defects such as scratches and stains. We will use a trained RF-DETR object detection model and ask Codex to create an annotated image with a visual analytics report.
Step 1: Find a trained RF-DETR detection model
Ask the plugin to find models that match both the manufacturing application and the required architecture.
@Roboflow Find up to 3 public trained RF-DETR object detection models on Roboflow Universe for manufacturing surface-defect inspection, preferably including scratches or stains.
For each candidate, show:
- Project name and Universe link
- Verified model architecture and task
- Exact inference model ID
- Classes supported by that trained model
Recommend a suitable model for a demonstration. Verify the selected model's architecture from its model or deployment information, rather than relying only on project tags.
If you cannot verify a detail, mark it as unverified.
I received the following response:
Review the classes and select a model that matches your inspection task. If you already have a trained RF-DETR manufacturing model, provide its project link and ask the plugin to check the same details.
Keep the exact model ID for Step 3. A project can contain multiple architectures, so verify the selected trained model before continuing.
Step 2: Provide a manufacturing test image
Attach an original image of the surface or component you want to inspect. Choose an image with a similar material, camera distance, and defect appearance to the selected model’s examples. Use an image without existing annotation overlays so the new predictions remain easy to review. You can also provide the full path to an image in a folder Codex can access. I'll be using the following image:
Step 3: Run inference through the plugin
Use the following prompt. Attach the image and specify the verified model ID from Step 1. I am using metal-defect-detection-d3wl6/11.
@Roboflow Run object detection on the attached image using the verified RF-DETR model metal-defect-detection-d3wl6/11.
Use the Roboflow plugin's direct model inference capability without creating a Workflow.
Use a confidence threshold of 0.40 as an initial setting.
First confirm that you can read the image. Then run inference and:
- Save the complete returned predictions as manufacturing-predictions.json.
- Preserve the model's original class names and bounding box coordinates.
- Show the model ID, image dimensions, threshold, total detections, and counts by class.
If the image cannot be accessed or inference fails, report the problem instead of generating example results.
The response should summarize the predictions returned by the model. The confidence threshold determines which predictions are retained. Here, 0.40 is an initial setting for exploration; the appropriate threshold depends on your model and inspection requirements.
Step 4: Visualize the predictions and defect analytics
Ask Codex to draw the returned boxes and create a report from the same predictions.
@Roboflow Use the same image and manufacturing-predictions.json to create a visual inspection report.
Use the Roboflow drawing skill and Python Supervision to render the actual predictions. Write and execute the necessary code, then show the finished visualization.
Include:
1. The original and annotated images side by side.
2. Bounding boxes with class names and confidence scores.
3. A consistent color for each defect class and a readable legend.
4. Summary cards for total detections, detected classes, and the confidence threshold.
5. A bar chart showing detection counts by class.
6. A table listing each detection's ID, class, confidence, and bounding box coordinates.
Use a clean gray layout with clear typography and enough space around the images and charts.
Align the boxes correctly with the original image dimensions. Derive every count and chart from the same predictions shown on the image.
Save the annotated image as manufacturing-annotated.png, the full report as manufacturing-report.png, and the detection table as manufacturing-detections.csv.
Display the annotated image and report, and provide download links. If there are no detections, show that clearly. Do not describe confidence scores as model accuracy.
You will receive the following response.
Click an image to enlarge it. The output of the report will be similar to this:
Review the annotated image for misplaced boxes, duplicate predictions, missed defects, and incorrect labels. The chart summarizes the predicted defect instances in this image. It does not establish a production defect rate. You now have an annotated manufacturing image, a report, and prediction files that you can inspect or compare with later runs.
Tutorial 3: Run RF-DETR Instance Segmentation for a Warehouse
For the warehouse example, we will identify individual pallets and visualize their predicted shapes. Instance segmentation produces a separate mask for each detected object. This lets us inspect pallet boundaries and distinguish neighboring instances more precisely than a bounding box alone.
Step 1: Find a trained RF-DETR segmentation model
Ask the plugin specifically for an RF-DETR Instance Segmentation model trained on pallets.
@Roboflow Find up to 3 public trained RF-DETR instance segmentation models on Roboflow Universe for warehouse pallets.
For each candidate, show:
- Project name and Universe link
- Verified model architecture and task
- Exact inference model ID
- Classes supported by that trained model
Recommend a suitable model for a demonstration.
Verify that the selected model is an RF-DETR segmentation model that returns instance masks or polygons. Check the selected model's deployment information, because a project may contain several model architectures.
If you cannot verify a detail, mark it as unverified.
I received the following response:
Choose a model whose examples resemble your warehouse images. If you already have a trained RF-DETR segmentation project, provide its link and ask the plugin to identify the available model.
Continue once the architecture, task, and exact inference ID are confirmed. The model uses its trained classes; mentioning additional objects in a prompt does not add new classes to it.
Step 2: Provide a warehouse test image
Attach an original warehouse image containing pallets. Choose a view where pallet boundaries are reasonably visible. An image containing several pallets can help you inspect whether the model separates neighboring instances. You can also provide the full path to an image Codex can access. I'm using the following image.
Step 3: Run instance segmentation through the plugin
Now use the model ID from Step 1, attach the image from step 2, and run the following prompt.
@Roboflow Run instance segmentation on the attached image using the verified RF-DETR segmentation model pallet-nwpgy/1.
Use the Roboflow plugin's direct model inference capability without creating a Workflow.
Use a confidence threshold of 0.40 as an initial setting.
First confirm that you can read the image. Then run inference and:
- Save the complete returned predictions as warehouse-predictions.json.
- Preserve all returned mask or polygon data, class names, and confidence scores.
- Show the model ID, image dimensions, threshold, and instance counts by class.
- Confirm whether usable segmentation masks or polygons were returned.
Keep the full segmentation data in the saved file rather than printing every coordinate in the chat. If masks or polygons are missing, report that clearly.
You will see output similar to the following.
The response should identify the detected instances and confirm that segmentation data is available. Keep the complete prediction file. Codex will use the returned masks or polygons to draw the overlay and calculate image-based measurements.
Step 4: Visualize the masks and warehouse analytics
Ask Codex to create a report that connects each colored mask to its corresponding measurements. To visualize the results, use the following prompt:
@Roboflow Use the same image and warehouse-predictions.json to create a warehouse segmentation report.
Use the Roboflow drawing skill and Python Supervision to render the returned instance masks or polygons. Write and execute the necessary code, then show the finished visualization.
Include:
1. The original and segmented images side by side.
2. A distinct color for each instance, with a transparent mask fill and a clear outline.
3. Labels containing an instance ID, the original class name, and confidence.
4. Summary cards for total instances, confidence threshold, and combined predicted mask coverage.
5. A bar chart showing mask area in pixels for each instance.
6. A table showing instance ID, class, confidence, mask area in pixels, and mask area as a percentage of the image.
Use the same instance IDs and colors across the image, chart, and table. Match the clean gray layout of the manufacturing report.
Calculate areas from the actual masks at the original image dimensions. Calculate combined coverage from the union of the masks so overlapping pixels are counted once.
Label coverage as "percentage of image pixels covered by predicted masks." Do not describe it as warehouse floor occupancy or a physical area measurement.
Save the segmented image as warehouse-segmented.png, the full report as warehouse-report.png, and the instance table as warehouse-instances.csv.
Display the segmented image and report, and provide download links. If segmentation data is missing, explain the issue; do not substitute filled bounding boxes for masks.
You will get the following output.
Click on the report image to enlarge and view.
Review whether each mask follows the visible pallet boundary and whether adjacent pallets remain separate. Look for masks that extend into the floor, include surrounding objects, or miss part of a pallet.
The area chart compares predicted masks within this image. Camera perspective affects their apparent size, so a larger mask does not necessarily represent a physically larger pallet.
You now have a warehouse segmentation overlay and an analytics report built from the same predictions, with matching instance IDs for easy review.
Deploy Your Model After Testing in Codex
Running inference in Codex is a fast way to check how a model performs on your images. Once you’re happy with the results, the next step is to run the model in production, on live images or video from your application or cameras. Roboflow gives you two ways to get there.
Option 1: Call the model directly from your application
Ask the plugin to write the integration code for the model you tested:
@Roboflow Write Python code that sends an image to metal-defect-detection-d3wl6/11 using Roboflow's hosted inference API and prints the predicted classes and confidence scores. Read the API key from an environment variable.
Codex returns code you can add to your application, using the same model ID from your tests.
Option 2: Build and deploy a Workflow
If you need more than raw predictions, such as filtering by class, counting objects, drawing results, or sending alerts, build a Roboflow Workflow. A Workflow chains your model with other processing steps in a single pipeline. You can describe the Workflow you need to Codex, or build it in the Workflow Editor. When it’s ready, publish it, then click </> Use in the Workflow Editor to get deployment code for images, video files, webcams, and RTSP camera streams.
Choose where your model runs:
In the cloud: The Serverless Hosted API runs your model or Workflow on Roboflow’s infrastructure, so you don’t need to provision or manage a GPU.
On your own hardware: Run the same model or Workflow on your own machines with Roboflow Inference, including edge devices like NVIDIA Jetson and Raspberry Pi. This works well when images need to stay on-site or internet access is limited. Self-hosted inference is free on every Roboflow plan, including the free tier.
For deployments with many cameras or devices, Roboflow’s Deployment Manager lets you monitor devices, streams, and logs from one place.
Run Inference in Codex with the Roboflow Plugin
The Roboflow plugin provides a straightforward way to run inference from Codex using natural-language prompts. You can begin with an existing project, select a trained model, and display its predictions directly on an image.
The RF-DETR examples extend this process with manufacturing defect analytics and warehouse segmentation reports. Codex handles the visualization code, while you specify the model, image, and output you need.
Start by reviewing individual predictions visually, then evaluate the model on representative labeled images before relying on it for an operational task.
Try inference with Codex on your own project today.