# GPT-6 Astra Is the Best Vision Model We Have Tested

> Source: <https://blog.roboflow.com/gpt-6-astra-vision/>
> Published: 2026-09-18 12:50:17+00:00

OpenAI focused much of the GPT-6 Astra release on computer use. Models operating computers need to locate interface elements, read visible text, and understand what each element does. Those requirements also improve the skills needed for computer vision.

To measure those capabilities, we ran Astra through the [Roboflow Vision Evals](https://playground.roboflow.com/evals?ref=blog.roboflow.com). The benchmark covers common vision tasks, including object detection, counting, and visual reasoning. We also tested capabilities outside the benchmark, including box prompting, segmentation, re-identification, and robot control.

Astra is the strongest vision model we have tested so far. You can try Astra for free in [Roboflow Playground](https://playground.roboflow.com/?ref=blog.roboflow.com) and compare its results with other models on your own images.

[Roboflow Playground](https://playground.roboflow.com/object-detection/gemini-3-5-flash-vs-gpt-5-6-sol-vs-claude-fable-5?ref=blog.roboflow.com)

## Object Detection

Astra is the strongest detector we have tested in [Roboflow Playground](https://playground.roboflow.com/?ref=blog.roboflow.com). At low reasoning effort, it scores 82.1% mAP@50, 5.4 points ahead of Qwen3.8 Max and 13.7 points ahead of GPT-5.6 Sol. No competing model reaches that score even at high reasoning effort.

Astra combines attention to small details with an understanding of what the requested classes mean. In the LEGO example, it separates 4×1, 4×2, and 3×2 bricks across different colors, rotations, and partial overlap, scoring 99.8% mAP@50.

For most datasets, Astra is a strong starting point for auto-annotation. It understands common categories and produces boxes that are often more accurate than manual labels. In most cases, people will mainly need to review the results and correct the few mistakes Astra makes. The exception is projects with object classes that rarely appear in public data and require knowledge specific to the company or project.

Astra is now the default model powering [Roboflow Auto Annotate](https://docs.roboflow.com/annotate/use-roboflow-auto-annotate?ref=blog.roboflow.com). In the example below, we use it to split basketball players into Celtics and Knicks classes using team names alone. The same prompt works when the arena, court, lighting, uniforms, and players on the floor all change.

### Prompting with Text

Astra is a general vision-language model, rather than a standard object detector. We provide an image and a list of classes we are looking for, then ask it to return a JSON list of detected objects and their bounding boxes. We resize large inputs so their longest side is at most 2,048 pixels, a workaround we introduced after finding that [Sol could produce unstable coordinates on larger images](https://blog.roboflow.com/openai-gpt-5-6/). The boxes use absolute pixel coordinates measured on the resized image. Roboflow converts the response into standard detections and maps the boxes back onto the original image.

The class names can describe concepts that are much richer than a color or shape. We asked Astra to separate empty and full gas cylinders without telling it that the difference appears in a small plug on each valve. The result combines attention to detail with the ability to connect that detail to the class of the whole object. We also requested five jelly bean flavors without telling the model which colors or patterns belong to each flavor. To solve that task, Astra had to use what it already knew about how those flavors look.

### Prompting with Boxes

Box prompting lets us define the target with visual examples instead of a class name. We first observed this capability in Qwen3.8 Max, and Astra can do it as well. Positive boxes show what to find, while negative boxes show what to ignore. Astra returns bounding boxes for the remaining matches.

More examples help Astra understand which visual details define the target. With one positive die, the model detected objects across several dice classes. Three positive and three negative boxes narrowed the result from 49 predictions to nine, increasing mAP@50 from 21.3% to 90.1%. The football example follows the same pattern and improves from 46.9% to 100.0%. Green boxes show examples, red boxes show counterexamples, and blue boxes show the resulting detections.

Box prompts can also transfer between images. We provide positive examples in a reference image and ask Astra to find matching objects in a new image. In our tests, we marked white tablets in one image, and Astra found yellow-and-white capsules in another. We also marked bottle caps in one conveyor video and found them in footage recorded a week later. Both examples achieved 100.0% mAP@50.

## Segmentation

Prompting Astra for segmentation works much like prompting it for object detection. The main difference is that we request polygon vertices instead of bounding boxes. Astra returns the vertices in structured JSON, which Roboflow converts into masks. We tested the same images with SAM 3 to compare language understanding and mask accuracy.

The biggest advantage Astra has over SAM 3 is its language understanding. SAM 3 missed all three tire-size markings because it did not understand which text we wanted. It also produced masks for the car image but assigned several of them to the wrong classes. Astra found and labeled the correct regions in both examples.

The carrot comparison shows the cost of representing a mask with generated polygon points. Astra finds the target objects, but some outlines are shifted and others simplify the carrot edges. SAM 3 produces masks with more boundary detail and follows the objects more closely. This makes it the stronger option when exact mask shape matters.

Generating polygon coordinates with Astra is also slower and more expensive than running SAM 3. For the best result, we can use Astra to understand the class and return bounding boxes, then pass those boxes to SAM 3. Astra selects the correct objects, while SAM 3 produces masks that follow their boundaries more closely.

## Object Counting

Counting also depends on understanding which objects belong to the requested class. Astra scores 80.2% at low effort and 81.1% at high effort in our counting evaluation, up from Sol’s 74.3% and 76.1%.

We asked Astra to count 12 metal brackets overlapping inside a parts bin and 22 empty slots in a contact-lens display. Both answers were correct. The first task requires separating objects whose edges overlap; the second requires recognizing where an item is missing.

## Visual Reasoning

The ability to connect a small visual detail with its meaning also helps Astra answer questions that require several pieces of information. It leads our visual-reasoning evaluation with 87.2% accuracy at low effort and 91.2% at high effort. The nearest competitors score 82.1% and 84.5%, respectively.

In the technical drawing, Astra calculated a distance that was not labeled directly, returning 2.742 from the dimensions shown. Solving it requires reading the values and understanding which lines and centers they refer to. In the container example, it correctly answered that a 28,000 kg load plus another 900 kg would go over the printed weight limit of 28,800 kg. The task combines reading text, choosing the relevant weight limit, and doing the math.

## Re-Identification

In our earlier basketball AI tutorial, we combined detection, segmentation-based tracking, team assignment, and jersey-number recognition. Getting these stages to work together required tuning, including adapting team clustering to the players in each game. Errors in one stage could affect the stages that followed. We also treated every play separately and never attempted to preserve player identities between plays.

We wanted to see whether Astra could connect those separate plays. We sampled the video at one frame per second and sent the frames as images, asking the model to detect players and assign consistent IDs across the sequence. In the tested clips, Astra kept the same IDs when players appeared in later plays and handled substitutions between them. The video below shows those sampled frames, so the one-frame-per-second playback reflects the input we gave the model.

We also asked Astra to assign teams and read jersey numbers from the sampled frames. It could combine evidence across the sequence, where a number hidden in one frame may be visible in another. In this experiment, those tasks worked without the separate team-clustering and number-recognition stages used in our earlier pipeline. The second visualization shows team colors and jersey numbers next to the player boxes.

The cost makes processing every video frame with Astra impractical. We observed roughly $0.02–$0.08 per frame, which would amount to $36–$144 for one minute of video at 30 frames per second. Our combined pipeline uses local detection and tracking between Astra calls, with Astra handling the cross-frame interpretation. Models such as [RF-DETR](https://github.com/roboflow/rf-detr?ref=blog.roboflow.com) and trackers such as [BoT-SORT](https://github.com/NirAharon/BoT-SORT?ref=blog.roboflow.com) can handle the frequent updates, while Astra supplies identities and team information from sampled frames.

The same visual reasoning can also guide physical actions. In a vision-language-action setup, a model receives camera images and a language instruction, then uses robot-control tools to act on what it sees. New images provide feedback for the next movement. These experiments connect a general VLM to a robot controller, so the model still depends on the controller to carry out movements.

Thijs connected Astra to an SO-101 arm, a brush, and a camera, then asked it to paint the Golden Gate Bridge. The model planned movements and used camera feedback to check its progress. Thijs gave it starting reference points and feedback between attempts, and Astra updated its instructions and adjusted the calibration as it went. The result improved over several runs, showing what was possible with both visual feedback and human guidance.

MrDee reported a different result in a robot-drawing experiment. Astra and Grok failed in his setup, while Fable 5.1 completed a drawing from a reference image. His setup included guiding the arm to the four corners of the paper, showing where the pen touched the paper, and describing the positions of two cameras. He also accounted for the pen length, gripper angle, vibration, and arm sag. These details show how much a robot experiment still depends on the physical setup around the model.

## Trade-offs

Astra delivers the best detection result, but it is one of the most expensive models in our evaluation. At low effort, Qwen3.8 Max reaches 76.7% mAP@50 compared with Astra’s 82.1%, while costing roughly four times less per image. For many datasets, that difference may not justify the additional cost. We recommend testing a cheaper model first and moving to Astra if the results still require too many manual corrections.

These models are not intended for real-time image processing. In our detection runs at low effort, Astra takes about 11 seconds per image, while Qwen3.8 Max takes roughly 30 seconds. For auto-annotation, users can start a job and leave it running until the dataset is processed. A slower model may be worth the wait if it saves enough money.

Higher reasoning effort does not always provide enough improvement to justify its cost. For Astra, detection increases from 82.1% to 83.6% mAP@50, while the cost rises from $0.050 to $0.101 per image. Latency also increases from 11 to 32 seconds. We recommend starting with low effort and switching only when testing shows a meaningful improvement for the target dataset.

## Takeaways

Astra is the strongest vision model we have tested so far. It can detect objects from text or visual examples, generate segmentation polygons, interpret relationships within a scene, and re-identify objects across video. These capabilities make it useful for tasks ranging from auto-annotation to complex video analysis.

Astra’s flexibility does not make it the best choice for every part of a workflow. SAM 3 follows object boundaries more closely, and dedicated detectors and trackers process video at much lower cost. Human input is also important for classes whose meaning cannot be learned from public information.

The next test is your own dataset. Try the classes you actually need, including the images that usually take the most time to label, and check how much work remains after the model responds. You can run that comparison for free in [Roboflow Playground](https://playground.roboflow.com/?ref=blog.roboflow.com).

### **Cite this Post**

Use the following entry to cite this post in your research:

[Piotr Skalski](https://blog.roboflow.com/author/skalskip/). (Sep 18, 2026).
      GPT-6 Astra Is the Best Vision Model We Have Tested. Roboflow Blog: https://blog.roboflow.com/gpt-6-astra-vision/