cd /news/computer-vision/what-is-model-distillation-for-compu… · home › topics › computer-vision › article
[ARTICLE · art-170170] src=roboflow.com ↗ pub= topic=computer-vision verified=true sentiment=· neutral

What Is Model Distillation for Computer Vision?

Roboflow published a guide describing a label-generation approach to model distillation for computer vision, in which a large teacher model labels images, a human reviews and corrects the annotations, and a compact student detector such as RF-DETR is trained on the reviewed dataset for local inference. The guide covers generating labels with a Gemini-powered Workflow or Roboflow Auto Label, then testing the student model against a device's accuracy, latency and power limits before deployment, aimed at cases like a battery-powered security camera that cannot send every frame to a cloud model. It distinguishes response-based, feature-based and relation-based knowledge and notes that teacher and student can use different architectures, such as a convolutional neural network teaching a transformer.

by read19 min views4 publishedNov 15, 2026
What Is Model Distillation for Computer Vision?
Image: Roboflow (auto-discovered)

When a large vision model is too slow or heavy for your edge hardware, use it as a teacher: have it label your images, review and correct the labels, then train a compact student model that runs locally without a cloud call for each frame. In Roboflow, you can generate those labels with a Gemini Workflow or Auto Label, train RF-DETR on the reviewed dataset, and test it against your device’s accuracy, latency and power limits before you deploy.

A large vision model can help you build a computer vision application even when it is too demanding to run on your deployment hardware. Consider a battery-powered security camera: it needs to detect faces quickly, with limited compute and memory. Sending every frame to a cloud model adds network latency and requires connectivity. A smaller detector that runs locally may be a better fit.

Model distillation transfers knowledge from a larger “teacher” model to a smaller “student” model. In this guide, we use a label-generation approach: the teacher predicts face bounding boxes, you review and correct those annotations, and a compact detector trains on the resulting dataset. The teacher helps prepare the training data; the student handles live detections.

With Roboflow, you can connect this process from annotation through deployment. We’ll walk through two ways to generate labels: a Gemini-powered Workflow and Roboflow Auto Label, then train a compact RF-DETR model and evaluate it for local inference. Along the way, we’ll cover how to compare labeling costs, check annotation quality, and test whether the student meets your camera’s accuracy, latency, and power requirements.

What is model distillation?

Model distillation, also called knowledge distillation, is a model compression and acceleration technique that transfers knowledge from a large teacher model to a small student model.

The teacher guides training, while the student learns the task for deployment. This knowledge transfer allows different models to handle training and deployment. They can also have different architectures (opens in a new tab). For example, a convolutional neural network can teach a transformer model.

A learning signal is information that helps the student update its parameters during training. The teacher can provide response-based knowledge, feature-based knowledge, or relation-based knowledge.

Response-based knowledge comes from the teacher’s final outputs.

Feature-based knowledge can come from feature maps in intermediate layers.

Relation-based knowledge captures relationships between different layers or data samples.

What does the teacher provide?

For this example, the teacher provides annotations for object detection. Each annotation gives the class face and a bounding box, a rectangle around the face. These are the training targets for the smaller student. Use an Object Detection project to store the images and boxes.

A pseudo-label (opens in a new tab) is a model prediction used as a training label for an unlabeled example. In hard pseudo-labeling for classification, it is the class with the highest predicted probability. In this object-detection example, the teacher predicts classes and bounding boxes. Treat them as suggested annotations and review them before adding them to the training dataset.

The student trains on image-annotation pairs. For example, an image may show three faces. Its annotations give each face the class face and its own bounding box. Training adjusts the student’s weights, the numerical parameters it learns, so it can predict objects in new images.

Generated labels and classical knowledge distillation

In classical soft-target distillation, the student learns from the teacher’s probability distribution across classes. For example, a classifier might assign 0.80 to person, 0.15 to vehicle, and 0.05 to background. Keeping all three values gives the student more information than keeping only the person label. Temperature (opens in a new tab) is a parameter that can soften this distribution, reducing the differences between class probabilities.

Hard-label distillation (opens in a new tab) uses the teacher’s selected class as a training target. The student learns from that class rather than the full probability distribution. DeiT’s hard-label objective uses two cross-entropy losses. One uses the ground-truth label, and the other uses the teacher’s selected class. Both have equal weight.

The practical labeling approach below uses model-assisted annotation followed by supervised training. The teacher’s predictions help build the student’s training dataset, and a person reviews and corrects them. This is the label-generation approach used here. It does not implement classical soft-target distillation or DeiT’s hard-label loss.

Why distill a model for a security camera?

Face detection locates faces in an image. In this example, the output is a set of bounding boxes with the class face. This output does not identify the person or determine whether a security threat exists. Any application response needs separate logic.

Edge computing places computation at the edge of the network, using resources between data sources and cloud data centers. Here, the camera itself is the edge device. Training can happen on a more powerful computer or cloud service. Roboflow Inference (opens in a new tab) supports local processing on compatible computers and edge devices. The deployment steps below distinguish testing on a computer from running the model inside the camera.

Fit the available compute and memory

Large models may need more compute and memory than an edge device can provide. A smaller student can help fit the task within those limits. This is one reason to use model compression.

Reduce latency

Latency is the time between receiving an input and producing a result. Running the student on the camera avoids sending each frame to a cloud model and waiting for its response.

The total response time also includes image decoding, resizing, and result handling. Measure these along with model execution in Step 6.

Work within a battery or power budget

A compact student can reduce the computation needed for inference, but battery life depends on the whole system. Inference means using a trained model to make predictions. Energy per inference is the energy used to process one input. Power is the rate of energy use. The full system uses energy to capture images and communicate, as well as for computation, memory access, and data movement (opens in a new tab). Distillation alone does not guarantee a fixed energy saving. Check system power and battery life on the actual hardware.

Avoid cloud model calls for live detections

Once the student runs locally, it can detect objects without sending each frame to the teacher. You may still use a cloud teacher when creating or updating the dataset. Labeling and live inference happen at different stages.

Local inference does not automatically make the whole application work without internet access. A locally hosted Workflow that calls a cloud VLM still needs a connection. Step 7 covers the separate requirements for fully offline operation.

Choose a teacher and training a student with Roboflow

Both paths below serve the same goal. A teacher generates face annotations, the annotations are reviewed, and a compact student learns from the resulting dataset. The teacher is used during dataset preparation; the student handles live detections.

Teacher option 1: A standalone Gemini 3.1 Pro Workflow

A vision-language model, or VLM, processes images together with text instructions. This path uses Gemini 3.1 Pro in the Google Gemini block (opens in a new tab). Configure its object-detection task to find face. The model supplies candidate face boxes without being fine-tuned for this camera.

For an example that covers labeling and training, see the Gemini Workflow section of YOLO Distillation. Its older configuration differs from the v6 block used here. Follow the current block settings below.

Teacher option 2: A teacher selected in Roboflow Auto Label

Auto Label (opens in a new tab) is the dedicated annotation feature used for this second path. It can use a foundation model or your own trained model as the teacher. An object detector predicts object classes and locations. Here, its face boxes become candidate training annotations for the student. The current choices and setup steps are listed below.

Step 1: Define the task and prepare the images

Use an Object Detection project with the class face. For Option 1, create a project (opens in a new tab) to receive the generated annotations. For Option 2, upload the images to the project you want to label. A foundation model can label a new project. To use your own trained detector, use the project containing that model.

Prepare a small sample that covers the scenes your camera will see. Include daytime and nighttime frames, scenes without faces, small faces, and partly hidden faces. Write down how to label faces seen from the side, partly hidden faces, and very small faces. Use the same rules for drawing boxes throughout the dataset.

For Option 1, the Workflow uploads the images with their predictions. For Option 2, use Upload Data (opens in a new tab) to add the images as an annotation batch before opening Auto Label.

Keep separate data for evaluation. Training data updates the model’s weights. Validation data helps select settings during development. Test data is used for the final evaluation. Set these splits using the dataset-version controls (opens in a new tab). For this camera example, use a separate recording session for testing and check its labels manually.

Step 2: Configure and test one labeling path

Choose one path below. Use the sample and annotation rules from Step 1 to check how the teacher handles difficult lighting and partial occlusion (partly hidden faces).

Option 1. Configure Gemini and save its annotations

Roboflow Managed API Key, or your own Google API key

Keep the predictions output connected to $steps.google_gemini.predictions. Version 6 decodes detections inside the block. You do not need a separate VLM-as-detector block. Use Object Detection because Open Prompt does not populate predictions.

Add Roboflow Dataset Upload v2 after Gemini. Connect its image input to $inputs.image and its predictions input to $steps.google_gemini.predictions. Select the destination project and configure the required Roboflow API key (opens in a new tab). Apply these upload settings (opens in a new tab).

Setting

Value

Block name

dataset_upload

Target Project

Your face-detection project ID

Persist Predictions

true

Data Percentage

100

Disable Sink

false

Fire And Forget

false

Usage Quota Name

face-labeling

Labeling Batch Prefix

gemini-face-labels

Check the upload quotas and image-size settings. Setting Data Percentage to 100 does not bypass the upload quotas. Setting fire_and_forget to false waits for the upload result. Add $steps.dataset_upload.error_status and $steps.dataset_upload.message as Workflow outputs. Use them to check for failed or skipped uploads.

You may also add Gemini’s error_status, input_tokens, and output_tokens as Workflow outputs. Use them to check errors and estimate costs. Use the corresponding $steps.google_gemini field for each output. A decoding error does not mean the image contains no faces.

Option 2. Choose and test a teacher in Auto Label

Open the uploaded batch and choose Auto Label. Some screens call this Auto-Label And Review.

Open the model dropdown. It groups the choices under Foundation Models, My Models, and My Workflows, as shown in the screenshots below.

Foundation Models includes GPT-6.1 Terra, GPT-6 Astra, GPT-6.1 Sol, Gemini 3.8 Flash, Gemini 4 Argon, and Claude Opus 5.5 for bounding boxes. SAM 3 returns masks or polygons, which train as boxes in an object-detection project.

My Models lists your trained models. Select an object detector that already predicts face. For example, a trained RF-DETR detector could serve this role if its classes include face. Auto Label currently supports trained models from the same dataset as the annotation batch. These models use their trained classes, not text descriptions.

My Workflows lists compatible saved Workflows. This option requires mapping the Workflow’s output classes to project classes. It is available inside Auto Label, while Option 1 in this article runs its Gemini Workflow separately through Batch Processing.

These seven foundation models are shown in the supplied object-detection screenshot. Choose a teacher by testing its face annotations on the sample images. Check the current selector for availability.

For a foundation model, add the class face and a short visual description, such as human face. With a trained model, confirm that face is already one of its classes.

Select preview images and click Generate Preview, as shown in the screenshot. The documentation (opens in a new tab) calls this Generate Test Results. Check the generated labels before starting the full job.

Adjust descriptions and test again if needed. Adjust the confidence threshold only when the selected model provides one. For object detection, foundation models other than SAM 3 do not show a confidence slider.

Select the teacher whose annotations meet the face-labeling rules with an acceptable amount of correction. Those reviewed annotations will supervise the student.

Step 3: Generate annotations and review them

After testing the teacher on a small sample, use the selected Workflow or Auto Label path to generate annotations for the remaining images. Review and correct the face boxes before adding the images to the dataset. These reviewed image–annotation pairs will be used to train the smaller student model.

Option 1. Generate labels with the saved Workflow

After the sample test in Step 2, use Batch Processing (opens in a new tab) to run the saved Workflow on your remaining images. The job sends each image through Gemini. The connected Roboflow Dataset Upload block saves the image and its predictions in the project selected in Target Project.

Prepare the labeling run

Save the upload settings from Step 2 and check that the quotas allow the planned batch. The browser’s Sinks switch (opens in a new tab) affects test runs only; it does not change the saved Workflow or a Batch Processing job.

Run the images

Open Deployments in the left sidebar and select Batch Processing.

Click New Batch Job and select the saved Gemini labeling Workflow.

Upload the images from your prepared folder.

Choose CPU or GPU hardware. Leave Advanced Options at their defaults unless your setup needs a change.

Click Create Batch Job. Follow progress in the job list and open the job to see its details.

These controls are covered in the Batch Processing web app guide (opens in a new tab). Begin with a few images to confirm that the saved annotations reach the correct project, then process the remaining images. The job uses credits, and the cost section below explains the relevant charges.

Check that the labels were saved

Inspect the Workflow’s returned outputs. In the supplied JSON, gemini_error reports Gemini errors. upload_error and upload_message report the upload result. Read the message as well as the error flag because sampling or quotas can skip an upload. These fields come from the Dataset Upload block’s outputs (opens in a new tab).

Open the destination project and find the labeling batch whose name starts with gemini-face-labels, the prefix set in Step 2. The full name depends on the batch creation settings. Open an image and check that its predicted face boxes appear as editable annotations. The Batch Processing job produces Workflow results; the connected upload block is what saves the training annotations in your project.

If an image or its labels are missing, check the Gemini error, upload message, target project, quotas, and prediction-persistence setting before rerunning that image. Once the images and annotations are present, continue with the review steps below.

Both options now move to annotation review. Open the resulting images in the annotation interface (opens in a new tab). Check each face box and its class. Resize or move incorrect boxes, add missed faces, and remove false detections. Each face needs its own box, so split any box that covers two faces.

Review images with no annotations too. A frame may contain no faces, or the teacher may have missed one.

If Review Mode (opens in a new tab) is enabled, approve checked images and reject those needing more work. Review Mode is available on Teams Add-On and Enterprise workspaces. Before creating a version, select the checked images and use Add N Images to Dataset in the job controls (opens in a new tab).

Step 4: Create a dataset version

Open Versions, select Generate New Version, and choose the data split and image preparation settings. Then generate the version. A dataset version (opens in a new tab) saves a fixed copy of your images, annotations, and preprocessing and augmentation settings.

Record the teacher, class prompts, review rules, and dataset version for each experiment. These details help you understand why results change between training runs.

Step 5: Train a compact student

Open Train, then Custom Training. Select a single architecture, choose RF-DETR Nano (opens in a new tab), and select your saved dataset version. Check the checkpoint, training settings, and cost estimate, then click Start Training. Follow the Custom Train steps (opens in a new tab). RF-DETR Nano is one model to try for this example. Test whether it can run on your battery-powered camera. Check its model size, memory needs, supported operations, and runtime compatibility. A device with tighter limits may need a different student architecture.

For the first run, choose a supported public pretrained checkpoint (opens in a new tab). A checkpoint stores model parameters. These weights give the student a starting point; the reviewed teacher annotations provide supervision for the face-detection task.

The trained student detects the classes in its dataset. It does not gain all the language and reasoning abilities of a Gemini teacher. What it learns depends on the labels used to train the target model (opens in a new tab).

Another training option with Autodistill

Autodistill (opens in a new tab) provides another way to train a smaller model using automatically generated labels. A VLM such as Florence-2 (opens in a new tab) acts as the Base Model and labels the images. After reviewing the annotations, a Target Model, such as YOLOv8, trains on the dataset. For a practical walkthrough, see Tutorial 1 in YOLO Distillation.

Step 6: Evaluate accuracy and test on the target device

If your plan includes it, open the trained model and select View Full Model Evaluation. Model Evaluation (opens in a new tab) shows class performance and prediction errors against dataset annotations.

Check precision and recall. Precision is the fraction of kept detections that are correct. Recall is the fraction of labeled objects that the model detects correctly. Object-detection evaluation also needs a rule for matching predicted boxes to labeled objects. Check missed objects and incorrect detections, especially at night and when faces are far from the camera.

Use validation data to choose a confidence threshold, the minimum score needed to keep a detection. A model’s confidence can differ from its observed correctness, so assess the student against manually checked labels.

Next, benchmark the model by measuring its performance on the actual camera hardware and runtime you plan to use. Roboflow’s reported Cloud API latency (opens in a new tab) comes from a different environment and excludes network time.

Record detection errors, capture-to-result latency, memory use, and total system power at your intended frame rate. Compare these measurements with your application’s limits. Consider accuracy, latency, and energy together when choosing a model.

Step 7: Deploy the student on compatible hardware

Use Run a Model Locally (opens in a new tab) to test the trained student with Roboflow Inference on compatible hardware. Follow the linked guide for installation, authentication, and inference. To run inside the camera, the student also needs a runtime supported by that camera.

After it meets the device requirements, the student processes live frames locally and returns face boxes. The teacher is no longer needed for each frame. If using Roboflow Inference without internet access, follow the Enterprise Offline Mode requirements (opens in a new tab), including initial authorization and model-weight caching.

Labeling cost for a dedicated annotator versus Gemini Pro

Labeling is part of preparing the student’s training data. Compare the cost of usable, reviewed annotations, including teacher usage, runtime charges, and corrections. A lower labeling bill is useful only if the resulting dataset trains a student that meets the camera’s requirements.

1 credit per 1,000,000 input tokens and 1 credit per 166,667 output tokens with a managed key

Workflow execution or hosting, image uploads, and storage. This standalone path does not run an Auto Label job.

Option 2. Selected teacher in Auto Label

Base image fee of 1 credit per 1,000 images, plus any model usage charges

The selected model’s job estimate, image uploads, storage, and review effort.

These rates alone do not show which option costs less for your dataset. Compare the selected Auto Label teacher’s job estimate with the measured Gemini Workflow cost. Include the review effort for both options.

For Option 1, Serverless Cloud API pricing (opens in a new tab) bills VLM blocks for execution time at 500 execution seconds per credit. Other runtimes have their own billing. The credit rates also list upload and storage charges. Include the charges that apply to your setup. Do not add Auto Label’s image fee to the standalone Workflow simply because it produces annotations.

If you choose My Models and need to train a teacher first, include that setup cost. Both options also need human review, student training, and evaluation.

Tokens measure model input and output. Image usage depends on the model and media resolution (opens in a new tab). Measure the chosen settings on a sample instead of assuming a fixed token count for every frame.

Before estimating a larger batch, run a small sample that covers your typical images, using the chosen model and settings. Record the v6 block’s input_tokens and output_tokens (opens in a new tab), check error_status, and inspect the annotations. For Gemini 3.1 Pro, thinking_level="low" is a supported setting to test. It does not turn thinking off. Check label quality as well as token usage. Use the measured usage and current rates to estimate the larger job. Actual usage can vary between images.

Include review effort in the decision

Use the same sample images and class definitions to compare teachers. Record the number of images with acceptable annotations and the time needed for corrections. Include retries in labeling charges. Then calculate the cost per accepted image, meaning an image whose annotations pass review.

Cost per accepted image = (labeling charges + human review cost) / accepted images

Use the same currency for labeling and review costs. This formula helps you budget for the project. (It's not a Roboflow billing metric.)

Conclusion

Model distillation can help turn a large vision model’s predictions into a compact detector for a specific task. Get started with model distillation in Roboflow by turning images from your own camera into a training dataset. Use Auto Label to generate annotations, review the results, and train a compact detector such as RF-DETR Nano. You can manage each step in Roboflow, from preparing the data to testing and deploying your model.

Start with a small, representative batch of images and build your first baseline. Evaluate its accuracy, test it on your target hardware, and use those results to improve your dataset. Create a Roboflow project to begin.

── more in #computer-vision 4 stories · sorted by recency
── more on @roboflow 3 stories trending now
sponsored brought to you by zahid.host 4,200+ EU-deployed projects
reading about agents? ship yours in a single git push.

Run your AI side-project on zahid.host

EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.

$git push zahid main
→ Live at https://your-agent.zahid.host ✓
Get free account → Pricing
from €0/mo · no card required
LIVE [news/what-is-model-distil…] indexed:0 read:19min 2026-11-15 · —