Automate tire sidewall OCR by pairing a custom-trained RF-DETR detector that isolates the tire in a busy image with a multimodal LLM that reads the cropped sidewall and returns DOT code, size, brand, load index, and date as a structured record. Built as a Vision Agent in Roboflow Workflows, the detector localizes and crops so the LLM only reads clean, high-density text, and each result is flagged accept, manual review, or retake before it logs to your fleet database.
Maintaining fleet tires and organizing inventory are massive headaches. Missing a wrong tire size, an expired manufacturer date, or a bad load rating is an easy way for a brutal highway blowout and expensive downtime.
Historically, grabbing details like tire dimensions and DOT codes meant sending a technician to squint at the low-contrast lettering molded into the rubber manually. Trying to read faint, black-on-black text on a dirty tire is a chore and prone to human error.
Computer vision can change this. By setting up an automated vision pipeline, you can instantly isolate the tire sidewall, run high-precision Optical Character Recognition (OCR), and log critical tire metadata straight to your database without breaking a sweat.
Automated Tire Sidewall OCR with Roboflow #
This system utilizes a multi-stage framework. We can use a Vision Agent architecture that splits tasks between specialized perception models and a central reasoning engine.
Stage 1: The Detector (Perception Layer)
An RF-DETR object detection model operates as the vision system's eyes. Its objective is to scan incoming vehicle imagery and isolate the precise coordinate boundaries of wheels and tires.
Stage 2: The Inspector (Reasoning & Extraction Layer)
A Multimodal Large Language Model (LMM) serves as the system's brain. It processes high-resolution crops of the isolated tire region, performing simultaneous Optical Character Recognition (OCR) and context verification to interpret markings molded into the rubber sidewall.
Step-by-Step Implementation Guide #
Step 1: Access the Roboflow Platform
Initialize the workspace by logging into your Roboflow account. New users can set up a complimentary tier to begin building and organizing custom vision projects.
Step 2: Gather Images or Fork a Dataset
Computer vision systems rely on relevant training data. For this application, you can acquire or assemble a specialized tire and wheel dataset on
__Roboflow Universe__Forking an existing project variant brings the imagery straight into your environment for immediate customization.
Step 3: Annotate and Classify Images
For custom inspection imagery, utilize Roboflow's web-based annotation tools to construct bounding boxes around structural boundaries. Label your targets using clear class definitions: car-tire, center, rim, Wheel-Rim, and wheels.
Step 4: Execute RF-DETR Model Training
We will train the dataset on the RF-DETR Small architecture for our primary object detection phase. This real-time transformer design offers strong spatial localization capabilities while maintaining a compact operational footprint.
- Targeted Localization: The model handles the heavy lifting of finding the asset within a busy environment, allowing downstream language models to focus purely on textual analysis.
- Edge Capability: The small architecture variant runs efficiently on embedded hardware such as an NVIDIA Jetson, keeping data processing entirely local to the service bay.
- High-Velocity Inference: Rapid processing speeds ensure that inspections finish in milliseconds, preventing bottlenecks in automated drive-through inspection lanes.
Step 5: Partition the Data Split
We divide the dataset using a standard distribution: 70% for Training, 20% for Validation, and 10% for Testing.
- Training Set - 70%: Provides the model with core patterns, surface textures, and angles.
- Validation Set - 20%: Acts as a diagnostic check during training to tune hyperparameters and mitigate overfitting.
- Testing Set - 10%: Serves as the final grade using entirely unseen images to measure real-world performance.
Step 6: Configure Preprocessing and Augmentations
To ensure the pipeline handles diverse real-world garage environments and varying outdoor lighting conditions, apply explicit input transformations.
Preprocessing: Apply Auto-Orient to strip orientation metadata and maintain a uniform image directional flow. Use a Resize operation to stretch inputs to 512x512 pixels.Augmentations: Make the pipeline to generate two variations per training image. Introduce a Brightness variance between -15% and +15% alongside an Exposure variance between -10% and +10% to condition the network against strong glare, deep shadows, and changing weather conditions.
Step 7: Analyze Training Performance
Once training concludes, assess the model's production readiness through the metrics explorer. The custom wheel detector achieved strong performance benchmarks on the evaluation set:
- mAP@50: 94.1%
- Precision: 95.0%
- Recall: 95.0%
- F1-Score: 95.0%
Optimization Strategies
- Address High-Risk Gaps: Class 0 exhibited a higher frequency of false negatives. Collecting and annotating additional images containing this class will improve the model's sensitivity.
- Implement Image Tiling: If tire sidewall markings appear exceptionally small within wide-angle captures, utilizing Slicing Aided Hyper Inference (SAHI) will help the detector isolate minuscule targets.
- Rectify Class Imbalances: Class 0 only contained 5 instances within this dataset version. Expanding underrepresented categories ensures uniform model proficiency across all labeled classes.
Constructing the Tire OCR Vision Agent Workflow #
Training the perception model completes the first phase of development. To automate the entire data extraction pipeline, assemble a custom workflow using
Roboflow Workflows### 1. Object Detection Model
- Block Type: object detection
- Configuration: wheel-tire-detection-ceyns-5jgw5/1
- Function: Runs the custom-trained RF-DETR Small model directly on the input image to discover the bounding regions of the tires and wheels.
2. Detections Filter
This block examines the incoming predictions and discards weak or ambiguous detections. Setting this threshold ensures that only highly certain tire regions pass downstream, preventing background elements from entering the OCR block.
Add a detection filter block and configure the JSON like so:
{
"type": "roboflow_core/detections_filter@v1",
"name": "detections_filter",
"predictions": "$steps.model.predictions",
"operations": [
{
"type": "DetectionsFilter",
"filter_operation": {
"type": "StatementGroup",
"operator": "and",
"statements": [
{
"type": "BinaryStatement",
"left_operand": {
"type": "DynamicOperand",
"operand_name": "_",
"operations": [
{
"type": "ExtractDetectionProperty",
"property_name": "confidence"
}
]
},
"comparator": {
"type": "(Number) >="
},
"right_operand": {
"type": "StaticOperand",
"value": 0.5
},
"negate": false
}
]
}
}
],
"operations_parameters": {}
}
3. Dynamic Crop
This block uses the bounding coordinates from the filter to extract sub-images directly from the original full-frame asset. This isolates the tire, maximizing pixel density and text legibility for the language model.
4. OpenAI LMM
In many workflows, this component serves as the central decision-making engine. The core intelligence comes from the LLM-powered Vision Agent. For this implementation, configure an OpenAI block with the task set to Structured Output Generation, connect its image input to the Dynamic Crop output, and choose GPT-5-mini as the underlying model.
Specify the following JSON schema for extracting information from the tires:
{
"status": "One of: SUCCESS, PARTIAL, or UNREADABLE. Use SUCCESS when the tire sidewall text needed for identification is clearly readable; PARTIAL when some useful tire markings are readable but important fields are missing or uncertain; UNREADABLE when the crop is too blurry, occluded, low contrast, curved, overexposed, or otherwise not reliable for OCR.",
"tire_brand": "Tire manufacturer or brand visible on the sidewall, for example Michelin, Goodyear, Bridgestone, Pirelli, Continental, Dunlop, BFGoodrich, Yokohama, Hankook, Kumho, Toyo, or Unknown. Preserve capitalization as seen when possible.",
"tire_model": "Tire product line or model name visible on the sidewall, for example Defender, Pilot Sport, Eagle, Turanza, Potenza, Scorpion, or Unknown. Do not invent a model if it is not visible.",
"tire_size": "The standardized tire size exactly as printed, such as P215/65R16, 225/45ZR17, LT265/70R17, 11R22.5, or Unknown. Preserve prefixes like P, LT, ST and speed construction letters like R or ZR.",
"service_description": "The load index and speed rating together if printed near the size, for example 91V, 104H, 121/118R, or Unknown.",
"load_index": "The numeric load index only, for example 91, 104, 121/118, or Unknown.",
"speed_rating": "The speed rating letter only, for example T, H, V, W, Y, R, or Unknown.",
"dot_tin": "The full DOT / Tire Identification Number exactly as visible, starting with DOT when present. Include plant code, size code, optional manufacturer code, and date code if readable. Use Unknown if not visible.",
"manufacture_week": "The 2-digit DOT manufacture week from the last four digits of the DOT/TIN date code, for example 07 from 0721. Use Unknown if the date code is not readable.",
"manufacture_year": "The 4-digit manufacture year inferred from the last two DOT date-code digits, for example 2021 from 0721. Use Unknown if the date code is not readable.",
"sidewall_markings": "A concise list or string of other useful sidewall markings visible, such as M+S, 3PMSF, TUBELESS, RADIAL, OUTSIDE, INSIDE, ROTATION, XL, SL, E4, load range, max pressure, max load, treadwear, traction, temperature, or ply information.",
"ocr_confidence": "One of: high, medium, or low. Base this on image clarity, angle, contrast, occlusion, and how much of the lettering is readable.",
"unreadable_reason": "If status is UNREADABLE or PARTIAL, briefly explain what prevented complete OCR, for example blur, glare, shadow, low contrast, crop misses sidewall, text too small, or occlusion. If status is SUCCESS, use an empty string.",
"next_action": "One of: ACCEPT, MANUAL_REVIEW, or RETAKE_IMAGE. Use ACCEPT when status is SUCCESS and key fields are reliable; MANUAL_REVIEW when status is PARTIAL or important fields are uncertain; RETAKE_IMAGE when status is UNREADABLE or the crop does not show enough sidewall text."
}
5. JSON Parser
This block accepts the structured text block from the LMM and splits it into independent, separate output variables. This flattens the response payload into individual fields, making the data instantly ready to plug into a fleet database, an enterprise API, or an automated maintenance dashboard.
These should be the expected JSON fields: status, tire_brand, tire_model, tire_size, service_description, load_index, speed_rating, dot_tin, manufacture_week, manufacture_year, sidewall_markings, ocr_confidence, unreadable_reason, next_action
6. Output block
The following outputs can be retrieved:
7. Test the workflow
Use a sample image and test out the workflow:
Automated Tire Sidewall OCR Conclusion #
Combining automated object detection with advanced vision-language models creates a reliable verification system for transport and automotive tracking. By splitting the work between a fast, local RF-DETR model and a detailed multimodal reasoning agent, you build an efficient double-check architecture. This pipeline ensures that every vehicle passing an inspection point has its tire assets cataloged, verified, and safely recorded.
Ready to build your own asset tracking pipelines? Sign up for a free Roboflow account and explore thousands of open-source datasets on
Roboflow UniverseCite this Post
Use the following entry to cite this post in your research:
Automated Tire Sidewall OCR. Roboflow Blog: https://blog.roboflow.com/automated-tire-sidewall-ocr/