YOLO Object Detection on Android for Robotics

Object detection is an important capability for autonomous robots. A robot can use detections to identify people, vehicles, tools, obstacles, and other objects in its environment.

YOLO-family models are widely used for real-time object detection. In this tutorial, we will design an Android application that captures camera frames and performs YOLO inference locally.

CameraX
   |
Preprocessing
   |
YOLO Model
   |
Postprocessing
   |
Bounding Boxes
   |
Robot Perception Layer

The exact model format depends on the runtime you choose. For Android edge deployment, an exported model may be converted to a mobile-compatible format and executed using an appropriate inference runtime.

Organize the project into separate layers:

vision/
├── CameraManager.kt
├── YoloDetector.kt
├── Detection.kt
└── DetectionOverlay.kt

This prevents camera handling, inference, and rendering from becoming tightly coupled.

Create a Kotlin model:

data class Detection(
    val classId: Int,
    val label: String,
    val confidence: Float,
    val boundingBox: RectF
)

Use CameraX ImageAnalysis

to obtain frames.

imageAnalysis.setAnalyzer(executor) { image ->
    detector.process(image)
    image.close()
}

For real-time robotics, use a backpressure strategy that drops stale frames rather than allowing an inference queue to grow indefinitely.

Most object-detection models expect a fixed input size.

The preprocessing stage normally performs:

For example:

Camera Frame
   ↓
Resize
   ↓
Normalize
   ↓
Tensor
   ↓
YOLO

The preprocessing code must match the model's training/export requirements.

Create a detector abstraction:

class YoloDetector {
    suspend fun detect(frame: ImageFrame): List<Detection> {
        // preprocess
        // inference
        // postprocess
        return emptyList()
    }
}

Run inference outside the Android main thread.

Object detectors can return multiple candidate boxes. Postprocessing commonly includes:

For example:

Raw Predictions
      ↓
Confidence Filter
      ↓
NMS
      ↓
Final Detections

The Android UI can display detection results over the live camera preview.

+--------------------------+
|                          |
|     +------------+       |
|     |   bottle   |       |
|     |    92%     |       |
|     +------------+       |
|                          |
+--------------------------+

Remember to map coordinates correctly when the preview and model input have different aspect ratios.

Detection results can be passed to a robot control layer:

{
  "object": "person",
  "confidence": 0.92,
  "bbox": [120, 80, 350, 500]
}

The robotics layer can combine this information with depth, odometry, LiDAR, or other sensors.

Do not treat a 2D bounding box as a physical distance measurement unless the system has additional calibration or depth information.

For edge robotics:

A smaller model running consistently can be more useful for robotics than a larger model with unstable frame rates.

Test the detector with:

For robotics, also measure end-to-end latency:

Capture → Inference → Decision → Robot Command

YOLO-style object detection can turn an Android device into a useful edge-vision component for robotics. Kotlin, CameraX, and a mobile inference runtime provide the foundation for building perception prototypes that can later integrate with ROS 2 and autonomous navigation.

SDK Flutter: https://github.com/v-modal/vmodal_sdk_flutter

SDK Android: https://github.com/v-modal/vmodal_sdk_android

Discord: https://discord.gg/K72z28KUx