YOLO Object Detection on Android for Robotics

A developer detailed how to build an Android application that performs real-time YOLO object detection for robotics, using CameraX for frame capture and a mobile inference runtime for local processing. The tutorial emphasizes modular architecture, proper preprocessing, and postprocessing techniques like non-maximum suppression, while warning against treating 2D bounding boxes as physical measurements. It also highlights the importance of consistent frame rates and end-to-end latency for edge robotics applications.

Object detection is an important capability for autonomous robots. A robot can use detections to identify people, vehicles, tools, obstacles, and other objects in its environment. YOLO-family models are widely used for real-time object detection. In this tutorial, we will design an Android application that captures camera frames and performs YOLO inference locally. CameraX | Preprocessing | YOLO Model | Postprocessing | Bounding Boxes | Robot Perception Layer The exact model format depends on the runtime you choose. For Android edge deployment, an exported model may be converted to a mobile-compatible format and executed using an appropriate inference runtime. Organize the project into separate layers: vision/ ├── CameraManager.kt ├── YoloDetector.kt ├── Detection.kt └── DetectionOverlay.kt This prevents camera handling, inference, and rendering from becoming tightly coupled. Create a Kotlin model: data class Detection val classId: Int, val label: String, val confidence: Float, val boundingBox: RectF Use CameraX ImageAnalysis to obtain frames. php imageAnalysis.setAnalyzer executor { image - detector.process image image.close } For real-time robotics, use a backpressure strategy that drops stale frames rather than allowing an inference queue to grow indefinitely. Most object-detection models expect a fixed input size. The preprocessing stage normally performs: For example: Camera Frame ↓ Resize ↓ Normalize ↓ Tensor ↓ YOLO The preprocessing code must match the model's training/export requirements. Create a detector abstraction: class YoloDetector { suspend fun detect frame: ImageFrame : List<Detection { // preprocess // inference // postprocess return emptyList } } Run inference outside the Android main thread. Object detectors can return multiple candidate boxes. Postprocessing commonly includes: For example: Raw Predictions ↓ Confidence Filter ↓ NMS ↓ Final Detections The Android UI can display detection results over the live camera preview. +--------------------------+ | | | +------------+ | | | bottle | | | | 92% | | | +------------+ | | | +--------------------------+ Remember to map coordinates correctly when the preview and model input have different aspect ratios. Detection results can be passed to a robot control layer: { "object": "person", "confidence": 0.92, "bbox": 120, 80, 350, 500 } The robotics layer can combine this information with depth, odometry, LiDAR, or other sensors. Do not treat a 2D bounding box as a physical distance measurement unless the system has additional calibration or depth information. For edge robotics: A smaller model running consistently can be more useful for robotics than a larger model with unstable frame rates. Test the detector with: For robotics, also measure end-to-end latency: Capture → Inference → Decision → Robot Command YOLO-style object detection can turn an Android device into a useful edge-vision component for robotics. Kotlin, CameraX, and a mobile inference runtime provide the foundation for building perception prototypes that can later integrate with ROS 2 and autonomous navigation. SDK Flutter: https://github.com/v-modal/vmodal sdk flutter https://github.com/v-modal/vmodal sdk flutter SDK Android: https://github.com/v-modal/vmodal sdk android https://github.com/v-modal/vmodal sdk android Discord: https://discord.gg/K72z28KUx https://discord.gg/K72z28KUx