Object Detection on Android for Autonomous Robots

Autonomous robots need to recognize objects in their environment. Object detection can identify people, vehicles, tools, signs, and obstacles from camera frames.

Android can perform edge inference locally, reducing dependency on network connectivity.

CameraX
   |
Preprocessing
   |
Object Detection Model
   |
Postprocessing
   |
Detection Results
   |
Robot Perception Gateway

Define a reusable result type:

data class Detection(
    val label: String,
    val confidence: Float,
    val left: Float,
    val top: Float,
    val right: Float,
    val bottom: Float
)

This keeps the rest of the application independent from a particular model runtime.

The camera analyzer should process frames asynchronously:

imageAnalysis.setAnalyzer(executor) { image ->
    detector.detect(image)
    image.close()
}

Use a latest-frame strategy when real-time responsiveness is more important than processing every frame.

The detector can be implemented behind an interface:

interface ObjectDetector {
    suspend fun detect(frame: ImageFrame): List<Detection>
}

Possible mobile inference approaches include TensorFlow Lite or ONNX Runtime, depending on the model and deployment requirements.

Not every prediction should be passed to the navigation system.

val valid = detections.filter {
    it.confidence >= 0.6f
}

The threshold should be evaluated against the target environment rather than chosen arbitrarily.

Detection models may produce overlapping predictions.

Prediction A  ───────
Prediction B    ───────
        ↓
       NMS
        ↓
Single Detection

Use the postprocessing method expected by your selected model.

The Android device can send detections to the robot:

{
  "label": "person",
  "confidence": 0.94,
  "bbox": [120, 80, 350, 500]
}

For autonomous navigation, the robot should combine this with physical measurements such as depth or LiDAR when distance matters.

Instead of detecting every object from scratch at every stage, an additional tracking layer can maintain object identities between frames.

Detection
   ↓
Tracking
   ↓
Object ID
   ↓
Navigation / Behavior

Tracking can reduce redundant processing and provide temporal context.

Important optimization techniques include:

Keep AI perception separate from robot control:

Camera
  ↓
Object Detection
  ↓
Perception State
  ↓
Navigation / Behavior
  ↓
Safety Controller
  ↓
Robot

This separation makes the system easier to test and safer to operate.

Evaluate the system using representative scenarios:

Measure both detection accuracy and real-time performance.

Object detection on Android can provide useful edge perception for autonomous robots. Kotlin, CameraX, and a mobile inference runtime create a flexible foundation that can later be connected to ROS 2, sensor fusion, navigation, and Physical AI agents.

SDK Flutter: https://github.com/v-modal/vmodal_sdk_flutter

SDK Android: https://github.com/v-modal/vmodal_sdk_android

Discord: https://discord.gg/K72z28KUx