{"slug": "real-time-object-detection-and-tracking-with-kotlin-and-yolo-on-android", "title": "Real-Time Object Detection and Tracking with Kotlin and YOLO on Android", "summary": "A developer detailed the architecture for building a real-time object detection and tracking app on Android using Kotlin and YOLO-style models. The approach separates camera capture, inference, tracking, and rendering, and includes techniques like non-maximum suppression and IoU-based tracking to maintain object identities across frames.", "body_md": "Real-time computer vision is one of the most useful applications of machine learning on mobile devices. Android phones provide cameras, GPU acceleration, and enough processing power to run optimized detection models locally.\n\nIn this tutorial, we will build the architecture for a Kotlin application that captures camera frames, runs a YOLO-style object detector, and tracks detected objects across frames.\n\n```\nCameraX\n   |\nImageAnalysis\n   |\nFrame Conversion\n   |\nYOLO Detector\n   |\nNon-Maximum Suppression\n   |\nObject Tracker\n   |\nUI Overlay\n```\n\nThe important part is to keep camera capture, inference, tracking, and rendering separate.\n\nAdd the CameraX dependencies compatible with your project:\n\n```\ndependencies {\n    implementation(\"androidx.camera:camera-camera2:<version>\")\n    implementation(\"androidx.camera:camera-lifecycle:<version>\")\n    implementation(\"androidx.camera:camera-view:<version>\")\n}\n```\n\nCreate an `ImageAnalysis`\n\nuse case:\n\n```\nval analysis = ImageAnalysis.Builder()\n    .setBackpressureStrategy(\n        ImageAnalysis.STRATEGY_KEEP_ONLY_LATEST\n    )\n    .build()\n```\n\n`KEEP_ONLY_LATEST`\n\nis important for real-time applications. If inference is slower than the camera, processing every old frame creates a growing queue and increases latency.\n\nImplement an analyzer that receives camera frames:\n\n```\nclass ObjectAnalyzer(\n    private val detector: YoloDetector\n) : ImageAnalysis.Analyzer {\n\n    override fun analyze(image: ImageProxy) {\n        try {\n            detector.detect(image)\n        } finally {\n            image.close()\n        }\n    }\n}\n```\n\nAlways close `ImageProxy`\n\nwhen processing is complete.\n\nYour detector should convert camera input into the tensor format expected by your model.\n\nConceptually:\n\n```\nclass YoloDetector {\n\n    fun detect(image: ImageProxy): List<Detection> {\n        val input = preprocess(image)\n        val output = runModel(input)\n\n        return postprocess(output)\n    }\n}\n```\n\nA detection normally contains a bounding box, class ID, confidence score, and optionally additional metadata.\n\n```\ndata class Detection(\n    val classId: Int,\n    val confidence: Float,\n    val box: RectF\n)\n```\n\nMost object detection models expect a fixed input size.\n\nFor example:\n\n```\nCamera frame\n    ↓\nRotate\n    ↓\nCrop / resize\n    ↓\nNormalize\n    ↓\nTensor\n```\n\nBe careful with aspect ratio. Incorrect scaling can make objects appear distorted and reduce detection accuracy.\n\nYOLO-style models can return many candidate boxes. Filter low-confidence detections:\n\n```\nval filtered = detections.filter {\n    it.confidence >= 0.5f\n}\n```\n\nThen apply non-maximum suppression (NMS) to remove overlapping boxes representing the same object.\n\nThe exact threshold should be tuned using your target model and dataset.\n\nDetection tells you what is visible in a frame. Tracking attempts to maintain an object's identity across multiple frames.\n\nA simplified tracked object could be:\n\n``` js\ndata class TrackedObject(\n    val id: Int,\n    var box: RectF,\n    var classId: Int,\n    var confidence: Float\n)\n```\n\nA basic tracker can associate the current detection with an existing object using Intersection over Union (IoU).\n\n```\nfun iou(a: RectF, b: RectF): Float {\n    val left = maxOf(a.left, b.left)\n    val top = maxOf(a.top, b.top)\n    val right = minOf(a.right, b.right)\n    val bottom = minOf(a.bottom, b.bottom)\n\n    if (right <= left || bottom <= top) return 0f\n\n    val intersection =\n        (right - left) * (bottom - top)\n\n    val union =\n        a.width() * a.height() +\n        b.width() * b.height() -\n        intersection\n\n    return intersection / union\n}\n```\n\nFor more robust tracking, consider established algorithms such as SORT or ByteTrack.\n\nUse a custom Android view to draw detections over the camera preview.\n\n``` js\nclass DetectionOverlay : View(context) {\n\n    var detections: List<Detection> = emptyList()\n\n    override fun onDraw(canvas: Canvas) {\n        super.onDraw(canvas)\n\n        for (detection in detections) {\n            canvas.drawRect(\n                detection.box,\n                paint\n            )\n        }\n    }\n}\n```\n\nYou will need coordinate transformation because model coordinates and preview coordinates may differ.\n\nReal-time inference requires careful optimization.\n\nUseful techniques include:\n\nDo not optimize only for FPS. A smaller model with slightly lower accuracy may provide a much better mobile user experience.\n\nTrack at least:\n\n```\nCamera FPS\nInference latency\nEnd-to-end latency\nMemory usage\nCPU/GPU utilization\n```\n\nFor example:\n\n```\nval start = System.nanoTime()\n\nval detections = detector.detect(image)\n\nval elapsedMs =\n    (System.nanoTime() - start) / 1_000_000\n```\n\nUse real devices for benchmarking because emulator performance does not represent typical mobile hardware.\n\nCamera frames can have different orientations depending on the device.\n\nPass rotation information into preprocessing:\n\n```\nval rotation = image.imageInfo.rotationDegrees\n```\n\nFailing to handle rotation correctly can produce incorrect bounding boxes or significantly reduce accuracy.\n\nA production application should also handle:\n\nCombining Kotlin, CameraX, YOLO, and object tracking creates a powerful on-device computer vision pipeline.\n\nThe key to a production-ready implementation is not simply running a model. You must also control frame backpressure, coordinate transformations, memory allocations, inference latency, and camera lifecycle.\n\nThis foundation can be extended into applications such as traffic monitoring, retail analytics, industrial inspection, sports analysis, and smart navigation.\n\nSDK Flutter: [https://github.com/v-modal/vmodal_sdk_flutter](https://github.com/v-modal/vmodal_sdk_flutter)\n\nSDK Android: [https://github.com/v-modal/vmodal_sdk_android](https://github.com/v-modal/vmodal_sdk_android)\n\nDiscord: [https://discord.gg/K72z28KUx](https://discord.gg/K72z28KUx)", "url": "https://wpnews.pro/news/real-time-object-detection-and-tracking-with-kotlin-and-yolo-on-android", "canonical_source": "https://dev.to/vmodal_ai/real-time-object-detection-and-tracking-with-kotlin-and-yolo-on-android-3p78", "published_at": "2026-08-14 19:07:04+00:00", "updated_at": "2026-08-14 19:35:40.754445+00:00", "lang": "en", "topics": ["computer-vision", "machine-learning", "developer-tools"], "entities": ["Kotlin", "YOLO", "Android", "CameraX", "SORT", "ByteTrack"], "alternates": {"html": "https://wpnews.pro/news/real-time-object-detection-and-tracking-with-kotlin-and-yolo-on-android", "markdown": "https://wpnews.pro/news/real-time-object-detection-and-tracking-with-kotlin-and-yolo-on-android.md", "text": "https://wpnews.pro/news/real-time-object-detection-and-tracking-with-kotlin-and-yolo-on-android.txt", "jsonld": "https://wpnews.pro/news/real-time-object-detection-and-tracking-with-kotlin-and-yolo-on-android.jsonld"}}