{"slug": "geometry-beats-estimated-depth-rgb-only-multi-camera-3d-tracking-under-sim2real", "title": "Geometry Beats Estimated Depth: RGB-Only Multi-Camera 3D Tracking under Sim2Real", "summary": "A study from the AI City Challenge 2026 Track 1 finds that a geometry-first RGB-only pipeline achieves 13.0 3D HOTA (51.6 LocA) in multi-camera 3D tracking under Sim2Real, while a pseudo-LiDAR approach using monocular depth collapses to 0.12 HOTA (9.2 LocA). The authors attribute the collapse to cross-view inconsistency of monocular depth, which domain-adaptation fine-tuning does not repair within budget. The geometry pipeline's bottleneck is detection quality (DetA), while pseudo-LiDAR is limited by localization consistency (LocA).", "body_md": "arXiv:2608.07579v1 Announce Type: new\nAbstract: The AI City Challenge 2026 Track 1 evaluates multi-camera 3D perception in large indoor warehouses under a synthetic-to-real (Sim2Real) setting; depth is available only for training and validation, so inference is RGB-only. We use two RGB-only routes as a controlled test of one hypothesis: that cross-view geometric consistency, not monocular depth accuracy, governs performance under Sim2Real. The first is a geometry-first pipeline: YOLO11x detection, homography lifting to the world frame, class-level 3D size priors, multi-camera fusion, world-coordinate tracking, and offline tracklet stitching. The second is estimated-depth pseudo-LiDAR: monocular depth (D4RT, Metric3D~v2) back-projected into a fused point cloud and passed to a 3D detector (V-DETR), mirroring prior point-cloud winners that used depth. The gap is decisive: geometry-first reaches 13.0 3D HOTA (51.6 LocA), whereas pseudo-LiDAR collapses to 0.12 (9.2 LocA). We trace the collapse to cross-view inconsistency of monocular depth---scale correction is necessary but not sufficient---which domain-adaptation fine-tuning does not repair within budget. Within the geometry pipeline, offline stitching is the only intervention that helps; SAHI detection, appearance Re-ID, learned lifting, RT-DETR ensembling, test-time augmentation, and domain randomization all fail to beat the baseline detector. The bottlenecks are complementary: detection quality bounds the geometry route (DetA), localization consistency bounds pseudo-LiDAR (LocA). We release a complete, reproducible RGB-only pipeline and ablation.", "url": "https://wpnews.pro/news/geometry-beats-estimated-depth-rgb-only-multi-camera-3d-tracking-under-sim2real", "canonical_source": "https://arxiv.org/abs/2608.07579", "published_at": "2026-08-11 04:00:00+00:00", "updated_at": "2026-08-11 04:24:53.517536+00:00", "lang": "en", "topics": ["computer-vision", "artificial-intelligence", "machine-learning"], "entities": ["AI City Challenge 2026", "YOLO11x", "D4RT", "Metric3D v2", "V-DETR", "SAHI", "RT-DETR"], "alternates": {"html": "https://wpnews.pro/news/geometry-beats-estimated-depth-rgb-only-multi-camera-3d-tracking-under-sim2real", "markdown": "https://wpnews.pro/news/geometry-beats-estimated-depth-rgb-only-multi-camera-3d-tracking-under-sim2real.md", "text": "https://wpnews.pro/news/geometry-beats-estimated-depth-rgb-only-multi-camera-3d-tracking-under-sim2real.txt", "jsonld": "https://wpnews.pro/news/geometry-beats-estimated-depth-rgb-only-multi-camera-3d-tracking-under-sim2real.jsonld"}}