EgoHand: Monocular 3D Hand Reconstruction from Egocentric Video

Researchers introduced EgoHand, a monocular 3D hand reconstruction model that achieves state-of-the-art performance on egocentric video, trained on a curated corpus of 14 million images combining synthetic data, in-the-wild video, and multi-camera motion capture. EgoHand uses the SOMA-Hand parametric representation and a transformer encoder over 2D hand crops with camera cues and an auxiliary 2D branch, delivering lower joint-position, root-translation, and acceleration errors than competing baselines on zero-shot evaluations of HOT3D, ARCTIC, and H2O at roughly 9.69 ms inference latency. A blinded user preference study found participants favored EgoHand's reconstructed hand poses over those from Apple EgoDex and Ropedia Xperience-10M, and the model and source code are released under the OpenMDW license, permitting free commercial use.

arXiv:2610.30293v1 Announce Type: new Abstract: We introduce EgoHand, a monocular 3D hand reconstruction model that achieves state-of-the-art performance on egocentric video. Central to our approach is the curation of a high-quality training corpus of 14 million images, combining synthetic data, in-the-wild video, and multi-camera motion capture to balance scene diversity and supervision quality. The model adopts SOMA-Hand as its parametric representation and processes 2D hand crops with a transformer encoder, incorporating camera cues and an auxiliary 2D branch to improve geometric consistency and image alignment. Zero-shot evaluations on HOT3D, ARCTIC, and H2O demonstrate strong out-of-domain generalization, with lower joint-position, root-translation, and acceleration errors than competing baselines at an inference latency of approximately 9.69 ms. A blinded user preference study further shows that participants favor EgoHand's reconstructed hand poses over those from Apple EgoDex and Ropedia Xperience-10M. We release the EgoHand model and source code under the OpenMDW license, allowing free commercial use of the model.