{"slug": "show-hn-impressive-hand-detection-ai-algorithm", "title": "Show HN: Impressive hand detection AI algorithm", "summary": "Idiap Research Institute researchers released a standalone demo of Hiera-Hand, a hand-action recognition pipeline that tracks people, localizes their hands, and labels every hand in every frame as grasp, hold, operate, or release, based on the ChildPlay-Hand dataset paper presented at ECCVW 2024. The demo uses YOLO26m-pose with BoT-SORT for tracking and a 51M-parameter, 205 MB Hiera-Base model fine-tuned on ChildPlay-Hand, with code under GPL-3.0 and non-commercial CC BY-NC 4.0 checkpoints on Zenodo. The authors note the paper used HRNet-W32 for pose while the demo substitutes YOLO26m-pose for speed, so predictions may differ slightly from reported results.", "body_md": "Hand-action recognition on any video with Hiera-Hand\n([ChildPlay-Hand, ECCVW 2024](https://arxiv.org/abs/2409.09319)). It tracks\npeople, localizes their hands, and labels every hand in every frame as\n**grasp**, **hold**, **operate**, or **release**.\n\n[idiap/childplay_hand](https://github.com/idiap/childplay_hand).\n\n⚠️ This is a standalone demo based on the original research work; for training, evaluation, and the dataset, see\n\n```\n./setup_env.sh              # needs uv; Python 3.11 venv in .venv\nsource .venv/bin/activate\n```\n\nFetch the checkpoints from [Zenodo](https://zenodo.org/records/14958923) into `checkpoints/`:\n\n```\npython src/download_checkpoints.py              # manipulation (~0.4 GB download)\npython src/download_checkpoints.py --task all   # + object (~0.8 GB download)\npython src/demo.py input.mp4 --output result.mp4\n```\n\n| Option | Default |  | \n|---|---|---|\n| `--stride N` | 1 | Run Hiera every N frames; 2 ≈ 2× faster | \n| `--task` | manipulation | or `object` (object in hand) | \n| `--device` | auto | CUDA → MPS → CPU | \n| `--batch-size` | 16 / 4 / 2 | CUDA / MPS / CPU | \n| `--smoothing-window` | 5 | frames; 0 = off | \n| `--save-intermediates` | off | writes `pose.pkl` ,`pred.pkl` | \n\nWorks best on clips where people are fully visible, without camera cuts.\n\n1. **Track people and their pose** : YOLO26m-pose + BoT-SORT, in one pass.\n2. **Find hands** : each hand box sits just past the wrist, along the\nelbow→wrist direction.\n3. **Recognize** : for every hand and frame, a ~1 s window (32 frames, 16 sampled)\nis cropped around the hand at 224×224 and fed to**Hiera-Base** (51M params,\n205 MB; MAE-pretrained on Kinetics-400, fine-tuned on ChildPlay-Hand), which\noutputs background / grasp / hold / operate / release.\n4. **Display** : smoothed hand boxes and hand actions.\n\n**Note:** The paper used HRNet-W32 for pose; this demo uses YOLO26m-pose for speed, so\npredictions may differ slightly from the reported results.\n\n- **Code** : GPL-3.0, based on[idiap/childplay_hand](https://github.com/idiap/childplay_hand) (© Idiap Research Institute).\n- **Checkpoints** : CC BY-NC 4.0 (non-commercial), from[Zenodo](https://zenodo.org/records/14958923) .\n- **Pose model** :[Ultralytics](https://github.com/ultralytics/ultralytics) YOLO26, AGPL-3.0.\n- **Hiera architecture** (`src/hiera/` ): Apache-2.0, © Meta.\n\n```\n@inproceedings{Farkhondeh_ECCVW_2024,\n  author    = {Farkhondeh*, Arya and Tafasca*, Samy and Odobez, Jean-Marc},\n  title     = {ChildPlay-Hand: A Dataset of Hand Manipulations in the Wild},\n  booktitle = {Proceedings of the European Conference on Computer Vision (ECCV) Workshops},\n  year      = {2024},\n  note      = {* Equal contribution}\n}\n```\n\n", "url": "https://wpnews.pro/news/show-hn-impressive-hand-detection-ai-algorithm", "canonical_source": "https://github.com/aryafarkhondeh/cp_hand_demo", "published_at": "2026-09-28 08:50:45+00:00", "updated_at": "2026-09-28 09:20:03.877067+00:00", "lang": "en", "topics": ["computer-vision", "machine-learning", "artificial-intelligence", "ai-research", "developer-tools"], "entities": ["Idiap Research Institute", "Hiera-Hand", "ChildPlay-Hand", "Hiera-Base", "YOLO26m-pose", "BoT-SORT", "Arya Farkhondeh", "Samy Tafasca"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/show-hn-impressive-hand-detection-ai-algorithm", "markdown": "https://wpnews.pro/news/show-hn-impressive-hand-detection-ai-algorithm.md", "text": "https://wpnews.pro/news/show-hn-impressive-hand-detection-ai-algorithm.txt", "jsonld": "https://wpnews.pro/news/show-hn-impressive-hand-detection-ai-algorithm.jsonld"}}