Uncovering the Blind Spots of OKS-mAP: A Unified Protocol for Occlusion, Joint-Angle, and Temporal Evaluation of 2D Pose Estimators

A new arXiv paper (2611.12911v1) proposes a three-module supplementary evaluation protocol for 2D human pose estimation, arguing that the standard OKS-mAP metric has blind spots on occlusion, joint-angle error and temporal stability. Across COCO val2017, CrowdPose, OCHuman and PoseTrack18 and four models from two architecture families (YOLO11s-pose, an occlusion-domain fine-tuned variant, ViTPose-B/L), the authors report the occlusion gap Delta_occ was negative in every combination, ranging from -0.19 to -0.47, meaning aggregate mAP overestimates occluded-scene performance. On PoseTrack18 val, pixel-domain OneEuro smoothing cut jump rates 9-10% at no matched-scope AP cost, and the protocol and harness are open-sourced as a complement to, not a replacement for, OKS-mAP.

arXiv:2611.12911v1 Announce Type: new Abstract: The dominant metric for 2D human pose estimation, OKS-mAP, has three blind spots: it does not distinguish visible from occluded keypoints, does not measure joint-angle error, and is silent on the temporal stability of tracking-free per-frame inference. We propose a supplementary protocol with three modules: M1 OKS-mAP stratified by ground-truth visibility, with Delta occ = mAP v=1 - mAP full , quantifying how much the aggregate metric masks occlusion-specific differences; M2 joint-angle error RMSE/MAE/P95 plus knee-valgus detection consistency; M3 frame-to-frame displacement, median-filter residual jitter, and jump rate, before and after OneEuro smoothing. We ran it on four datasets COCO val2017, CrowdPose, OCHuman, PoseTrack18 and four models from two architecture families YOLO11s-pose, an occlusion-domain fine-tuned variant, ViTPose-B/L , under a common detector-free oracle protocol and an open-source harness. Delta occ is negative in every combination -0.19 to -0.47 : aggregate mAP overestimates occluded-scene performance in every tested combination. Joint-angle error is amplified in occluded layers; angle metrics reveal differences mAP cannot localize, e.g., the smaller occlusion-trained model significantly outperforming a larger transformer in occluded elbows. On PoseTrack18 val, raw jitter matched between the two YOLO models; pixel-domain OneEuro smoothing cut jump rates 9-10% at no matched-scope AP cost; residual jitter persists after detector removal; the transformer pair's jump rate is significantly lower; smoothing benefits replicate across architectures. Stratified mAP tracks full mAP closely sign-consistency 1.0, 12/12 , serving magnitude diagnosis rather than re-ranking; angle-metric difference vectors especially knee partially diverge, suggesting independent information. The protocol and harness are open-sourced, complementing rather than replacing OKS-mAP.