Sparse-View Interpretable 3D Animal Behavior Representations for Neural Encoding and Decoding

Researchers introduced SABLE (Sparse-view Animal Behavior Latent Embeddings), a self-supervised framework that reconstructs 3D animal behavior from two-view video without ground-truth 3D labels, according to the arXiv paper 2609.36217v1. Across the International Brain Lab and Cheese3D datasets, SABLE matched or exceeded prior state-of-the-art performance in neural encoding and decoding, while state-of-the-art methods failed or produced degenerate solutions on the same sparse-view task. Once pretrained across animals, SABLE generalizes zero-shot to unseen animals without animal-specific calibration or retraining.

arXiv:2609.36217v1 Announce Type: new Abstract: A deeper understanding of brain function requires a precise, structured characterization of behavior.Yet, extracting behavioral representations from video in a form suitable for scientific analysis remains a fundamental challenge. Many prior studies represent behavior via pose estimation or nonlinear video embeddings. However, pose tracking discards rich information beyond predefined keypoints, while nonlinear video embeddings lack interpretability. We address this limitation with SABLE Sparse-view Animal Behavior Latent Embeddings , a self-supervised framework that leverages a geometric inductive bias to learn behavior representations.By augmenting a multi-view transformer with priors from monocular depth and pose estimation, SABLE reconstructs 3D animal behavior from extremely sparse views while learning explicit 3D latent structure. Without ground-truth 3D labels, it reliably recovers 3D behavior from two-view videos, whereas state-of-the-art SOTA methods fail or yield degenerate solutions. Across the International Brain Lab and Cheese3D datasets, we demonstrate that SABLE learns 3D representations that match or exceed prior SOTA performance in neural encoding and decoding. Once pretrained across animals, SABLE serves as an off-the-shelf model that generalizes zero-shot to unseen animals without animal-specific calibration or retraining. Our method establishes 3D-aware video embeddings that capture complex behavior, opening new avenues for studying brain-behavior relationships.