Generalized Audio-Driven Synthesis of Precise Drummer Motion

Researchers from Tel-Aviv University, DisneyResearch|Studios, and ETH Zurich introduced a generative diffusion framework with a dual-objective loss function that decouples skeletal integrity from drumstick precision, enabling centimeter-level stick precision without sacrificing natural body dynamics. The model, trained on a new dataset with data augmentation, generalizes to in-the-wild audio and outperforms existing methods on two novel metrics: impact-to-target distance for spatial precision and audio-motion correlation for temporal alignment. User studies show the generated motion is often indistinguishable from ground-truth performances.

Generalized Audio-Driven Synthesis of Precise Drummer Motion In this work, we introduce a generative diffusion framework featuring a dual-objective loss function that decouples skeletal integrity from drumstick precision, thus enabling centimeter-level stick precision without sacrificing natural body dynamics. August 18, 2026 arXiv 2026 Authors Álvaro G. Iñesta The Blavatnik School of Computer Science and AI, Tel-Aviv University Mattia Ryffel DisneyResearch|Studios Amit H. Bermano DisneyResearch|Studios/The Blavatnik School of Computer Science and AI, Tel-Aviv University/ETH Zurich Robert W. Sumner DisneyResearch|Studios/ETH Zurich Martin Guay DisneyResearch|Studios Generalized Audio-Driven Synthesis of Precise Drummer Motion Music-driven character animation enables and enhances transformative applications in entertainment and interactive education. However, synthesizing realistic drumming motion from audio remains challenging due to the inherent tension between high-acceleration dynamics and the need for extreme spatial-temporal precision. Existing approaches, often reliant on motion matching or MIDI input, struggle with generalizing to diverse real-world audio. Moreover, the field lacks standardized evaluation metrics capable of distinguishing precise drumming from noisy motion. In this paper, we introduce a generative diffusion framework featuring a dual-objective loss function that decouples skeletal integrity from drumstick precision, thus enabling centimeter-level stick precision without sacrificing natural body dynamics. Additionally, leveraging our own dataset and data augmentation strategy, the model generalizes to non-curated, in-the-wild audio. To rigorously evaluate performance, we propose two novel metrics: an impact-to-target distance to quantify spatial precision and an audio-motion correlation score to assess temporal alignment. Our quantitative analysis and user studies demonstrate that our system generates high-quality motion that is often indistinguishable from ground-truth performances.