In this work, we introduce a generative diffusion framework featuring a dual-objective loss function that decouples skeletal integrity from drumstick precision, thus enabling centimeter-level stick precision without sacrificing natural body dynamics.
August 18, 2026
arXiv (2026)
Authors
Álvaro G. Iñesta (The Blavatnik School of Computer Science and AI, Tel-Aviv University)
Mattia Ryffel (DisneyResearch|Studios) Amit H. Bermano (DisneyResearch|Studios/The Blavatnik School of Computer Science and AI, Tel-Aviv University/ETH Zurich)
Robert W. Sumner (DisneyResearch|Studios/ETH Zurich)
Martin Guay (DisneyResearch|Studios)
Generalized Audio-Driven Synthesis of Precise Drummer Motion
Music-driven character animation enables and enhances transformative applications in entertainment and interactive education. However, synthesizing realistic drumming motion from audio remains challenging due to the inherent tension between high-acceleration dynamics and the need for extreme spatial-temporal precision. Existing approaches, often reliant on motion matching or MIDI input, struggle with generalizing to diverse real-world audio. Moreover, the field lacks standardized evaluation metrics capable of distinguishing precise drumming from noisy motion. In this paper, we introduce a generative diffusion framework featuring a dual-objective loss function that decouples skeletal integrity from drumstick precision, thus enabling centimeter-level stick precision without sacrificing natural body dynamics. Additionally, leveraging our own dataset and data augmentation strategy, the model generalizes to non-curated, in-the-wild audio. To rigorously evaluate performance, we propose two novel metrics: an impact-to-target distance to quantify spatial precision and an audio-motion correlation score to assess temporal alignment. Our quantitative analysis and user studies demonstrate that our system generates high-quality motion that is often indistinguishable from ground-truth performances.