Harmony-Aware Music-driven Motion Synthesis with Perceptual Constraint on UGC DatasetsXinyi Wu, Haohong Wang, Aggelos K. Katsaggeloshttps://arxiv.org/abs/2506.07076
Harmony-Aware Music-driven Motion Synthesis with Perceptual Constraint on UGC DatasetsWith the popularity of video-based user-generated content (UGC) on social media, harmony, as dictated by human perceptual principles, is critical in assessing the rhythmic consistency of audio-visual UGCs for better user engagement. In this work, we propose a novel harmony-aware GAN framework, following a specifically designed harmony evaluation strategy to enhance rhythmic synchronization in the automatic music-to-motion synthesis using a UGC dance dataset. This harmony strategy utilizes refin…