Additional results on the MTBench_HQ dataset further demonstrate that our method consistently performs effective motion transfer across a wide range of videos.
Diffusion models have enabled motion transfer that reflects motion from a reference video while aligning with a given target prompt. Prior methods typically require model training or fine-tuning, limiting their flexibility and broad applicability. Recently, training-free methods have received attention as a more generalizable alternative. Among these, the approach utilizing self-attention query features from the reference video provides a simple yet effective strategy. However, direct use of reference query features often generates videos with unwanted visual details from the reference video and frame-level flickering. In this paper, we analyze query features in terms of motion transfer and propose a query fusion method that modulates the reference query features to address these problems. Specifically, our method, AQUA, adaptively fuses the reference and target query features to remain faithful to the target prompt while following the reference motion. Furthermore, AQUA employs a multi-frame guidance to ensure temporal consistency. Extensive experiments demonstrate that our method outperforms existing methods without additional training or optimization.
Query analysis. Visualization of the reference video, PCA projection query features, and their t-x slices. The query feature maintains consistent spatial representations across time, aligned with the reference video.
Query-induced issues. (a) Failure example. Reference bias is visible in the leaked mountain background and the backpack-like artifact (green box), while temporal inconsistency appears as an abrupt green object (yellow box). (b) Our result. The generated video aligns with the target prompt and exhibits improved temporal consistency.
Overall framework of AQUA. (a) Query features extracted via DDPM inversion guide video generation. (b) DAWN adaptively fuses reference and target queries via the discrepancy function D. (c) RAM ensures temporal consistency by aggregating attention outputs from the current (i), previous (i-1), and first frames.
We provide qualitative results for our method and other baseline models, where videos are generated based on the reference video and guided by the target prompt. Our method produces videos that are both temporally consistent and semantically aligned with the given prompt.
We conduct our experiments on the MTBench HQ. The evaluation consists of three aspects: (1) Automatic metrics, referring to algorithmically computed measures that assess prompt alignment, temporal consistency, and motion fidelity; (2) Human evaluation, providing subjective assessment of the same criteria; and (3) Efficiency metrics, including computation time and peak memory usage for generating a motion-transferred video from a reference video
