Wan-Animate:阿里开源统一角色动画和替换与整体复制AI大模型 Wan-Animate: Unified Character Animation and Replacement with Holistic Replication
美国
人工智能 AI大模型 AI视频
Wan-Animate:阿里开源统一角色动画和替换与整体复制AI大模型 Wan-Animate: Unified Character Animation and Replacement with Holistic Replication

Wan-Animate:阿里开源统一角色动画和替换与整体复制AI大模型 Wan-Animate: Unified Character Animation and Replacement with Holistic Replication

标签:
其他站点: 魔搭测试

我们推出了 Wan-Animate,一个用于角色动画和替换的统一框架。给定一张角色图像和一段参考视频,Wan-Animate 可以通过精确复制视频中角色的表情和动作来为角色制作动画,从而生成高保真度的角色视频。或者,它可以将动画角色集成到参考视频中替换原始角色,复制场景的光照和色调,实现与环境的无缝融合。

Wan-Animate 基于 Wan 模型构建。为了使其适用于角色动画任务,我们采用了一种改进的输入范式,以区分参考条件和生成区域。这种设计将多个任务统一为一个通用的符号表示。我们使用空间对齐的骨架信号来复制从源图像中提取的肢体运动和隐式面部特征,从而重现表情,从而生成具有高度可控性和表现力的角色视频。

此外,为了增强角色替换过程中的环境融合,我们开发了一个辅助模块 Relighting LoRA。该模块在应用适当的环境光照和色调的同时,保持角色外观的一致性。实验结果表明,Wan-Animate 达到了最佳性能。我们致力于开源模型权重及其源代码

We introduce Wan-Animate, a unified framework for character animation and replacement. Given a character image and a reference video, Wan-Animate can animate the character by precisely replicating the expressions and movements of the character in the video to generate high-fidelity character videos. Alternatively, it can integrate the animated character into the reference video to replace the original character, replicating the scene's lighting and color tone to achieve seamless environmental integration.

Wan-Animate is built upon the Wan model. To adapt it for character animation tasks, we employ a modified input paradigm to differentiate between reference conditions and regions for generation. This design unifies multiple tasks into a common symbolic representation. We use spatially-aligned skeleton signals to replicate body motion and implicit facial features extracted from source images to reenact expressions, enabling the generation of character videos with high controllability and expressiveness.

Furthermore, to enhance environmental integration during character replacement, we develop an auxiliary Relighting LoRA. This module preserves the character's appearance consistency while applying the appropriate environmental lighting and color tone. Experimental results demonstrate that Wan-Animate achieves state-of-the-art performance. We are committed to open-sourcing the model weights and its source code.

资源下载此资源仅限注册用户下载,请先

相关导航

暂无评论

您必须登录才能参与评论!
立即登录
暂无评论...