ComfyUI + AnimateDiff-Lightning本机快速生成高质量视频动画

本文是Youtube影片的辅助文档，在观看完成Youtube原影片后，使用本文档效果会更佳。

Youtube完整影片

AnimateDiff-Lightning介绍

AnimateDiff-Lightning是一款由字节跳动基于AnimateDiff优化算法推出的全新视频和动画生成模型。相比于AnimateDiff，它的最大特点在于速度更快，生成高质量视频所需步数大大减少。而ComfyUI则提供了便捷的工作流，使得生成视频和动画的过程更加简单易行。

ComfyUI安装

需要注意的是ComfyUI的显存使用策略，如果策略应用不当的话。会频繁遭遇Out of memory错误。

AnimateDiff_lightning安装

工作流下载

下载下面两个工作流，保存到本地，稍后运行工作流的时候需要。

https://huggingface.co/ByteDance/AnimateDiff-Lightning/raw/main/comfyui/animatediff_lightning_workflow.json

https://huggingface.co/ByteDance/AnimateDiff-Lightning/raw/main/comfyui/animatediff_lightning_v2v_openpose_workflow.json

Custom Nodes安装

打开ComfyUI Manager，点击Install Missing Custom Nodes，逐个安装：

ComfyUI-AnimateDiff-Evolved
ComfyUI-VideoHelperSuite
ComfyUI-Advanced-ControlNet
comfyui_controlnet_aux

文字生视频模型下载

下载以下模型，保存到/models/checkpoints/目录下

• DreamShaper
• MajicMix Realistic
• ToonYou
• Mistoon Anime

AnimateDiff-Lightning模型下载

打开下面链接，下载所有animatediff_lightning_*step_comfyui.safetensors文件，并保存到/custom_nodes/ComfyUI-AnimateDiff-Evolved/models/下

https://huggingface.co/ByteDance/AnimateDiff-Lightning/tree/main

ControlNet OpenPose模型下载

打开ControlNet模型下载页面
https://huggingface.co/lllyasviel/ControlNet-v1-1/tree/main
下载control_v11p_sd15_openpose.pth，保存到/models/controlnet/下

视频生动画

• 从Load Video节点上传一个少于30秒的视频
• 把Frame rate设置成和原视频一样的帧率
• 在Load Checkpoint选择一个动画风格模型MistoonAnime
• 设置Load ControlNet Model为control_v11p_sd15_openpose
• 选择AnimateDiff Loader 4步模型
• 在Clip Text Encode输入正负提示词
  • 正提示词：A beautiful girl dancing, black long hair, 25years old
  • 负提示词：blurry, blur, text, watermark, render, 3D, NSFW, nude, CGI, B&W, plastic, low-resolution, deep-fried, oversaturated
• 设置KSampler Steps为4, cfg为1, denose为1
• 点击Queue Prompt开始视频生动画

文字生视频

示例一正提示词
masterpiece, 25 year old women, smile, cleavage, black hair, medium breast, garden, sensual pose, facing the camera,close up, upper body 0: red dress 16: white dress 32

示例二负提示词
worst quality, low quality, lowres, monochrome, greyscale, multiple views, comic, sketch, bad anatomy, deformed, disfigured, watermark, multiple_views, mutation hands, mutation fingers, extra fingers, missing fingers, watermark, nude, nsfw

示例二正提示词
source_anime, cowboy shot, looking at viewer, smile, open mouth, surprised, 1girl, long hair, black hair, aqua eyes, bangs, sundress, holding basket, running, outdoors, easter egg, field, flower, tree, lake, mountain

示例二负提示词
3d, monochrome, greyscale