Just got a commission to pair a character portrait with a voiceover clip. Tried out this MultiTalk thing—it’s an open-source framework by MeiGen. The idea is you throw in one image plus an audio clip, and it syncs the lip movements and facial expressions automatically.
I ran the official image2video workflow in ComfyUI. For the red nodes, just use Manager to install whatever’s missing. The real dealbreaker is the VRAM—the dev says you need 48G to run it stable. My card can barely handle it, so I can’t crank up the resolution. Trying to run it in HD just crashes it outright. For the audio driving part, I used chinese-wav2vec2, and honestly, the Chinese lip sync turned out way better than I expected.
One little trap: you gotta include the LoRA trigger word “detailz” in your prompt. Forget it, and the image comes out blurry. Don’t mess with the step count either—the dev says this workflow is already teetering on the edge, and I believe it. Make sure your source image is as clear and high-quality as possible; if you feed it a blurry one, the output will be even blurrier. Overall, it works great for single character portraits reciting poems or singing, and it even supports multi-person dialogue. But man, the hardware requirements are brutal.