Remake the Trend

How AI face swap videos work

Updated · Remake the Trend

Two different ways. For AI scenes, your photo is turned into the scene's first frame and a video model animates it from a written description. For real TikTok clips, a motion model takes the moves from the original clip and puts them on the person in your photo. Neither pastes a face onto a video frame by frame, as older face swaps did.

AI scenes: first frame, then motion

  1. Your photo and the scene's description go to an image model (Google's, here), which draws you in the scene: that is the free preview.
  2. The preview becomes the first frame of the video.
  3. An image-to-video model (here ByteDance's Seedance 2.0 Fast) continues it for 10 seconds from a written description of what happens, and makes the sound with it.

Real clips: motion transfer

For a real TikTok clip, the model (here Kuaishou's Kling 2.6 Motion Control) reads how the dancer moves in the original 10 seconds and makes the person from your first frame move the same way, with the clip's own sound. The whole body is yours, not just the face.

How is that different from old face swap apps?

Classic face swap found the face in every frame of a video and replaced it with yours. It was fast, but the rest of the body, the hair and the skin tone stayed the original person's, and the edge of the face often showed. Video models draw every frame from scratch, so hair, body and clothes match, at the cost of a few minutes per video.

Why faces sometimes drift

Sources

Which models we use comes from our own setup; it may change when a better one appears.

  1. ByteDance Seed: Seedance 2.0 (text, image, audio and video in; video with sound out)
  2. OpenRouter: Seedance 2.0 (image-to-video with first-frame control)

Related