{"slug": "the-ai-zombie-trend-from-2-photos-per-job-prompt-assembly-role-drift-fixes-and-a", "title": "The AI zombie trend from 2 photos: per-job prompt assembly, role-drift fixes, and a 5-model reference-to-video bench", "summary": "A developer building the Prismojo reference-to-video pipeline documented that Vidu Q4 produced a correct zombie-transformation clip in 4 of 4 attempts, while Kling O3 standard succeeded in 1 of 6, Wan 2.7 in 0 of 1, and Seedance 2.0 Mini rejected real-person reference photos outright, in a five-model bench using two input photos. The developer reported that a fixed prompt failed four ways — role drift, wrong scale, prop misassignment and dead time — and fixed them with capitalized role names (THE ZOMBIE / THE SURVIVOR), explicit real-world proportions for mixed species, assigning the flashlight to the survivor, and timed beats that place the hug at about 45% of the clip with the match cut 1–2 s later. Vidu Q4 was the most reliable model but slow at roughly 5–7 minutes per 5-second clip, so jobs run in the background, and the developer noted some strong models refuse real-person photos on policy grounds.", "body_md": "The “zombie love story” format is everywhere on TikTok and Reels: one person walks out of fog as a zombie, the other recognises them, and on the hug there is a hard match cut to a warm field where both are healthy again. It looks like a template, but getting it reliably from two arbitrary photos took real work. Here is the prompt structure, the failure modes, and a small model bench.\n\nDisclosure: I build [Prismojo](https://prismojo.com/), which runs this pipeline. Everything below also works with your own API keys.\n\n**Two inputs.** Slot A becomes the zombie, Slot B is the survivor. One subject per photo.\n\n**Subject detection.** AWS Rekognition labels + faces. A face means “person”; otherwise the top animal label ≥70% gives a noun (“dog”, “cat”…).\n\n**Per-job prompt.** Each subject is named, roles are cast once, and the beats are timed to the clip length.\n\n**Reference-to-video.** Both photos are passed as references. Output is a 9:16 MP4 with no audio; users add the trending sound when posting.\n\nOn a dog (A) + person (B) pair, our first fixed prompt broke in four ways:\n\n**Role drift:** the model’s prior is “zombies are people”, so the person became the zombie.\n\n**Scale:** the zombie pet came out giant.\n\n**Props:** the flashlight jumped to the zombie, or was “held” by the dog.\n\n**Dead time:** 10–15 s clips mostly stood still.\n\nWhat fixed it: capitalised role names (THE ZOMBIE / THE SURVIVOR) instead of pronouns; explicit real-world proportions for mixed species; the light always belongs to the survivor; and timed beats. The hug lands at about 45% of the clip and the cut 1–2 s later, so the match cut has a pose to match.\n\nReplace `Image 1` / `Image 2` with your model’s reference syntax:\n\n```\nCast — exactly two characters. THE ZOMBIE is the person from Image 1, as a zombie: the same face, hair, body and clothing as in Image 1, with pale ash-grey skin, faint dark veins, sunken milky-white eyes and dusty, torn clothes. THE SURVIVOR is the person from Image 2, looking exactly as in Image 2 and healthy the whole time. Never swap or merge their roles, faces or bodies.\n0–4 s: a cold, desaturated ruined street with fog and drifting ash. THE SURVIVOR stands in the foreground, side-on, face visible, holding a lit flashlight. THE ZOMBIE appears out of the fog and moves steadily toward THE SURVIVOR, holding nothing. THE SURVIVOR recognises them and slowly lowers the flashlight; at 4 s they embrace.\nAt 5 s, hard match cut on the hug to a warm golden-hour wheat field: the same two in the same hug, and only now THE ZOMBIE is healthy again, exactly as in Image 1; both happy.\nSingle smooth camera, continuous motion. No other people, no text, no blood, wounds, weapons, biting or violence. Cinematic, not graphic. Vertical 9:16, no audio.\n```\n\nFor a pet zombie, use: *“…the dog from Image 1, as a zombie dog: still exactly that dog on four legs, same breed and markings, at its real-life size. It never gets a human body or clothes.”* That last sentence fixed a “dog-headed person” output.\n\nThe “no blood / weapons / biting” line matters: without it, several providers’ filters reject the job.\n\n| **Model** | **Accepts real faces** | **Zombie look + story correct** | \n|---|---|---|\n| Vidu Q4 | yes | 4/4 | \n| Kling O3 standard | yes | 1/6 (no zombie effect or swapped roles) | \n| Hailuo H3 | yes | 1/1, clothing/hair drift | \n| Wan 2.7 | yes | 0/1 (ending stayed zombie) | \n| Seedance 2.0 Mini | rejected real-person references | — | \n\nTiny samples, so directional only. The surprise was the policy side: some strong models refuse real-person photos outright. Vidu was the most reliable but slow (about 5–7 min per 5 s clip), so jobs run in the background.\n\nSharp, front-facing photos, one subject each, no sunglasses.\n\nMatch the lighting of both photos; indoor + outdoor breaks the match cut.\n\nRun a short 4–5 s draft first to check photo order.\n\nAdd sound when posting: a beat that drops as the flashlight lowers sells the cut.\n\nIf you’d rather not wire up APIs, the hosted version is at [prismojo.com/features/ai-zombie](https://prismojo.com/features/ai-zombie).\n\n**Which photo becomes the zombie?** Image 1 / Slot A. Swap them to flip roles.\n\n**Pets?** Yes, with the species-aware line above. They come out dusty, not gory.\n\n**Why no audio?** Trend audio is licensed per platform, so adding it at post time is safer.\n\n**Open question:** has anyone kept identity stable across a hard cut on an open-weight model (Wan, HunyuanVideo)? I’d love to compare prompts.", "url": "https://wpnews.pro/news/the-ai-zombie-trend-from-2-photos-per-job-prompt-assembly-role-drift-fixes-and-a", "canonical_source": "https://discuss.huggingface.co/t/the-ai-zombie-trend-from-2-photos-per-job-prompt-assembly-role-drift-fixes-and-a-5-model-reference-to-video-bench/190939#post_1", "published_at": "2026-10-23 06:57:31+00:00", "updated_at": "2026-10-23 07:10:13.124318+00:00", "lang": "en", "topics": ["generative-ai", "ai-tools", "ai-products"], "entities": ["Prismojo", "Vidu Q4", "Kling O3 standard", "Hailuo H3", "Wan 2.7", "Seedance 2.0 Mini", "AWS Rekognition", "TikTok"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/the-ai-zombie-trend-from-2-photos-per-job-prompt-assembly-role-drift-fixes-and-a", "markdown": "https://wpnews.pro/news/the-ai-zombie-trend-from-2-photos-per-job-prompt-assembly-role-drift-fixes-and-a.md", "text": "https://wpnews.pro/news/the-ai-zombie-trend-from-2-photos-per-job-prompt-assembly-role-drift-fixes-and-a.txt", "jsonld": "https://wpnews.pro/news/the-ai-zombie-trend-from-2-photos-per-job-prompt-assembly-role-drift-fixes-and-a.jsonld"}}