# HeyGen details TAVR, a video-reference system for steadier AI avatars > Source: > Published: 2026-08-27 10:50:16+00:00 # HeyGen details TAVR, a video-reference system for steadier AI avatars **First disclosed on March 3rd, the production research system uses up to 48 reference frames to preserve a person's identity across new scenes.** By [RuntimeWire Staff](/author/runtimewire-staff) ยท Published Primary source: [Hugging Face Newsroom](https://huggingface.co/blog/HeyGenAI/tavr) ## Why it matters TAVR shows how HeyGen is turning its founder's camera-replacement thesis into proprietary model infrastructure. Video references improve avatar consistency, but they also raise compute, consent and verification stakes. [Joshua Xu](https://www.heygen.com/author/joshua-xu?ref=runtimewire) built [HeyGen](https://www.heygen.com/?ref=runtimewire) around a blunt premise: people should be able to make videos without standing in front of a camera. TAVR, a research system developed with Nanyang Technological University, pushes that premise one step further by asking users to record a short reference clip rather than expecting an AI model to reconstruct a moving person from one photograph. [HeyGen](/models/fal/heygen-avatar3-digital-twin) first [disclosed TAVR on March 3rd](https://www.heygen.com/research/tavr?ref=runtimewire), and the researchers submitted their [technical report](https://arxiv.org/abs/2604.27918?ref=runtimewire) on April 30th. A [Hugging Face community article](https://huggingface.co/blog/HeyGenAI/tavr?ref=runtimewire) published August 27th offers a more accessible account of the system and says the work has been accepted to the SIGGRAPH Asia 2026 conference track. The paper also says TAVR has been deployed in production. Lead author [Zujin Guo](https://gseancdat.github.io/?ref=runtimewire) is an AI researcher at HeyGen and a PhD student in NTU's MMLab, where he is advised by professor Chen Change Loy. Guo previously earned an artificial intelligence master's degree from NTU and studied automation at Xi'an Jiaotong University. His earlier research covered video frame interpolation and human-centric video generation, the same underlying problem that TAVR turns toward a commercial constraint: an avatar has to remain recognizably yours after the background, pose and camera angle change. ### A video provides the details a photograph cannot Single-image avatar systems receive one angle, expression and lighting setup. They must infer how a person's mouth, face and silhouette should look across all the frames that follow. TAVR accepts a variable-length reference video, giving the model multiple poses and expressions from which to extract identity cues. The system can place the resulting avatar against a separate target background in one generation pass. That matters because adapting a single reference image to a new setting can introduce artifacts before the avatar model has generated its first frame. TAVR instead conditions directly on the video reference and the requested scene. The architecture starts with the 14-billion-parameter Wan2.1-T2V video diffusion model. A token-selection module uses facial bounding boxes to retain identity-relevant portions of the reference and discard background or redundant tokens. Reference Self-Attention then lets the target and reference tokens share context, while an audio cross-attention layer aligns speech with the generated frames and uses the reference audio to help locate speaking dynamics. That filtering step addresses the obvious cost of using video: every additional frame introduces tokens and compute. HeyGen's tests showed identity similarity improving as the reference grew from 12 to 48 frames, while its reported lip-sync and general quality measurements remained stable. The researchers also found a visible practical benefit. At 12 frames, one example generated incorrect teeth; the artifact disappeared when the system received 48 frames containing better inner-mouth evidence. TAVR generates clips of about three seconds at a time. For longer output, it passes final latent motion frames from one clip into the next and retains the first latent frame as a global appearance anchor. The mechanism is designed to curb the familiar avatar failure in which a face gradually drifts toward a similar-looking stranger. ### HeyGen trained for scene changes, then optimized for identity The training process runs in three stages. TAVR first learns appearance copying from reference and target segments taken from the same scene. HeyGen then fine-tunes it on different videos of the same person, forcing the model to separate identity from lighting and background. A final preference-optimization stage uses ArcFace identity similarity as its reward, with spatial masking that concentrates the signal on the avatar instead of the surrounding scene. That progression reflects the distinction between a polished demo and a dependable product. A model can reproduce a person inside the environment it has already seen. HeyGen needs the same person to survive a new outfit, backdrop, camera position and runtime because those changes form the commercial pitch for its avatar tools. HeyGen introduced [Avatar V](https://www.heygen.com/blog/announcing-avatar-v?ref=runtimewire) in May, describing a product that creates multi-angle and longer-form avatar videos from a 15-second recording. The TAVR paper directs readers to Avatar V, while the Hugging Face article says the research directly enables HeyGen's video-reference avatar product. HeyGen has not identified every model component used in the commercial system, so TAVR is best understood as a disclosed piece of the research stack behind that product direction rather than a full technical specification for Avatar V. The connection runs back to Xu's founding thesis. A Carnegie Mellon computer science graduate who studied robotics and computer vision, Xu has said his own discomfort recording camera tests helped inspire the product. Before HeyGen, Xu was a lead engineer at Snapchat. He worked on ads ranking, machine learning and computational photography. Xu and co-founder [Wayne Liang](https://www.crunchbase.com/person/wang-liang-027e?ref=runtimewire) started HeyGen in December 2020 after developing an idea they described as replacing the camera. HeyGen's model work now supports a substantial software business. In June, [Xu said HeyGen had passed $200 million in annual recurring revenue](https://www.heygen.com/blog/heygen-surpasses-200m-arr?ref=runtimewire), doubling the company-reported figure in eight months. HeyGen also claimed over 30 million users and usage within 85% of the Fortune 100. Those figures have not been independently audited. Benchmark led HeyGen's [$60 million Series A](https://www.heygen.com/blog/announcing-our-series-a?ref=runtimewire), announced June 20, 2024, at an [approximately $500 million post-money valuation](https://news.bloomberglaw.com/private-equity/ai-video-startup-heygen-valued-at-500-million?ref=runtimewire). Thrive Capital, BOND and SV Angel also participated. ### The strongest result comes from HeyGen's own test On a benchmark of 158 cross-scene video pairs constructed by the authors, TAVR recorded an overall quality score of 16.42 using 20 reference frames. HuMo, the next-highest method in the reported comparison, scored 14.13. With 48 frames, TAVR reached identity-similarity scores of 0.83 against the reference and 0.69 against the target. The comparison included StableAvatar, EchoMimicV3, OmniAvatar, HuMo and [LongCat-Video](/models/fal/longcat)-Avatar. The researchers tested baselines using a raw cross-scene image and a second setup in which [Qwen-Image](/models/fal/qwen-image)-Edit first adapted the image to the target setting. TAVR remained ahead when HuMo received the single frame most similar to the target, according to the authors. The benchmark and headline metric were created by the TAVR team, and the result has yet to carry the weight of independent replication. The [project page](https://gseancdat.github.io/projects/TAVR.html?ref=runtimewire) labels its code as "Coming Soon," while the paper presents TAVR as production research rather than an open model release. The materials also leave production scale, inference cost and TAVR-specific customer usage undefined. HeyGen says its platform requires consent verification for digital twins. Better identity preservation raises the stakes of that safeguard: a model that can maintain a person's face across scenes is more useful to legitimate users and more capable when permission is missing. TAVR's technical contribution is a clearer route from a short recording to a persistent on-screen identity. HeyGen's product challenge is proving that persistence remains affordable, controllable and tied to the person who supplied the reference.