No Tongues! AI Researchers Address Missing Anatomy in Facial Animation

An Australia-China academic collaboration published TongueReenact in July, a framework it describes as the first for cross-identity tongue dynamics transfer in face reenactment, combining 3D Gaussian Splatting, the FLAME parametric face model, Neural Radiance Fields and diffusion-based video generation. The pipeline builds a tongue-segmentation framework on SegmentAnything to generate pseudo-labels, addressing the lack of annotated tongue data, and spatially grounds the synthesized tongue to the source face geometry rather than treating it as generic inpainting. The paper's promised code and a supplementary video had not been released as of publication.

Anderson's Angle https://www.unite.ai/series/andersons-angle/ No Tongues AI Researchers Address Missing Anatomy in Facial Animation Add Unite.AI to your preferred sources on Google https://www.google.com/preferences/source?q=unite.ai Missed in 2026: It’s that time of year when the research scene dips and spikes in quality and volume in lockstep with the last few conferences https://www.r-bloggers.com/2024/12/should-you-submit-papers-before-christmas-submission-percentages-across-months-of-the-year/ :~:text=Submissions%20then%20decline%20in%20November of November and December. To traverse these sporadic famines, we’re taking a look at some of the more interesting papers that nearly made the cut in 2026, but were ousted by other works, in more competitive submission periods. Until recently, deepfake practitioners, amateur and professional alike, had bigger concerns to address than whether or not the reconstructed human appeared to have a tongue. After all, the tongue is the most mercurial of facial features – not evident at all in ‘mumblers’, and only intermittently observed in most normal contexts. Consequently, when you do need it, as far as AI reconstructions are concerned, it’s not there. In a climate where amateur diffusion-based reconstruction has reached a stunning level of realism, and our critical faculties in regard to AI humans is becoming ever more discerning, it is perhaps time to address the issue. Licking the Problem In July an academic collaboration https://arxiv.org/pdf/2607.28039 between Australia and China offered a pipeline to address the lack of tongue simulation in AI workflows. Titled TongueReenact , the proposed framework throws practically every method in the deepfake toolkit at the problem, including Gaussian Splatting https://www.unite.ai/image-editing-with-gaussian-splatting/ , parametric humans https://www.unite.ai/restructuring-faces-in-videos-with-machine-learning/ , Neural Radiance Fields https://www.unite.ai/nerf-the-challenge-of-editing-the-content-of-neural-radiance-fields/ NeRF , and diffusion-based https://www.unite.ai/a-notable-advance-in-human-driven-ai-video/ video generation sorry GANs https://www.unite.ai/gan-as-a-face-renderer-for-traditional-cgi/ , you didn’t make this cut . Though the project’s promised code does not appear to have been delivered since the paper was published, and though no supplementary video is yet released, this is nonetheless a landmark offering, since it is the first to address the tongue in AI-based reconstruction, outside of the medical sphere – and to address the critical lack of annotated data, which is essential for any proposed solution. The researchers made innovative use of diverse existing frameworks and data, in order to create a tongue-segmentation framework based on SegmentAnything https://www.unite.ai/segment-anything-model-computer-vision-gets-a-massive-boost/ , and to generate pseudo-labels https://arxiv.org/pdf/2408.07221v1 for the newly-segmented data. The paper states: ‘ We present the first framework for cross-identity tongue dynamics transfer in face reenactment. Given a source frame and a driving video, our method transfers both facial motion and tongue articulation onto the source identity. ‘The central insight of our work is that tongue synthesis cannot be treated as a generic inpainting problem. The tongue must be spatially grounded to the source face geometry, which itself changes with every driving frame.’ The system begins by transferring the driving person’s facial movements onto the source face, using 3D Gaussian Splatting and the FLAME https://is.mpg.de/uploads/publication attachment/attachment/400/paper.pdf facial model. This produces an animated face with the correct pose and expression, but still lacking a tongue. A Stable Diffusion https://www.unite.ai/stable-diffusion-3-5-innovations-that-redefine-ai-image-generation/ model is then used to generate the missing tongue, guided by its appearance in the driving video and the geometry of the reconstructed face. Regarding the need to address this shortfall in facial reconstruction, the authors observe : ‘During expressive actions such as laughter, disgust, or surprise, the tongue can dominate the visible oral region entirely. Yet nearly all existing reenactment and avatar systems either model the oral interior as a textureless void or leave it https://arxiv.org/abs/2401.10215 entirely https://arxiv.org/abs/2312.02069 unconstrained https://arxiv.org/abs/2507.23143 . ‘This neglect is not coincidental, as the tongue is a uniquely transient structure, fully concealed at rest, intermittently exposed during speech, and suddenly dominant in extreme expressions, making it fundamentally resistant to the static parametric modeling applied to the rest of the face. ‘Addressing this overlooked challenge is the central motivation of our work.’ In tests, the new method substantially outperformed five competing facial animation systems, achieving a more than 100% accuracy advantage over its closest rival on key measures of tongue positioning and size – though it has to be conceded that tests of this kind are hard to contrive when one is breaking new ground, and when prior frameworks address the new challenge minimally, or not at all. Let’s take a closer look at the new approach. Method To develop a system capable of identifying tongues in ordinary facial footage, the NERSemble https://tobias-kirschstein.github.io/nersemble/ dataset of multi-view recordings of human heads was used alongside synthetically generated frames from the VFHQ https://liangbinxie.github.io/projects/vfhq/ dataset, which features over 16,000 high-quality facial videos: Press to play An interestingly ‘lingual’ example of Radiance Field reconstruction from the Nersemble project site. However, the tongue seen here is treated as generic geometry, and has no distinct domain. Source https://tobias-kirschstein.github.io/nersemble/ The M askGAN https://arxiv.org/abs/1907.11922 face-parsing model was used to locate the mouth in each image, after which the TongueSAM https://arxiv.org/pdf/2308.06444 medical detector was used to identify the likely tongue region, before passing it to SegmentAnything, from which an initial outline, i.e., pseudo-label of the visible tongue was generated. The resulting masks were divided into three groups, according to how much of the tongue was visible. Incorrect examples were discarded, and promising masks manually corrected to distinguish the tongue from surrounding lips, teeth, and other mouth tissue. These refined annotations were used to train the BiSeNet https://arxiv.org/pdf/2004.02147 image segmentation network. The model was then applied to additional NERSemble subjects and synthetic VFHQ frames, with its predictions checked and corrected by humans before further training – a process that was repeated until performance on previously unseen subjects stabilized: For tongue synthesis, the oral region identified by BiSeNet was slightly enlarged before being passed to the diffusion model, giving the generated tongue room to extend beyond the initial segmentation, as the mouth changed shape. Generation was performed in the latent space of Stable Diffusion Image Variations https://huggingface.co/lambda/sd-image-variations-diffusers , with separate guidance derived from the reconstructed facial geometry and the tongue visible in the driving frame, and only the identified mouth/tongue region regenerated. Data and Tests BiSeNet was trained on two output classes derived from the aforementioned data processing stages: background and tongue . Training was carried out over 50,000 iterations at a learning rate of 10<sup -3</sup , using 512x512px images on batch size 4, on a single NVIDIA A40 GPU with 48GB of VRAM. The training data consisted of the aforementioned NERSemble and VFHQ frames. For the reenactment portion, the pretrained GAGAvatar https://arxiv.org/pdf/2410.07971 Gaussian Splatting backbone was used. The Stable Diffusion Image Variations checkpoint was also used, with the VAE https://www.unite.ai/what-is-synthetic-data/ :~:text=VAEs%20are%20unsupervised%20machine%20learning%20models%20that%20make%20use%20of%20encoders%20and%20decoders and CLIP https://arxiv.org/pdf/2103.00020.pdf encoder frozen https://arxiv.org/pdf/2106.09226 . The Guidance encoders and denoising UNET https://web.archive.org/web/20260930121243/https://www.geeksforgeeks.org/machine-learning/u-net-architecture-explained/ were then trained on the NERSemble data for 85,000 iterations at batch size https://archive.is/8quL2 batchsize 8, with 512×512 inputs. For inference, 20 DDIM https://arxiv.org/html/2010.02502v4 :~:text=denoising%20diffusion%20implicit%20models%20%28DDIMs%29%2C steps were used for many current diffusion models, this is considered a high amount, designed to favor detail, with certain inevitable tradeoffs . Rival avatar frameworks tested were GPAvatar https://arxiv.org/abs/2401.10215 ; the feted LivePortrait https://github.com/KlingTeam/LivePortrait ; X-NeMo https://arxiv.org/pdf/2507.23143 ; and X-Portrait https://github.com/bytedance/X-Portrait . Each of these was evaluated against 9,000 cross-identity VFHQ source/driving pairs. Metrics used were Learned Perceptual Image Patch Similarity https://archive.ph/QRkZl lpips LPIPS ; Tongue Presence the proportion of frames where a tongue is detected ; Tongue IoU the overlap between the generated and driving tongue regions ; and Tongue Area Similarity how closely the generated tongue’s size matched the driving tongue . Of the initial comparison, the authors state: ‘Our method achieves the best performance across all four tongue transfer metrics, with a Tongue IoU of 0.3582 and Tongue Area Similarity of 0.4338, more than double the nearest competitor, confirming substantially more accurate tongue dynamics transfer. ‘Tongue Presence of 0.7891 further demonstrates that our method reliably synthesizes visible tongue articulation where existing methods frequently produce none.’ Though it’s not something that we can judge for ourselves, due to the current lack of supplementary video material, the authors also tested the system for temporal consistency using T-LPIPS https://arxiv.org/pdf/2602.08395v1 : Here the authors comment: ‘Our full model achieves substantially lower flow jitter 0.218543 vs. 0.622901 and flow magnitude 0.513165 vs. 1.254073 , confirming that geometry-anchored denoising produces smooth and stable tongue motion. ‘Lower T-LPIPS further confirms that tongue synthesis does not compromise temporal coherence.’ Conclusion The uncanny valley https://www.unite.ai/why-physical-ai-is-harder-than-we-thought/ :~:text=The%20uncanny%20valley%20effect effect is being gradually overcome by some of the latest diffusion-based video models. However, there is a certain novel psychological barrier beginning to manifest, whereby people have a negative take on AI-generated material for reasons other than the quality of what they are watching https://academic.oup.com/jcr/advance-article/doi/10.1093/jcr/ucag013/8672493 . This is set to complicate the business of qualitative evaluation, though I suppose that metrics based on ‘blind’ tests where the user wants to downgrade AI output, but fears to downgrade real output could help to develop a kind of ‘shadow’ scale for AI realism. Therefore, though there are strong research strands for the reproduction via AI of secondary identity traits such as hair https://martinanderson.ai/articles.html :~:text=Hair , emotion simulation https://www.unite.ai/altering-emotions-in-video-footage-with-ai/ , gaze https://www.unite.ai/detecting-eye-contact-from-body-pose-with-machine-learning/ , and now tongue movement, they may, for the moment, be moot points. My conversion of the authors’ inline citations to hyperlinks. First published Wednesday, November 11, 2026