Abstract
Cross-lingual voice cloning (CLVC) aims to synthesize speech in a target language while preserving the vocal identity of a source speaker who has no recorded speech in that language. Despite recent advances in multilingual text-to-speech systems, zero-shot CLVC remains challenging due to phonetic divergence across languages and the difficulty of maintaining speaker identity alongside linguistic intelligibility. In this work, we present a systematic evaluation of four state-of-the-art CLVC systems spanning autoregressive and diffusion-based architectures. Using English source speakers from the ACL-60/60 dataset, we evaluate zero-shot voice transfer across multiple target languages, including Arabic, Chinese, French, German, Russian, and Japanese. Systems are assessed using speaker similarity and content consistency metrics under a unified multilingual evaluation pipeline. We analyze how different modeling approaches autoregressive language modeling and diffusion-based flow matching handle the tradeoff between speech accuracy and speaker identity preservation across different architectural approaches. We further observe substantial performance variation across languages, with Arabic remaining particularly challenging under zero-shot transfer settings.- Anthology ID:
- 2026.iwslt-1.12
- Volume:
[Proceedings of the 23rd International Conference on Spoken Language Translation (IWSLT 2026)](/volumes/2026.iwslt-1/)- Month:
- July
- Year:
- 2026
- Address:
- San Diego, USA (in-person and online)
- Editors:
Elizabeth Salesky,Antonios Anastasopoulos,Matteo Negri,Marcello Federico- Venues:
[IWSLT](/venues/iwslt/)|[WS](/venues/ws/)- SIG:
[SIGSLT](/sigs/sigslt/)- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 103–110
- Language:
- URL:
[https://aclanthology.org/2026.iwslt-1.12/](https://aclanthology.org/2026.iwslt-1.12/)- DOI:
- Cite (ACL):
- Mo Ahtasam, Jamal uddin, and Mohammad Nadeem. 2026. Balancing Linguistic Intelligibility and Speaker Identity in Zero-Shot Cross-Lingual Voice Cloning. InProceedings of the 23rd International Conference on Spoken Language Translation (IWSLT 2026), pages 103–110, San Diego, USA (in-person and online). Association for Computational Linguistics. - Cite (Informal):
[Balancing Linguistic Intelligibility and Speaker Identity in Zero-Shot Cross-Lingual Voice Cloning](https://aclanthology.org/2026.iwslt-1.12/)(Ahtasam et al., IWSLT 2026)- PDF:
[https://aclanthology.org/2026.iwslt-1.12.pdf](https://aclanthology.org/2026.iwslt-1.12.pdf)