Bridging the Stability-Expressivity Gap: Synthetic Data Scaling and Preference Alignment for Low-Resource Spoken Language Models

Researchers at the University of Cambridge have identified a fundamental trade-off between stability and expressivity in spoken language models trained on synthetic data, where phonetic accuracy improves but prosodic variability collapses. To address this, they developed two self-alignment frameworks—Disentanglement-Guided Self-Alignment and Temperature-Driven Self-Critique—that outperform commercial systems like ElevenLabs and Gemini Pro. The breakthrough enables the first zero-shot voice cloning capability for Lao, a low-resource language.

arXiv:2605.27383v1 Announce Type: new Abstract: Spoken Language Models SLMs have emerged as a promising paradigm for speech synthesis by bypassing explicit grapheme-to-phoneme pipelines. However, their effectiveness in low-resource languages remains fundamentally limited by the scarcity of transcribed speech. In practice, synthetic data has become the primary strategy for scaling SLMs in such settings, providing reliable phonetic supervision when real data is insufficient. In this work, we show that this reliance introduces a fundamental trade-off, which we term the Stability-Expressivity Gap: while synthetic data improves phonetic accuracy, it progressively suppresses prosodic variability, ultimately leading to a collapse of expressivity Synthetic Erosion . To bridge this gap, we propose two self-alignment frameworks. Disentanglement-Guided Self-Alignment DGSA recovers expressivity for complex languages by exploiting prosody-timbre separation. For regimes where authentic references are exceptionally limited, Temperature-Driven Self-Critique TDSC stabilizes generation through automated exploration and filtering. Our approach outperforms strong commercial systems, including ElevenLabs and Gemini Pro, and enables the first zero-shot voice cloning capability for Lao.