LuxSQA: Ask Me in Luxembourgish with TTS-Augmented Spoken Question Answering

Researchers developed a spoken question answering system for Luxembourgish using text-to-speech augmentation, training a SLAM-style architecture with frozen Whisper and multilingual LLM backends. Multi-source synthetic training data from four TTS systems achieved the best performance on real Luxembourgish speaker conditions, showing that TTS quality scores do not directly predict downstream QA success.

arXiv:2607.02763v1 Announce Type: new Abstract: Spoken Question Answering SQA remains largely focused on high-resource languages and carefully recorded speech, limiting the reach of speech-LLM methods in low-resource settings. This paper investigates whether text-to-speech TTS can provide task-specific training data for Luxembourgish SQA without requiring a large human-recorded QA corpus. Starting from existing text-based QA resources, we translate questions into Luxembourgish, synthesize spoken questions with multiple TTS systems, and pair them with textual answers. We train a parameter-efficient SLAM-style architecture that connects a frozen Whisper encoder to frozen multilingual LLM backends through a learned projector and LoRA adapters. We compare MMS-TTS, Qwen3-TTS, and OmniVoice variants, including single-source corpora of about 48k questions and a 4TTS multi-source mix of approximately 230k questions. Evaluation on LLAMA-LB-Test with two real Luxembourgish speaker conditions shows that multi-source and voice-design-based synthetic training configurations yield the strongest SQA performance. The results also show that no-reference TTS quality scores do not monotonically predict downstream QA performance, indicating that synthetic speech must be evaluated as task-specific training data rather than only as natural-sounding audio.