Evaluating Tibetan Speech Recognition Across Dialects and Adaptation Regimes: A Multi-Granularity Analysis

A study posted to arXiv (2610.27002v1) found that lower character error rates in Tibetan speech recognition do not always correspond to lower syllable or segmented-word error rates, with the ranking differences persisting across corpora. The authors combined character, syllable and segmented word error rates in one framework, compared dictionary-based, neural and language-model-based segmentation against expert annotations, and evaluated five ASR architectures on Lhasa Tibetan before testing zero-shot transfer and target-corpus adaptation on five additional corpus subsets. Higher LoRA ranks improved syllable and segmented-word scores but remained worse than full fine-tuning in the tested range, and metric correlations weakened in some unsuccessful adaptation settings.

arXiv:2610.27002v1 Announce Type: new Abstract: Tibetan speech recognition is usually evaluated with character or syllable error rates because the writing system does not mark word boundaries. These metrics can rank systems differently and respond differently to adaptation failures. We combine character error rate, syllable error rate, and segmented word error rate in one evaluation framework. We use dictionary-based, neural, and language-model-based segmentation and compare their boundaries with expert annotations. We evaluate five ASR architectures on Lhasa Tibetan, then test zero-shot transfer and target-corpus adaptation on five additional corpus subsets. We also vary LoRA rank and training-data size to examine adaptation failure. The results show that a lower character error does not always lead to a lower syllable or segmented-word error. These ranking differences persist across corpora. Metric correlations also weaken in some unsuccessful adaptation settings. Higher LoRA ranks improve syllable and segmented-word scores but remain worse than full fine-tuning in the tested range. A standardized phonological representation provides another view of spelling errors. These findings support evaluation at several linguistic levels and motivate better segmentation and adaptation methods for low-resource ASR.