A Computational Approach to Measuring Semantic Change in Sanskrit Literature

A new arXiv paper (arXiv:2609.25012v1) reports that diachronic word embeddings can track semantic change in Sanskrit, an ancient low-resource language, with 19 of 21 testable shifts moving in the philologically attested direction (sign test, p=0.00011). The author assembled a 2.7M-token corpus spanning four canonical periods and recovered word boundaries using a neural byte-level sandhi splitter and lemmatizer, then trained per-period embeddings across configurations. The work tests whether a paradigm largely validated on modern, high-resource, well-segmented languages transfers to a language whose sandhi, morphological inflection, compounding, and polysemy pose distinct challenges.

arXiv:2609.25012v1 Announce Type: new Abstract: Diachronic word embeddings have become the modern standard for tracking semantic change, yet they have been largely validated on modern, high-resource, and well-segmented languages. This paper tests whether the paradigm transfers to Sanskrit, an ancient, low-resource language whose phonological fusion sandhi , morphological inflection, compounding, and polysemy pose a unique challenge. I assemble a 2.7M-token corpus spanning four canonical periods, recover word boundaries with a neural byte-level sandhi splitter and lemmatizer, and train per-period embeddings across configurations. To evaluate the system, I curate a validation set from historical scholarship and test recovery directionally with anchor displacement. Of 21 testable shifts, 19 move in the philologically attested direction sign test, p=0.00011 . I further show which configuration the language forces and comment on opportunities for improvement.