Cross Lingual Transfer in Tulu Legal Comprehension: Script-Dependent Improvement and RAG-Induced Knowledge Conflict

A new arXiv study (2608.28645v1) found that three large language models—Llama3, Hex-1, and Sarvam—showed script-dependent improvement in classifying legal complaints in the low-resource Dravidian language Tulu when queries were transliterated across Dravidian scripts, with Kannada producing the strongest positive trend. Retrieval-augmented generation (RAG) using Kannada legal papers yielded mixed results, with failures attributed to fact substitution and confabulation, indicating that reasoning failures stem from the models' parsing and reasoning rather than corpus contents.

arXiv:2608.28645v1 Announce Type: new Abstract: Low-resource languages without an adequate training corpus often use a related, higher-resource language as a scaffold for comprehension. Still, there is a need to develop rigorous evaluation methods to identify when models fail in cross lingual low-resource environments. Using the legal domain as a backdrop, three models Llama3, Hex-1, Sarvam were tested on the ability to classify legal complaints written in a low resource Dravidian language Tulu . Transliterating queries across Dravidian scripts allowed models to gain a preliminary understanding of speakers' complaints without the use of wide scale training, though the level of comprehension was heavily script dependent with Kannada - another relatively low-resource language - producing the strongest positive trend . Retrieving from a corpus of Kannada legal papers across a RAG framework caused mixed results. Some models had a weak positive trend in comprehension under certain conditions, but when models failed, it was often across two axes: fact substitution fixating on specific passage excerpts that skewed reasoning and confabulation hallucination that had no basis in either query or corpus . Within low resource domains, results identify the model's parsing of information and subsequent reasoning as the source of reasoning failure, rather than corpus contents. Script-dependent comprehension and RAG robustness also seem to travel together. This is further supported by the reasoning-trace analysis and a statistical-honesty framework deployed - techniques that are more broadly applicable to low-resource multilingual RAG evaluation.