The Role of Prompt Language and Translation-Theory-Driven Prompts in Large Language Models: A Case Study on Spanish-Chinese Journalistic Translation

A study on GPT-5.2's Spanish-Chinese journalistic translations found that translation theory-driven prompts improved quality under expert evaluation, with the brief-oriented prompt scoring highest (MQM: 8.66 vs. 7.84), while automated metrics favored the baseline prompt. Prompt language had negligible impact, and unidiomatic style errors persisted across conditions.

arXiv:2607.03160v1 Announce Type: new Abstract: This study examines how prompt language and translation theory-driven prompt design influence the quality of Spanish-Chinese journalistic translations generated by GPT-5.2. A parallel corpus of four editorials from El Pais was translated under 48 experimental conditions 4 prompt types, 3 prompt languages, and 4 articles . Translation quality was assessed using BLEU and BERTScore-F1 for automated evaluation, alongside human evaluation based on the Multidimensional Quality Metrics MQM framework. Automated metrics identified the baseline prompt BASE as the best-performing condition, whereas human evaluation ranked the brief-oriented prompt BRIEF highest MQM: 8.66 vs. 7.84 , a reversal likely attributable to the single-reference constraint inherent in automated measures. Sub-error type analysis revealed that translation theory-driven prompts selectively reduced Awkward style errors, while Unidiomatic style errors persisted across conditions. Prompt language had a negligible impact under both evaluation paradigms. These results indicate that translation theory-driven prompts can yield measurable quality gains under expert evaluation of journalistic translations, although their pedagogical implications for language learners remain suggestive and require validation through user-based studies.