AI Post-Editing in Production: A 71,262-Segment Evaluation Across Five Domains, Ten Languages and Five Systems

A study presented at the 26th Annual Conference of the European Association for Machine Translation (EAMT 2026) evaluated an AI post-editing (AIPE) system across 71,262 production segments in five domains and ten target languages, with human evaluation on 6,618 segments by 60 professional translators. The AIPE configurations outperformed Google Translate, DeepL, and direct LLM translation in quality, though direct LLM translation may suit less quality-sensitive domains. The study also found that fuzzy translation memory matches were over-represented among severe errors.

Abstract This study evaluates an AI post-editing AIPE system in a professional translation setting, covering translation from English into ten target languages across five domains. We evaluate the system using automatic metrics on 71,262 production segments and human evaluation on a stratified sample of 6,618 segments approximately 600 segments per target language assessed by 60 professional translators. AIPE refines machine translation output using a secure publicly available LLM, retrieving language-specific style guides and high-quality bilingual examples to guide edits. We compare it with direct LLM translation LLMT , Google Translate, and DeepL. The two AIPE configurations evaluated consistently outperform the generic translation baselines in terms of quality. LLMT does not match this quality, though it may suit less quality-sensitive domains. We observe how AIPE’s gains vary according to pre-translation type, with fuzzy translation memory matches over-represented among severe errors, and discuss deployment implications.- Anthology ID: - 2026.eamt-2.24 - Volume: Proceedings of the 26th Annual Conference of the European Association for Machine Translation Volume 2 /volumes/2026.eamt-2/ - Month: - June - Year: - 2026 - Address: - Tilburg, The Netherlands - Editors: Dimitar Shterionov /people/dimitar-shterionov/unverified/ , Eva Vanmassenhove /people/eva-vanmassenhove/ , Mirella De Sisto /people/mirella-de-sisto/unverified/ , Fred Blain /people/frederic-blain/ , Javad Pourmostafa Roshan Sharami /people/javad-pourmostafa-roshan-sharami/ , Lisa Lepp /people/lisa-lepp/unverified/ , Chiara Manna /people/chiara-manna/unverified/ , Argentina Anna Rescigno /people/argentina-anna-rescigno/ , Alina Karakanta /people/alina-karakanta/unverified/ , Ayla Rigouts Terryn /people/ayla-rigouts-terryn/ , Manuel Lardelli /people/manuel-lardelli/ , Natalia Resende /people/natalia-resende/unverified/ , Elena Murgolo /people/elena-murgolo/unverified/ , Janiça Hackenbuchner /people/janica-hackenbuchner/ , Anna Zaretskaya /people/anna-zaretskaya/unverified/ , Miquel Esplà-Gomis /people/miquel-espla-gomis/unverified/ , Thierry Etchegoyhen /people/thierry-etchegoyhen/unverified/ , Dagmar Gromann /people/dagmar-gromann/ , Rachel Bawden /people/rachel-bawden/ , Barry Haddow /people/barry-haddow/unverified/ , Sara Szoc /people/sara-szoc/unverified/ , Mikel Forcada /people/mikel-l-forcada/ , Helena Moniz /people/helena-moniz/ - Venue: EAMT /venues/eamt/ - SIG: - Publisher: - European Association for Machine Translation - Note: - Pages: - 49–55 - Language: - URL: https://aclanthology.org/2026.eamt-2.24/ https://aclanthology.org/2026.eamt-2.24/ - DOI: - Cite ACL : - Mara Nunziatini and Mercedes Speroni. 2026. AI Post-Editing in Production: A 71,262-Segment Evaluation Across Five Domains, Ten Languages and Five Systems https://aclanthology.org/2026.eamt-2.24/ . In Proceedings of the 26th Annual Conference of the European Association for Machine Translation Volume 2 , pages 49–55, Tilburg, The Netherlands. European Association for Machine Translation. - Cite Informal : AI Post-Editing in Production: A 71,262-Segment Evaluation Across Five Domains, Ten Languages and Five Systems https://aclanthology.org/2026.eamt-2.24/ Nunziatini & Speroni, EAMT 2026 - PDF: https://aclanthology.org/2026.eamt-2.24.pdf https://aclanthology.org/2026.eamt-2.24.pdf