AI Translation Pipeline: Validators Around a gpt-4o Call A developer built an AI translation pipeline for the pi-pi.ee e-commerce shop that generates 27 locales from a Russian source file using gpt-4o, wrapping the model call in validators rather than relying on prompt engineering. The pipeline diffs missing keys so existing translations are never retranslated, and runs ICU, argument, tag and plural checks at both write time and commit time via a shared i18n-check.ts implementation, exiting with code 1 and a list of omitted strings when validation fails. An AI translation pipeline needs checks that can say ‘no’ more than it needs a better prompt. In the script that fills the 27 target languages of pi-pi.ee https://iurii.rogulia.fi/projects/pi-pi-b2b-ecommerce , the model call is one request per batch, and two sets of gates surround it. At write time: a diff of missing keys, a check on whether the reply was cut off, an ICU and plural validator that must pass before a string is written, and exit code 1 with a list of what was left out. At commit time: i18n:check and glossary:check , which run as part of npm run lint . Below is that code as it stands, the real defects behind some of the gates, and a section on what the gates still do not catch. Routing, hreflang and formatting are in Next.js i18n with 30 Languages https://iurii.rogulia.fi/blog/nextjs-i18n-30-languages ; I will not repeat them. The shop has a Russian source file and 27 generated locales. The first version of any such script sends strings to a model and saves the reply. It works on the first run and fails later, quietly: A prompt is a request, and these are hard to ask a model to detect in its own output. Code that runs after the call and is allowed to refuse is a better place for them. translate.py never translates a whole file. It computes the keys that exist in ru.json and not in the target, and sends only those: Simplified from get missing keys in scripts/translate.py for key in source: if key not in target: missing.append current path, source key elif isinstance source key , dict and isinstance target key , dict : missing.extend get missing keys source key , target key , current path An existing translated string is never retranslated. That is the property that lets a person edit a locale file by hand and have it survive the next run. Keys that exist in a locale file but not in the source are removed on every run, except the dotted paths listed in scripts/locale-overrides.json ; hand-written, locale-specific content has to be registered there or the next run deletes it. The same diff powers --dry-run : it prints missing, obsolete and invalid keys per locale, with no API call, no writes and no API key. It also validates the strings already in the files, so it checks the current state as well as previewing the next run. The cost of diffing is a rule people have to follow. translate.py only fills missing keys, so an edit to an existing Russian string leaves the other locales stale until the documented procedure is followed: delete that key from every other locale and rerun. That is a procedure, not an enforced check, and it comes back in the last section. Every string the model returns goes through a validator before it is placed in a file. The validator is scripts/i18n-check.ts , the same code that npm run i18n:check runs over the whole catalogue at commit time, called by the Python script in --validate mode: Simplified from translate batch in scripts/translate.py invalid = {f "key" : f "error" for f in validate translations candidates } for candidate in candidates: key = candidate "key" if key in invalid: failures.append f"{key}: {invalid key }" else: all translated key = candidate "target" Using one implementation for both means the write-time check and the commit-time check cannot drift apart: there is one ICU parser and one source of plural data. For each source and target pair the validator runs these checks: js // Simplified from validate in scripts/i18n-check.ts const want = shapeOf parse source ; const got = shapeOf parse target ; // a parse error is returned as a failure if sorted want.args == sorted got.args errors.push arguments differ from source ; if sorted want.tags == sorted got.tags errors.push tags differ from source ; return ...errors, ...pluralErrors locale, got ; The target must parse as ICU. It must use the same {arguments} and the same