AI coding: loop engineering a translator

A developer describes building a complex 'loop engineering' pipeline in December 2024 to translate large Korean documents to English using local LLMs, but ultimately abandoned the project after weeks because the critic loop failed to improve translation quality and the system could not satisfy its own validation criteria.

AI coding: loop engineering a translator /loop-engineering I accidentally did that years ago Posted by Since a week or two the term 'loop engineering' is coming up after 'prompt engineering' and 'agentic workflow' seem to not cut it anymore and we need more abstractions. The first time I tried to write something which would do something useful with a LLM was about two years ago in December 2024, I wanted to do translation of big documents from Korean to English on my local RTX 3060 with 12 GB VRAM. I couldn't just use the ChatGPT chat for that because back then it's context window was way too small for the massive documents I needed translated and it would stop translating after 10%. So I thought I can automate the splitting up of the document. But because I did not know anything about agents I had to come up with some architecture. So I made this very complicated agent pipeline: Agent pipeline / loops The pipeline turns a raw Korean transcript into an English translation through a plan → execute → critique → repair loop. A literal NLLB translation acts as an impartial reference, and a translation memory keeps terminology consistent across chunks. Components | Component | Model / Tool | What it does | |---|---|---| Sentence Segmentation | kss Korean String processing Suite | Splits the raw Korean transcript into individual sentences kss.split sentences before anything else runs. | Chunker | TextChunker python | Groups the segmented sentences into chunks of N sentences with optional overlap so each piece fits the models' context. | Planner | qwen3:14b | Reads an excerpt once and produces a global strategy as JSON content type, speaking style, terminology, per-component instructions . Shared by the Executor and Critic. | Executor | aya:8b | Translates each chunk. On a revise verdict it runs repair using the Critic's fix instruction. Strips the model's <think reasoning blocks. | Critic | qwen3:14b | Validates each translation by a 3-way comparison source vs. Executor output vs. NLLB and returns a status , issues, a fix instruction, and extracted terms/styles. | NLLB | facebook/nllb-200-distilled-600M | Produces a literal reference translation of the source chunk. Only the Critic sees it — it is a witness, not the author — which guards against the Executor confidently hallucinating. | Memory | TranslationMemory python dict | Accumulates terms/styles. The Executor reads it for context; the Critic's extracted terms/styles are written back on every passing chunk. | Formatter optional | qwen3:14b | Post-processes the accepted translation into readable paragraphs. | Output Writer | OutputWriter python file write | Appends each finished chunk to the output file incrementally. | Verdict I worked on this for some weeks because I had a real need to translate hours of educational YouTube video transcripts. It started as a simple loop but the quality of the local models was not good so I tried to improve the quality of the translation by introducing a critic and feed back the information from it as a prompt, then I realized things are drifting and the same words are being translated differently over the course of the document so I introduced a memory so it would not drift. I did many more small optimizations like a overlapping window of the chunks, etc. But in the end all the information from the critic and the memory did not lead to a better translation, what happened was that the critic kept flagging that the translation is not good enough and looping back and the translator was not able to translate good enough for the critic to be satisfied. I was not able to fine tune it so that it would improve the translation in any significant manner and after a couple of weeks I kind of gave up and am waiting for better local translation models to arrive, but then this whole loop thing will probably not be necessary anymore.