Hi everyone,
I am currently working on fine-tuning models for multi-directional translation tasks—specifically focusing on Japanese–English (ja-en), English–Arabic (en-ar), and Japanese–Arabic (ja-ar). To ensure the best possible translation quality and prevent common pitfalls (like hallucination or mixed-language outputs), I’m looking to optimize my workflow. Could the community share recommendations on:
- Dataset Formatting & Templates: What are the most effective prompt structures or chat templates for translation fine-tuning (e.g., standard source-target pairs vs. instruction-based formatting)?
- Instruction Design: Are there specific system prompts or instruction guidelines that yield the best results for low-resource pairs or distant language pairs like Japanese-to-Arabic?
- Recommended Base Models: Which open-source base models (e.g., Llama, Mistral, Qwen, or specific multilingual models) have you found to be most robust for these specific language directions?
Any advice, recommended templates, or references to successful repositories would be greatly appreciated!
Thanks in advance!
For three directions across three scripts, freeze the format before anything else: one identical template for every example, the target language named in the same place every time, and loss computed only on the translation itself. If the direction signal varies, the model learns to guess the target from the source instead — which is where mixed-language outputs start, worst on ja→ar where the signal is thinnest. Check it directly: on a held-out set scored per direction, count any output in the wrong language (a simple language-ID pass) as a failure before you look at BLEU or chrF, and choose the base model by that off-target rate after the same short fine-tune on a small slice for each candidate.