{"slug": "infomaniak-fixes-translategemma-4b-to-run-on-standard-production-servers", "title": "Infomaniak Fixes TranslateGemma 4B to Run on Standard Production Servers", "summary": "Infomaniak's AI team published a vLLM-compatible checkpoint of Google's TranslateGemma 4B that moves language metadata into ordinary message text, letting the translation model run on vLLM and SGLang production inference servers without wrapping the Transformers pipeline. The build replaces the custom source_lang_code and target_lang_code fields with a delimiter format (<<<source>>>...<<<target>>>...<<<text>>>...), flattens the upstream RoPE configuration into one linear-scaling block, and changes the end-of-sequence token from <end_of_turn> to <eos> so generation terminates correctly. Infomaniak retained the original weights, which cover 55 languages, accept text or 896x896 images for OCR-plus-translate in one call, and score MetricX 5.32 and COMET 81.6 on WMT24++; the model was fine-tuned from Gemma 3 with 4.3B SFT tokens plus 10.2M RL tokens under the Gemma license.", "body_md": "# Infomaniak Fixes TranslateGemma 4B to Run on Standard Production Servers\n\nInfomaniak repackaged Google's TranslateGemma 4B so it runs cleanly on vLLM and SGLang, adding a delimiter-based language code format for serving.\n\n- Infomaniak released a [vLLM-compatible build](https://huggingface.co/Infomaniak-AI/vllm-translategemma-4b-it?utm_source=alphasignal) of Google's TranslateGemma 4B translation model.\n- Custom language-code fields are replaced by a delimiter format: `<<<source>>>...<<<target>>>...<<<text>>>...`\n- Covers 55 languages, accepts text or 896x896 images for OCR-plus-translate in one call.\n- WMT24++ scores: MetricX 5.32, COMET 81.6 for the 4B variant.\n- RoPE config flattened and EOS token fixed so generation terminates correctly under vLLM and SGLang.\n- Fine-tuned from Gemma 3 with 4.3B SFT tokens plus 10.2M RL tokens, under the Gemma license.\n\nGoogle’s TranslateGemma has been awkward to run on production inference servers because its custom chat template expects `source_lang_code` and `target_lang_code` fields inside each message. vLLM’s OpenAI-compatible API does not accept those custom content fields. Infomaniak’s AI team has published a [compatible 4B checkpoint](https://huggingface.co/Infomaniak-AI/vllm-translategemma-4b-it?utm_source=alphasignal) that moves the language metadata into ordinary message text.\n\nThe conversion lets applications serve translation through familiar vLLM and SGLang endpoints without wrapping the Transformers pipeline. Existing OpenAI-compatible clients can submit requests after formatting the source language, target language, and text into one string.\n\n## Compatibility without new weights\n\nThe upstream [TranslateGemma 4B model](https://huggingface.co/google/translategemma-4b-it?utm_source=alphasignal) is a Gemma 3 derivative fine-tuned for translation. Infomaniak retained its weights and changed three pieces of serving metadata:\n\n- **Prompt serialization:** Source and target language codes now appear inside the message string, separated by fixed delimiters, instead of custom JSON fields.\n- **RoPE metadata:** The upstream Transformers configuration defines separate rotary position embedding settings for full and sliding attention. Rotary embeddings encode token positions for the attention layers. Infomaniak flattened those settings into one linear-scaling block that vLLM can parse.\n- **Stopping token:** The end-of-sequence token changed from`<end_of_turn>` to`<eos>` , allowing the server to stop generation on the model’s expected token.\n\n## One string carries the request\n\nEach translation request places the source code, target code, and input text in the user message’s `content` field. A Czech-to-German request uses this format:\n\n## This story is for Pro members\n\nYou've reached the end of the free preview. Upgrade to AlphaSignal Pro to read the full article - and everything else behind the paywall.", "url": "https://wpnews.pro/news/infomaniak-fixes-translategemma-4b-to-run-on-standard-production-servers", "canonical_source": "https://alphasignal.ai/news/infomaniak-fixes-translategemma-4b-to-run-on-standard-production-servers", "published_at": "2026-11-17 09:30:57+00:00", "updated_at": "2026-11-17 19:40:09.155383+00:00", "lang": "en", "topics": ["large-language-models", "ai-tools", "natural-language-processing", "ai-infrastructure", "machine-learning"], "entities": ["Infomaniak", "Google", "TranslateGemma 4B", "vLLM", "SGLang", "Gemma 3", "Hugging Face"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/infomaniak-fixes-translategemma-4b-to-run-on-standard-production-servers", "markdown": "https://wpnews.pro/news/infomaniak-fixes-translategemma-4b-to-run-on-standard-production-servers.md", "text": "https://wpnews.pro/news/infomaniak-fixes-translategemma-4b-to-run-on-standard-production-servers.txt", "jsonld": "https://wpnews.pro/news/infomaniak-fixes-translategemma-4b-to-run-on-standard-production-servers.jsonld"}}