Infomaniak repackaged Google's TranslateGemma 4B so it runs cleanly on vLLM and SGLang, adding a delimiter-based language code format for serving.
- Infomaniak released a [vLLM-compatible build](https://huggingface.co/Infomaniak-AI/vllm-translategemma-4b-it?utm_source=alphasignal) of Google's TranslateGemma 4B translation model.
- Custom language-code fields are replaced by a delimiter format: `<<<source>>>...<<<target>>>...<<<text>>>...`
- Covers 55 languages, accepts text or 896x896 images for OCR-plus-translate in one call.
- WMT24++ scores: MetricX 5.32, COMET 81.6 for the 4B variant.
- RoPE config flattened and EOS token fixed so generation terminates correctly under vLLM and SGLang.
- Fine-tuned from Gemma 3 with 4.3B SFT tokens plus 10.2M RL tokens, under the Gemma license.
Google’s TranslateGemma has been awkward to run on production inference servers because its custom chat template expects source_lang_code and target_lang_code fields inside each message. vLLM’s OpenAI-compatible API does not accept those custom content fields. Infomaniak’s AI team has published a compatible 4B checkpoint that moves the language metadata into ordinary message text.
The conversion lets applications serve translation through familiar vLLM and SGLang endpoints without wrapping the Transformers pipeline. Existing OpenAI-compatible clients can submit requests after formatting the source language, target language, and text into one string.
Compatibility without new weights #
The upstream TranslateGemma 4B model is a Gemma 3 derivative fine-tuned for translation. Infomaniak retained its weights and changed three pieces of serving metadata:
- Prompt serialization: Source and target language codes now appear inside the message string, separated by fixed delimiters, instead of custom JSON fields.
- RoPE metadata: The upstream Transformers configuration defines separate rotary position embedding settings for full and sliding attention. Rotary embeddings encode token positions for the attention layers. Infomaniak flattened those settings into one linear-scaling block that vLLM can parse.
- Stopping token: The end-of-sequence token changed from
<end_of_turn>to<eos>, allowing the server to stop generation on the model’s expected token.
One string carries the request #
Each translation request places the source code, target code, and input text in the user message’s content field. A Czech-to-German request uses this format:
This story is for Pro members #
You've reached the end of the free preview. Upgrade to AlphaSignal Pro to read the full article - and everything else behind the paywall.