I wanted to answer a practical question: if I have a Windows laptop with an NVIDIA GTX 1650 Ti and 4 GB of VRAM, which small local language model should I actually trust to translate an English novel into Serbian?
The obvious candidate was TranslateGemma, a model designed for translation. But after its first run, I tried a general-purpose Gemma 4 model, followed by Qwen3 and Qwen3.5. I kept the source passages and translation instructions fixed, recorded each response and its timings, and inspected the Serbian that came back.
The surprise was not only that Gemma 4 E2B finished fastest. It also completed every passage without a retry or script conversion. Yet when I read the translations, even the apparent winner produced invented words, awkward grammar, and meanings that drifted away from the English. A model can pass every automated check and still be a poor literary translator.
The experiment: one old novel, four local models
The source was The Adventures of Ferdinand Count Fathom, an English-language literary work. The prepared TXT file contained 3,694 non-empty source lines representing 158,396 words. For this exploratory comparison, I sent exactly the first 20 non-empty lines to each model. Those lines contained 943 English words in total.
“Line” matters here: these were not guaranteed to be individual grammatical sentences. Some contained several sentences, and some suffered from imperfect text preparation. For example, the source had joined boundaries such as “impracticable.Indeed” and “ostentation.I have”. These defects can make translation harder and limit what the results tell us.
All four tests used Q4_K_M GGUF quantization on the same laptop and local llama.cpp server, with a configured context of 4,096 tokens. The request used temperature 0, seed 0, a 512-token output cap, sequential calls, and cache_prompt=false. Each model used its own available server-side chat formatting, so this was a comparison of practical local configurations — not a controlled isolation of neural-network weights.
Models tested
TranslateGemma 4B - translategemma-4b-it-Q4_K_M.ggufGemma 4 E2B - google_gemma-4-E2B-it-Q4_K_M.ggufQwen3 4B - Qwen3–4B-Q4_K_M.ggufQwen3.5 4B - Qwen3.5–4B-Q4_K_M.gguf
The English instruction asked for fluent Serbian in Latin script, preservation of literary style and meaning, and no commentary around the translation. I did not change its text between these four experiments: the recorded prompt SHA-256 matched across runs.
First surprise: Gemma 4 E2B won the speed test
For generation throughput, Gemma 4 E2B reached 59.65 output tokens per second. TranslateGemma reached 41.76, Qwen3 reached 40.06, and Qwen3.5 reached 34.62. In this setup, Gemma 4 generated approximately 43% more tokens per second than TranslateGemma and 72% more than Qwen3.5.
Prompt processing showed the same general pattern: Gemma 4 E2B processed approximately 325.39 tokens/s; TranslateGemma, 209.74; Qwen3, 180.91; and Qwen3.5, 171.15.
The elapsed runs tell an even clearer operational story. Gemma 4 took 38.95 seconds for the 20-passages test; TranslateGemma, 55.48 seconds; Qwen3.5, 117.16 seconds; and Qwen3, 147.59 seconds. These are wall-clock figures including failed requests and retry overhead — not just clean inference benchmarks.
Mean latency for accepted translations was 1.81 seconds on Gemma 4, 2.63 seconds on TranslateGemma, 3.47 seconds on Qwen3.5, and 3.83 seconds on Qwen3. The respective 95th-percentile accepted-response latencies were 2.81, 4.06, 6.47, and 7.51 seconds. Those percentiles come from very small samples and should not be overinterpreted.
There is a practical caveat: a 20-line batch cannot establish a universal performance ranking across all workloads. Also, the Qwen models produced unsuccessful responses and extra requests. Comparing elapsed total time is therefore a measure of end-to-end usefulness under this workflow, rather than architecture-only speed.
Second surprise: a 100% pass rate does not mean good translation
TranslateGemma accepted 20 of 20 lines. Gemma 4 also accepted 20 of 20. Qwen3.5 accepted 19 of 20, and Qwen3 accepted only 16 of 20. These were technical acceptance rates: the checks caught empty output, invalid scripts, truncation, some formatting leaks, and line-alignment problems. They did not judge whether the Serbian was elegant, grammatical, or faithful.
That distinction became obvious as soon as I compared selected source passages with model outputs.
The first passage: a clear win, followed by a warning
The novel begins with a long sentence discussing how hard it is to write a dedication flattering a patron without inviting public ridicule. Near the end, the English says the task was “altogether impracticable”.
TranslateGemma: "…bili saglasni da je ova zadata posta."Gemma 4: "…složili da je zadatak potpuno nemoguć."
On that clause, Gemma 4 preserves the meaning; TranslateGemma does not. But the same Gemma 4 passage also contains “samopoželju patrona”, an unnatural and seemingly invented expression. A better local result is not automatically a publishable translation.
A false friend in a literary context
In another line, the source says: “In dedication, as in poetry, there is no medium…” Here “dedication” refers to the literary dedication; “no medium” means no middle ground. TranslateGemma returned “U čast, kao i u poeziji, nema sredstva…”. Gemma 4 returned “U posvećenju, kao i u poeziji, nema sredstva za to…”.
Gemma 4 recognized the dedication context more closely, but both misread “no medium”. This is exactly the kind of error a generic script validator cannot catch: the output contains Serbian-looking words and still gets the passage wrong.
When grammar changes the relationship between characters
In a passage about the narrator’s affection and sincerity, TranslateGemma turned the intended relationship around with wording resembling “you are credible and sincere to me.” Gemma 4 came closer to the intended sense: the addressee is convinced of the narrator’s affection and sincerity. Yet its translation still used stiff constructions such as “ovaj adres”.
Elsewhere, Gemma 4 produced “Osmešila sam se … i tremeo sam …”, switching grammatical gender and using a malformed verb in one sentence. On the phrase “disgrace and discomfiture of vice”, neither Gemma model expressed the moral meaning particularly well. The failures were not limited to spelling; they involved interpretation, register, and coherence.
Serbian is not just Latin vs. Cyrillic
Serbian can be written in Latin or Cyrillic script, and both Ekavian and Ijekavian are recognized pronunciation standards. My practical target, however, was consistent Serbian Latin in the Ekavian variety. That is a tighter constraint than simply producing something recognizable as Serbian.
TranslateGemma’s accepted outputs included Ijekavian or regionally different forms such as “svijet”, “mjeri”, and “posljedice”. Gemma 4 generally leaned Ekavian in these examples but still made numerous grammatical mistakes; Qwen3.5 mixed forms such as “cijenimo” and “nedosljednost” with Ekavian forms elsewhere.
To be precise, Ijekavian itself is not “incorrect Serbian”. The problem for this use case was inconsistency, sometimes combined with vocabulary drifting toward other closely related standards. A good editorial workflow would choose the desired variety up front, test for consistency, and still use human review. Mechanical spelling normalization cannot repair misunderstood sentences.
The Cyrillic problem was real — and mostly fixable
The first TranslateGemma prototype rejected every response containing Cyrillic. With temperature 0 and seed 0, retrying the same request three times tended to produce the same Cyrillic again. That wasted inference time without improving the answer.
I changed the pipeline to retain each raw response and, when safe, transliterate Serbian Cyrillic into Latin as a separate postprocessing step. The transformation handles Serbian-specific letters and digraphs, including љ→lj, њ→nj, џ→dž, ђ→đ, ћ→ć, ч→č, ш→š, and ж→ž. The original model output stays in the attempt log, so native script behavior remains measurable.
In the 20-line TranslateGemma test, five outputs needed transliteration (25%). All 20 were accepted without extra inference. Gemma 4 required none. Qwen3.5 also required none. Qwen3 accepted seven transliterated passages but generated some Cyrillic containing unsupported non-Serbian characters; three lines failed the script check despite repeated attempts.
This improvement is useful, but not a linguistic quality fix. Transliteration changes the writing system, not the meaning. An incoherent sentence remains incoherent when rendered in Latin letters.
The Qwen3.5 “nothing” loop: segment 4
The most memorable failure was passage four in the Qwen3.5 run. The source discusses an author who expresses genuine gratitude, only for the public to dismiss that praise as self-interest or flattery. Instead of finishing the translation, the model began repeating the Serbian word “ništa” (“nothing”) over and over.
A short excerpt from the failed continuation: "…ništa ništa ništa ništa ništa ništa…"
The three Qwen3.5 attempts each hit exactly 512 output tokens and ended with finish_reason=length. They repeated essentially the same degeneration because the prompt, seed, and decoding settings were unchanged. This was not evidence that the passage needed 512 useful tokens of translation; it was a generation loop. Increasing the cap to 1,024 would not address the underlying repetition by itself.
The earlier Qwen3 run showed a similar pattern on passage five: three truncated responses at the token limit, alongside three separate lines failing script validation. Qwen3 thus ended with 16 accepted passages after 28 HTTP calls. Qwen3.5 finished with 19 accepted passages after 22 calls.
An important engineering lesson followed: a retry policy should understand the error. Transport failures may deserve retries. Deterministic script violations and repetition loops need different treatment — perhaps model-specific decoding changes, bounded repetition detection, or explicit human review — not two automatic copies of the same failed request.
What the numbers really say
Under these exact local settings, Gemma 4 E2B was the strongest overall practical performer. It had the highest observed generation and prompt-processing throughput, the shortest end-to-end run, all 20 responses accepted, and no transliteration or retry overhead. On several passages, it also preserved the English meaning better than TranslateGemma.
But it did not win literary translation in the stronger editorial sense. It still created strange words, mishandled idioms, lost grammatical agreement, and rendered nuanced prose awkwardly. TranslateGemma — the model with translation in its name — did not escape these problems either. Qwen3.5 delivered 19 technically acceptable lines, but quality defects and a repetition loop weakened its showing. Qwen3 was the least reliable configuration in this initial batch.
None of this proves that one entire model family is “better” at Serbian, or that a different prompt, tokenizer configuration, decoding setup, or cleaned dataset would not change the ranking. The source sample was small, contiguous, and literary; it had text-preparation artifacts; and the models used different native chat templates. There was no independently curated Serbian reference translation, no BLEU/chrF++/COMET evaluation, and no blind multi-rater human scoring. These results are an honest pilot, not a publication-grade multilingual benchmark.
What I would change in a production translation pipeline
Clean the source text first, restoring lost spaces and separating real sentences before testing sentence-level batching.
Keep raw model responses, normalized outputs, prompt hashes, token timing, and exact model/server metadata for every request.
Treat script compliance, language-variety consistency, and semantic fidelity as three different checks.
Use targeted handling for repeated tokens and output truncation; stop repeating deterministic failures automatically.
Ask bilingual reviewers to score meaning, grammar, terminology, and style on a larger, stratified sample before translating an entire book.
The infrastructure already stores aligned source/target lines, checkpoint state, per-attempt CSV data, and machine-readable summaries. That made the failures as informative as the successes. Without those records, I would have had four vague impressions about models. With them, I could show exactly where and how each system broke.
My takeaway
The fastest model was also the most usable one in this small test: Gemma 4 E2B. But that is only half the story. Translation success is not the same as generating Latin-script Serbian text at 60 tokens per second. The real test is whether a native reader can trust the meaning, register, grammar, and voice.
If I were building a local, private, low-cost translation assistant on this laptop today, Gemma 4 would be my first candidate for a human-in-the-loop workflow — not for unattended publication of a classic novel. The biggest improvement might not come from another few tokens per second. It might come from better source preparation, better error handling, and a serious evaluation of the words the models actually write.
Experiment notes and sources
Hardware: NVIDIA GTX 1650 Ti Laptop GPU (4 GB VRAM), Windows, local llama.cpp server. Model files were quantized GGUF Q4_K_M. Sample: first 20 non-empty lines of The Adventures of Ferdinand Count Fathom (943 source words). Recorded llama.cpp build: b8729–243532e55. Settings: context 4096, max_tokens 512, temperature 0, seed 0, sequential requests, cache_prompt=false. Chat templates differed by model; all four runs share the same prompt SHA-256.
Metrics are taken from four local checkpointed experiment reports and their per-attempt records, dated October 20, 2026. Aggregate output-token rates are computed from paired server token counts and generation times for successful first attempts. Wall-clock duration includes failed/retried work. No quality metric was calculated from a reference corpus.
If you’re interested in exploring local language models, machine translation, and the challenges of evaluating multilingual AI systems, these resources provide useful technical background.
These resources provide the broader context for a central finding of this experiment: fast inference, technically valid output, and high-quality literary translation are three different things.
I Tested Four Local AI Models on a Literary Translation. Gemma 4 Won — With a Catch. was originally published in Towards AI on Medium, where people are continuing the conversation by highlighting and responding to this story.