{"slug": "need-advice-on-choosing-model-to-use-as-movies-and-tv-shows-translator-after-on", "title": "Need advice on choosing model to use as movies and tv shows translator after fine-tuning on my dataset", "summary": "A community analysis of local model memory costs found that at 128K context, Mistral Small 3.2 needs about 20 GB for the context cache on top of a 13.3 GB 4-bit GGUF file, while Gemma 4 12B needs only about 2.3 GB of context on a 6.6 GB file, and Qwen3.8 27B needs about 8 GB on 15.3 GB. The analysis, aimed at a user planning to fine-tune a model for Arabic and Japanese movie and TV subtitle translation, estimated a 5,000-line subtitle file at roughly 60-75K input tokens and the same in output, requiring well over 100K tokens of context. The poster also noted Qwen3 14B caps at about 40K tokens, ruling it out for long files, and suggested splitting long files by scene in 500-800 line chunks with overlap plus a running summary for smaller GPUs.", "body_md": "Can’t tell you which one translates Arabic or Japanese best, I haven’t tested that. But the “whole file at once” part is mostly a memory question, and there I have numbers.\n\nRough guess: a subtitle line is 10 to 15 tokens, so 5000 lines is somewhere around 60-75K tokens going in, and about the same coming out. So for your biggest files you need a model that can hold well over 100K tokens, and the smaller ones (200-1000 lines) fit in 32K easily.\n\nThe thing people miss is that the context itself costs memory, and how much depends heavily on the model. I keep a table of this (4-bit GGUF file + context cache), a few examples at 128K context:\n\nGemma 4 12B: 6.6 GB file, about 2.3 GB for the context\n\nGemma 4 26B-A4B: 15.8 GB file, about 2.7 GB for the context\n\nQwen3.8 27B: 15.3 GB file, about 8 GB for the context\n\nMistral Small 3.2: 13.3 GB file, about 20 GB for the context\n\nAnd Qwen3 14B stops at about 40K tokens, so it’s out for the long files no matter what card you have.\n\nSo before you spend time on fine-tuning I’d check two things: the max context of the model, and what that context costs on your card. What GPU do you have? With that I can tell you which of these actually hold a 5000-line file.\n\nOne more thing, for the long files you could also split by scene (every 500-800 lines with some overlap) and pass a short summary of what happened so far. Not as clean as one pass, but it works on much smaller cards.", "url": "https://wpnews.pro/news/need-advice-on-choosing-model-to-use-as-movies-and-tv-shows-translator-after-on", "canonical_source": "https://discuss.huggingface.co/t/need-advice-on-choosing-model-to-use-as-movies-and-tv-shows-translator-after-fine-tuning-on-my-dataset/191989#post_2", "published_at": "2026-11-18 12:38:26+00:00", "updated_at": "2026-11-18 12:43:01.648582+00:00", "lang": "en", "topics": ["large-language-models", "ai-infrastructure", "machine-learning"], "entities": ["Gemma 4 12B", "Gemma 4 26B-A4B", "Qwen3.8 27B", "Mistral Small 3.2", "Qwen3 14B"], "also_reported_by": [], "takeaway": "At 128K context, context cache memory ranges from about 2.3 GB on Gemma 4 12B to about 20 GB on Mistral Small 3.2, so check a model's max context and its context memory cost on your GPU before fine-tuning.", "recommendation": {"advice": "Budget context-cache VRAM alongside model weights before choosing a model: at 128K, Mistral Small 3.2 adds about 20 GB versus Gemma 4 12B's 2.3 GB, and split long inputs into 500-800 line chunks with overlap if the card is small.", "playbook": "inference-serving", "playbook_url": "https://wpnews.pro/playbooks/inference-serving"}, "alternates": {"html": "https://wpnews.pro/news/need-advice-on-choosing-model-to-use-as-movies-and-tv-shows-translator-after-on", "markdown": "https://wpnews.pro/news/need-advice-on-choosing-model-to-use-as-movies-and-tv-shows-translator-after-on.md", "text": "https://wpnews.pro/news/need-advice-on-choosing-model-to-use-as-movies-and-tv-shows-translator-after-on.txt", "jsonld": "https://wpnews.pro/news/need-advice-on-choosing-model-to-use-as-movies-and-tv-shows-translator-after-on.jsonld"}}