{"slug": "making-local-llm-tts-work-for-a-persian-voice-assistant-on-a-gtx-1650-ti-4gb-for", "title": "Making local LLM + TTS work for a Persian voice assistant on a GTX 1650 Ti (4GB VRAM) — looking for advice", "summary": "A developer building a real-time Persian voice conversation pipeline for a Reachy Mini robot is seeking advice on running both the LLM and text-to-speech locally on a GTX 1650 Ti with 4GB of VRAM, after cloud inference via the Qwen2.5-72B HF Inference API and the fa-IR-FaridNeural voice produced periodic ConnectTimeout errors on a roughly 0.7 Mbps connection. The developer reported that qwen2.5:3b run locally through Ollama on a CPU-only torch build produced poor Persian output with frequent code-switching into English, while the cloud 72B model fixed coherence and language consistency. The request asks for model, quantization, and alternative local Persian TTS recommendations.", "body_md": "Hi everyone,\n\nI’m building a real-time, two-way Persian voice conversation pipeline for a small robot project (Reachy Mini) with my professor. Current architecture:\n\n`fa-IR-FaridNeural` voice) — cloud-only\n**Why LLM/TTS ended up cloud-based:** I initially tried `qwen2.5:3b` locally via Ollama (CPU-only, since my venv’s torch build is CPU-only), but the output quality in Persian was poor — frequent code-switching into English/other languages, incoherent responses. Switching to Qwen2.5-72B via HF Inference API fixed this completely; responses are now coherent and consistently Persian. edge-tts also gives good voice quality.\n\n**The problem:** My internet connection is very slow and unstable (~0.7 Mbps down), so I’m getting periodic `ConnectTimeout` errors on the API calls (I’ve added retry logic, which helps but doesn’t eliminate the issue). For a robot that needs to work reliably, I’d like to move both LLM and TTS to fully local inference if possible.\n\n**My hardware:**\n\n**What I’m trying to figure out:**\n\nAny pointers to models, quantization strategies, or alternative local Persian TTS projects would be hugely appreciated. Happy to share more details (code, benchmarks) if useful.\n\nThanks in advance!", "url": "https://wpnews.pro/news/making-local-llm-tts-work-for-a-persian-voice-assistant-on-a-gtx-1650-ti-4gb-for", "canonical_source": "https://discuss.huggingface.co/t/making-local-llm-tts-work-for-a-persian-voice-assistant-on-a-gtx-1650-ti-4gb-vram-looking-for-advice/180568#post_1", "published_at": "2026-09-17 20:19:40+00:00", "updated_at": "2026-09-17 20:25:00.072526+00:00", "lang": "en", "topics": ["large-language-models", "ai-tools", "natural-language-processing", "ai-infrastructure"], "entities": ["Reachy Mini", "Qwen2.5-72B", "qwen2.5:3b", "Ollama", "HF Inference API", "fa-IR-FaridNeural", "edge-tts", "GTX 1650 Ti"], "alternates": {"html": "https://wpnews.pro/news/making-local-llm-tts-work-for-a-persian-voice-assistant-on-a-gtx-1650-ti-4gb-for", "markdown": "https://wpnews.pro/news/making-local-llm-tts-work-for-a-persian-voice-assistant-on-a-gtx-1650-ti-4gb-for.md", "text": "https://wpnews.pro/news/making-local-llm-tts-work-for-a-persian-voice-assistant-on-a-gtx-1650-ti-4gb-for.txt", "jsonld": "https://wpnews.pro/news/making-local-llm-tts-work-for-a-persian-voice-assistant-on-a-gtx-1650-ti-4gb-for.jsonld"}}