Making local LLM + TTS work for a Persian voice assistant on a GTX 1650 Ti (4GB VRAM) — looking for advice

A developer building a real-time Persian voice conversation pipeline for a Reachy Mini robot is seeking advice on running both the LLM and text-to-speech locally on a GTX 1650 Ti with 4GB of VRAM, after cloud inference via the Qwen2.5-72B HF Inference API and the fa-IR-FaridNeural voice produced periodic ConnectTimeout errors on a roughly 0.7 Mbps connection. The developer reported that qwen2.5:3b run locally through Ollama on a CPU-only torch build produced poor Persian output with frequent code-switching into English, while the cloud 72B model fixed coherence and language consistency. The request asks for model, quantization, and alternative local Persian TTS recommendations.

Hi everyone, I’m building a real-time, two-way Persian voice conversation pipeline for a small robot project Reachy Mini with my professor. Current architecture: fa-IR-FaridNeural voice — cloud-only Why LLM/TTS ended up cloud-based: I initially tried qwen2.5:3b locally via Ollama CPU-only, since my venv’s torch build is CPU-only , but the output quality in Persian was poor — frequent code-switching into English/other languages, incoherent responses. Switching to Qwen2.5-72B via HF Inference API fixed this completely; responses are now coherent and consistently Persian. edge-tts also gives good voice quality. The problem: My internet connection is very slow and unstable ~0.7 Mbps down , so I’m getting periodic ConnectTimeout errors on the API calls I’ve added retry logic, which helps but doesn’t eliminate the issue . For a robot that needs to work reliably, I’d like to move both LLM and TTS to fully local inference if possible. My hardware: What I’m trying to figure out: Any pointers to models, quantization strategies, or alternative local Persian TTS projects would be hugely appreciated. Happy to share more details code, benchmarks if useful. Thanks in advance