# Local AI on a PC: what actually works (Whisper, translation, LLMs)

> Source: <https://dev.to/cjdarklord/local-ai-on-a-pc-what-actually-works-whisper-translation-llms-1323>
> Published: 2026-11-19 01:10:06+00:00

**Recommendation:** Gate speech recognition with a VAD filter to suppress Whisper hallucinations on silence, and check model licenses like NLLB's CC BY-NC before shipping commercially.
More in the playbook "Inference and self-hosting": <https://wpnews.pro/playbooks/inference-serving>

Local AI is the most overpromised topic in dev tooling. "Run GPT on your laptop" — sure, if your laptop has 12 GB of VRAM and you enjoy 3 tokens per second.

I've been running local models in production-ish settings for a while (speech recognition, translation, TTS, document analysis). Here is what actually works on consumer hardware, and what doesn't.

## 
  
  
  Speech recognition: faster-whisper

The best value-per-watt local model I use. CTranslate2 backend, int8 quantization.

- 
**medium int8** on an RTX 2070: ~**4.4 GB of VRAM** , a 30-second fragment in**0.36–1.62 s** depending on speech density;
- CPU works too, several times slower — fine for batch transcription, too slow for real-time subtitles;
- hallucination filter is mandatory: silence and noise produce plausible-looking text that nobody said. A VAD gate before recognition fixes most of it.

## 
  
  
  Translation

- 
**Helsinki-NLP OPUS models** : ~74 MB, fast, quality is decent for short, well-formed sentences — exactly what subtitles are;
- 
**NLLB / M2M100** : better quality, but check the licence. NLLB is**CC BY-NC** — fine for personal use, not for a commercial product. I learned that the hard way and swapped it;
- for anything legal or medical, local NMT is not good enough and you should say so rather than ship it.

## 
  
  
  TTS

- 
**Piper** : tiny, fast, runs on CPU, sounds robotic but intelligible — good for notifications and accessibility;
- 
**XTTS** : much better quality, needs a GPU and a licence read.

## 
  
  
  LLMs

A 7–8B model in Q4 quantization takes 8–12 GB of VRAM and produces **20–40 tokens/sec** on a decent GPU. That is enough for:

- summarising long documents;
- extracting fields from PDFs and invoices;
- classification and routing;
- "explain this error" against your own logs.

It is not enough to replace a frontier model for open-ended reasoning — and if you're paying per token, the API is cheaper until you cross a volume threshold.

## 
  
  
  The decision that actually matters

Local vs API is an economics and privacy decision, not a religious one:

- 
**API is cheaper at the start** — dozens of operations a month don't justify hardware;
- 
**local is cheaper at volume** — thousands of documents a month amortise the GPU in months;
- 
**local is mandatory for sensitive data** — medical, personal, trade secrets;
- 
**hybrid is often best** — sensitive data locally, human-facing drafts through an API.

Start with a prototype on your own data. It tells you both quality and real speed, and the decision becomes arithmetic instead of vibes.

*Longer write-up with the hardware table, model sizes and real benchmarks: [Local AI on a PC: what actually works](https://tenlux.tech/en/blog/local-ai-on-a-pc.html). A real product built this way: [Stream-Subs](https://tenlux.tech/stream-subs/en/).*
