# Running local AI models

> Source: <https://discuss.huggingface.co/t/running-local-ai-models/192189#post_3>
> Published: 2026-11-20 09:25:14+00:00

**Recommendation:** On 8GB machines, start with quantized 1B–2B models for narrow tasks and watch Memory Pressure and Swap Used in Activity Monitor before attempting 7B–8B models.
More in the playbook "Inference and self-hosting": <https://wpnews.pro/playbooks/inference-serving>

I’m running Ollama on an 8GB M1 MacBook Pro, so pretty similar constraints to yours!

I’ve experimented with smaller Qwen models (0.6B and 1.7B), mainly for classification, information extraction and structured JSON outputs.

A few things I’ve learned:

Smaller models are surprisingly usable if you give them specific, narrow tasks. I wouldn’t expect them to perform like Claude or ChatGPT, though.

I use Activity Monitor to keep an eye on Memory Pressure and Swap Used. The 8GB fills up pretty quickly, especially with browsers and other apps running.

Quantized models help considerably. I’d start with something around 1B to 2B parameters rather than jumping straight into 7B or 8B models.

For more demanding tasks, hosted inference is definitely less frustrating.

My smaller models have managed around 1.5 to 1.9 seconds on some simple tasks, although that obviously depends on the prompt and output length.

I’m still learning and experimenting myself, but my biggest takeaway is that local inference on 8GB is absolutely possible. You just have to be realistic about what you’re asking the model to do.

Curious what models you’re planning to try!
