{"slug": "local-llm-on-macbook-m5-pro-totally-new-to-this", "title": "Local LLM on MacBook M5 Pro - Totally New to This!", "summary": "A non-programmer named Tim is setting up a local LLM on a MacBook M5 Max with 128GB unified memory, using Docker Desktop with Model Runner, Open WebUI, and models like Gemma 4 and Qwen3 30B-A3B-Q4_k_m for daily use and deep research. He has built RAG knowledge collections for personal topics and seeks guidance on advancing his setup beyond Claude Pro.", "body_md": "Ok to start with I am not a programmer or an IT person whatsoever. I have spent the last few months learning as much as I can about AI. I have been using Coursera to study topics on the various LLMS, CISSP, Python etc and continue to do so. I have a Claude Pro account that I am using to help me set this LLM up and have a temporary Gemini pro account via Coursera. I am trying to get a good general background on AI and security but I am still ignorant in so many ways.\n\nHardware:\n\nMacbook M5 Max, 18 core cpu, 40 core gpu, 128GB unified memory, 4TB hd, OS Tahoe\n\nWill eventually use Tailscale for remote access\n\nStack:\n\nDocker Desktop using Docker Model Runner for local inference (full Metal GPU/unified memory access currently). Open webUI as chat interface, in Docker via Compose\n\nModels:\n\nGemma 4 (~12B) - daily use\n\nQwen3 30B-A3B-Q4_k_m - deep research\n\nRAG:\n\nSentenceTransformers embedding, default function calling mode. Multiple topic-based knowledge collections (RV/camping, photography gear, truck, home equipment, etc) each one of these contain AI written md files with full manufacturer manuals pdf.\n\nOther tools:\n\nDrawThings - image/video generation\n\nMacWhisper Pro - audio/video transcription\n\nKokoro TTS - local voice output\n\nWhat is my objective?\n\nSo far this whole setup has been guided by Claude and has been a painful process but a learning one. I most likely went about this the wrong way and i’m ok with that but I feel like my next step is to make sure my models work the way I need them to and start using them more than Claude. I’m just not sure at my skill level where to go and what to do without it being too technical. I feel like there is a normal path that people follow but I don’t know what it is.\n\nI will likely be asking lots of questions of you guys, thank you,\n\nTim", "url": "https://wpnews.pro/news/local-llm-on-macbook-m5-pro-totally-new-to-this", "canonical_source": "https://discuss.huggingface.co/t/local-llm-on-macbook-m5-pro-totally-new-to-this/177286#post_1", "published_at": "2026-06-30 19:00:33+00:00", "updated_at": "2026-06-30 19:01:34.197885+00:00", "lang": "en", "topics": ["large-language-models", "ai-tools", "ai-infrastructure", "developer-tools"], "entities": ["MacBook M5 Max", "Docker Desktop", "Open WebUI", "Gemma 4", "Qwen3", "Claude Pro", "Coursera", "Tailscale"], "alternates": {"html": "https://wpnews.pro/news/local-llm-on-macbook-m5-pro-totally-new-to-this", "markdown": "https://wpnews.pro/news/local-llm-on-macbook-m5-pro-totally-new-to-this.md", "text": "https://wpnews.pro/news/local-llm-on-macbook-m5-pro-totally-new-to-this.txt", "jsonld": "https://wpnews.pro/news/local-llm-on-macbook-m5-pro-totally-new-to-this.jsonld"}}