{"slug": "run-llm-locally-which-open-model-fits-your-gpu-or-mac-and-how-fast", "title": "Run LLM Locally – which open model fits your GPU or Mac, and how fast", "summary": "A new interactive tool lets users enter their GPU or Mac hardware to see which open large language models will run locally, along with the memory each model requires and an estimated speed, or alternatively pick a model to see what hardware supports it. The tool's defaults suit most uses, with options for context length — 8K described as enough for chat while long documents need 32K or more — and quantization, where q8_0 roughly halves a conversation's memory and q4_0 nearly quarters it, provided the option is also enabled in the user's app. RAM speed is used to estimate performance only for the portion of the model that does not fit on the graphics card.", "body_md": "Enter your GPU or Mac to see which models run, with the memory they need and an estimated speed. Or pick a model and see what hardware runs it.\n\nThe defaults suit most uses. You can skip this section.\n\nHow much text the model keeps in memory. 8K is enough for chat; long documents need 32K or more. What is this?\n\nLeave f16 if unsure. q8_0 roughly halves the conversation's memory, q4_0 nearly quarters it; the option must also be enabled in your app. What is this?\n\nUsed to estimate RAM speed, which only matters for the part of the model that does not fit on the graphics card.", "url": "https://wpnews.pro/news/run-llm-locally-which-open-model-fits-your-gpu-or-mac-and-how-fast", "canonical_source": "https://www.runllmlocally.com/en", "published_at": "2026-10-19 14:57:49+00:00", "updated_at": "2026-10-19 15:18:04.603060+00:00", "lang": "en", "topics": ["large-language-models", "ai-tools", "ai-infrastructure"], "entities": [], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/run-llm-locally-which-open-model-fits-your-gpu-or-mac-and-how-fast", "markdown": "https://wpnews.pro/news/run-llm-locally-which-open-model-fits-your-gpu-or-mac-and-how-fast.md", "text": "https://wpnews.pro/news/run-llm-locally-which-open-model-fits-your-gpu-or-mac-and-how-fast.txt", "jsonld": "https://wpnews.pro/news/run-llm-locally-which-open-model-fits-your-gpu-or-mac-and-how-fast.jsonld"}}