Run LLM Locally – which open model fits your GPU or Mac, and how fast

A new interactive tool lets users enter their GPU or Mac hardware to see which open large language models will run locally, along with the memory each model requires and an estimated speed, or alternatively pick a model to see what hardware supports it. The tool's defaults suit most uses, with options for context length — 8K described as enough for chat while long documents need 32K or more — and quantization, where q8_0 roughly halves a conversation's memory and q4_0 nearly quarters it, provided the option is also enabled in the user's app. RAM speed is used to estimate performance only for the portion of the model that does not fit on the graphics card.