Show HN: What LLM fits my GPU? – VRAM measured on real hardware for 117 models

A Show HN project launched on 2026-10-23 that indexes 117 LLMs and 78 GPUs, pairing each model with VRAM measured on real hardware at 4K context and batch 1, including weights, KV cache and 10% activations. The site reports throughput and memory figures per quantization — for example, Llama 3.1 8B on an RTX 4090 24G with ExLlamaV2 EXL2 4.65bpw at 235 tok/s and 5.4 GB VRAM, versus 68 tok/s and 5.7 GB on an M3 Max 48G with Ollama GGUF Q4_K_M — and lists quality loss per quant, such as 3.8% for Mistral Small 24B Instruct AWQ INT4. The index aims to let users pick a card and task to see which models fit, the memory math behind each, and the command to run it.

2026-10-23latest update https://quantized.uk/changelog/ Will it fit on your card? Quantize everything. Pick your card and what you want it for. You get models that actually fit, the memory maths behind each one, and the command to run it. New here? Start with the 8GB guide https://quantized.uk/cookbook/8gb-gpu-starter-guide/ What fits your card Sized at 4K context, batch 1 — weights + KV cache + 10% activations, straight from the model index. ExampleShowing RTX 4060 Ti 16G as an example — pick your own hardware above to see yours. - Most capable that fits Mistral Small 24B Instruct https://quantized.uk/quant-hub/mistral-small-24b/ AWQ · INT4 - 13 GB of 16 GB - 2.8 GB free - 3.8% quality loss - 78 tok/s RTX 4090 Estimated The largest model in the index that still fits this card comfortably at 4K context. - Room to grow Falcon 3 10B Instruct https://quantized.uk/quant-hub/falcon-3-10b/ GGUF · Q4 K M - 7.3 GB of 16 GB - 8.7 GB free - 3.1% quality loss - 118 tok/s RTX 4090 Estimated The largest model that uses under 60% of the card, leaving space for much longer context. - Fastest Qwen2.5 0.5B Instruct https://quantized.uk/quant-hub/qwen2.5-0.5b/ GGUF · Q8 0 - 0.6 GB of 16 GB - 15 GB free - 0.5% quality loss - 540 tok/s RTX 4090 Estimated Highest measured throughput of everything that fits — usually a small model. Measured on an RTX 4090, so read it as a ranking, not as your number. All models that fit RTX 4060 Ti 16G → https://quantized.uk/gpu/rtx-4060-ti-16g/ Data updated 2026-10-23 Latest additions The newest models and hardware in the index · data last updated2026-10-23 Full changelog → https://quantized.uk/changelog/ Popular starting points By hardware Every card has its own page: what fits comfortably, what is tight, what to step up to. All 78 GPUs → https://quantized.uk/gpu/ By task Filtered views of the index. Start here Measured, not guessed A sample of the benchmark table. Every row is a run on real hardware, recorded with the stack it ran on. | Model | Hardware | Stack | tok/s | VRAM | |---|---|---|---|---| | Llama 3.1 8B | RTX 4090 24G | ExLlamaV2 · EXL2 4.65bpw | 235 | 5.4 GB | | Qwen3 30B-A3B | RTX 4090 24G | llama.cpp · GGUF Q4 K M | 95 | 19 GB | | Llama 3.1 8B | M3 Max 48G | Ollama · GGUF Q4 K M | 68 | 5.7 GB | | Llama 3.1 8B | RTX 3090 24G | ExLlamaV2 · EXL2 4.65bpw | 175 | 5.4 GB | Method: Meta Llama 3.1 8B Instruct · WikiText-2 · 4096 ctx · batch 1 Local inference runs on RTX 4090 / 3090 / M3 Max / M2 Ultra; llama.cpp b4000+, ExLlamaV2 0.2.x, vLLM 0.6.x, Ollama 0.3.x Full benchmark table and methodology → https://quantized.uk/benchmarks/ Editor's Picks Curated standout quant releases — click to view model details View All → https://quantized.uk/quant-hub/ - NEW https://quantized.uk/quant-hub/k2-horizon-7b/ K2 Horizon 7BGGUFQ4 K M · 6.5 GB at 4K · Fully open, data included, 8 GBIFMComfortable from RTX 4060 8 GB - NEW https://quantized.uk/quant-hub/minicpm-v-4-7-35b-a3b/ MiniCPM-V 4.7 35B-A3BGGUFQ4 K S · 20.9 GB at 4K · Vision MoE, 3B active, 24 GBOpenBMBComfortable from RTX 3090 24 GB - NEW https://quantized.uk/quant-hub/north-mini-code-1.0/ North Mini Code 1.0 30B-A3BGGUFQ4 K M · 20.0 GB at 4K · Coding MoE, 3B active, 24 GBbartowskiComfortable from RTX 3090 24 GB - NEW https://quantized.uk/quant-hub/gemma-4-12b/ Gemma 4 12B ITGGUFQ4 K M · 8.0 GB at 4K · 12B, 256K window on 16 GBunslothComfortable from Arc B570 10G 10 GB - NEW https://quantized.uk/quant-hub/beam-1.0-501b-a23b/ Beam 1.0 501B-A23BGGUFQ2 K · 190.9 GB at 4K · 501B, Apache 2.0, 1M windowDevQuasarComfortable from Mac M5 Ultra 512G 512 GB - NEW https://quantized.uk/quant-hub/ling-3.1-flash/ Ling-3.1-flashGGUFQ4 K M · 370.6 GB at 4K · 560B, 25B active, 9 of 72 layers cacheAtomicChatNo single card fits it comfortably - NEW https://quantized.uk/quant-hub/lfm2-5-24b-a2b/ LFM2.5 24B-A2BGGUFQ4 K M · 15.3 GB at 4K · 2.45B active, 128K windowLiquidAIComfortable from Radeon RX 7900 XT 20 GB - NEW https://quantized.uk/quant-hub/nemotron-3.5-lightning-30b-a3b/ Nemotron 3.5 Lightning 30B-A3BGGUFQ4 0 · 19.8 GB at 4K · 3B active, 256K on 24 GBggml-orgComfortable from RTX 3090 24 GB - NEW https://quantized.uk/quant-hub/nemotron-3.5-bolt-74b-a9b/ Nemotron 3.5 Bolt 74B-A9BGGUFIQ2 XXS · 23.1 GB at 4K · 74B built for 24 GBnvidiaComfortable from RTX 5090 32 GB - NEW https://quantized.uk/quant-hub/granite-4.2-8b/ Granite 4.2 8BGGUFQ4 K M · 6.3 GB at 4K · dense, thinking by defaultibm-graniteComfortable from RTX 4060 8 GB - NEW https://quantized.uk/quant-hub/olmo-3-7b/ OLMo 3 7B InstructGGUFQ4 K M · 6.8 GB at 4K · fully open, 64K windowallenaiComfortable from RTX 4060 8 GB - NEW https://quantized.uk/quant-hub/ernie-4.5-21b-a3b/ ERNIE 4.5 21B-A3BGGUFQ4 K M · 14.2 GB at 4K · 3B active, 128K windowbaiduComfortable from Radeon RX 7900 XT 20 GB - NEW https://quantized.uk/quant-hub/smollm3-3b/ SmolLM3 3BGGUFQ4 K M · 2.3 GB at 4K · training data publishedHuggingFaceTBComfortable from Mac M1 8G 8 GB - NEW https://quantized.uk/quant-hub/gemma-4-e4b/ Gemma 4 E4B ITGGUFQ4 K M · 4.9 GB at 4K · image + audio on 8 GBgoogleComfortable from Mac M1 8G 8 GB - NEW https://quantized.uk/quant-hub/gemma-4-26b-a4b/ Gemma 4 26B-A4B ITGGUFQ4 K M · 18.2 GB at 4K · 4B active, 256K windowgoogleComfortable from RTX 3090 24 GB - NEW https://quantized.uk/quant-hub/gemma-4-31b/ Gemma 4 31B ITGGUFQ4 K M · 21.1 GB at 4K · 10 of 60 layers cachegoogleComfortable from RTX 3090 24 GB - NEW https://quantized.uk/quant-hub/glm-4.7-flash/ GLM-4.7-FlashGGUFQ4 K M · 19.2 GB at 4K · 3B active, MLA cachezai-orgComfortable from RTX 3090 24 GB - NEW https://quantized.uk/quant-hub/kimi-linear-48b-a3b/ Kimi Linear 48B-A3B InstructGGUFQ4 K M · 30.4 GB at 4K · 1M context, 7 of 27 layers cachemoonshotaiComfortable from A100 40G 40 GB - NEW https://quantized.uk/quant-hub/qwen3-8-27b/ Qwen3.8 27BGGUFQ4 K M · 17.5 GB at 4K · 16 of 64 layers cache KVunslothComfortable from Radeon RX 7900 XT 20 GB - NEW https://quantized.uk/quant-hub/ministral-3-8b/ Ministral 3 8B InstructGGUFQ4 K M · 6.0 GB at 4K · official Mistral GGUFmistralaiComfortable from RTX 4060 8 GB - NEW https://quantized.uk/quant-hub/qwen3-vl-8b/ Qwen3-VL 8B InstructGGUFQ4 K M · 6.2 GB at 4K · vision-language at 8BbartowskiComfortable from RTX 4060 8 GB - HOT https://quantized.uk/quant-hub/gpt-oss-20b/ GPT-OSS 20BGGUFMXFP4 · 12.8 GB at 4K · native 4-bitopenaiComfortable from RTX 4060 Ti 16G 16 GB - HOT https://quantized.uk/quant-hub/qwen3-coder-30b-a3b/ Qwen3-Coder 30B-A3B InstructGGUFQ4 K M · 19.7 GB at 4K · agentic coderbartowskiComfortable from RTX 3090 24 GB Before you download a 20GB file All 27 questions https://quantized.uk/faq/ How much VRAM do I need to run a 7B model? About 5.1 GB at Q4 K M with a 4K context window — 4.2 GB of weights, 0.5 GB of KV cache and a 0.5 GB activation buffer, measured on WizardLM-2 7B 7B . Every 8 GB card in this index clears that with room left. The figure rises with context, not with how hard you use the model: the same file at 32K needs about 9.0 GB, because the KV cache grows to 4.0 GB. Can I run a 24B model on a 16GB card? At 4 bits and short context, yes — Mistral Small 24B Instruct at AWQ INT4 needs about 13.2 GB of 16 GB, leaving 2.8 GB. The level matters more than the parameter count: the same model at Q4 K M is 15.9 GB, which is 99% of the card — it loads on a card with nothing else on it and has no margin for a longer window. A 16 GB card runs 64 of the 117 models here comfortably. How much quality do you lose at 4-bit? At Q4 K M the median published perplexity loss across the 79 models here that report one is 2.9% — 97.1% retained — with the range running 1.4% to 5.2%. Two caveats that matter more than the median: smaller models lose more than larger ones at the same level, and perplexity is a language-modelling measure, not a measure of whether the model still does your task. Models without a published figure are shown as a dash here rather than assumed to be lossless. GGUF or AWQ — which should I use? GGUF unless you are running a server. GGUF runs on anything — CPU, NVIDIA, AMD, Apple — and 117 of the 117 models here ship it, against 53 for AWQ. AWQ's case is throughput under vLLM with batched requests, on an NVIDIA card or one of the Radeon cards vLLM's ROCm build supports; for one person talking to one model, it buys you nothing GGUF does not already do. 53 models here ship both, which is the only set where the two can be compared on the same weights. Found a number that looks wrong? This index is maintained by hand and checked against public model cards. Figures marked “estimated” are calculated from the architecture, not measured on a card — if one does not match what you see on your own hardware, say so and it gets corrected. Data Changelog Last updated 2026-10-23 Show all 195 entries https://quantized.uk/changelog/ 2026-10-23llama.cpp is now at b11829; Ollama v0.40.8 , vLLM v0.32.0 and ExLlamaV3 v1.6.4 are unchanged. Poolside's Laguna S 2.1, a 118B mixture-of-experts coding model with about 8B parameters active per token, is now on the not-runnable-yet list: llama.cpp supports its architecture, but its layer layout and file sizes have not been checked. All data is manually curated and verified against community sources. Always cross-check with official Hugging Face model cards before deployment.