Show HN: Quant Picker – which GGUF file fits your model and machine

Quant Picker is a new tool that calculates which GGUF quantization level fits a given model and machine, balancing file size, quality, and context budget. It recommends the highest quantization that leaves at least 8k context, based on community consensus from a quantization guide.

How to read the table Every GGUF model ships in multiple quantization levels — same model, different precision, different file size. The trade is simple: more bits = better quality = bigger file = less room left for context . This tool does the arithmetic for your exact machine: file size per quant, then whatever memory remains becomes your context budget the KV cache https://vettedconsumer.com/the-kv-cache-explained-why-long-context-eats-your-vram-and-how-to-fit-more/ eats it per token . The recommendation logic is the community consensus from our quantization guide https://vettedconsumer.com/gguf-vs-gptq-vs-awq-the-plain-english-guide-to-llm-quantization-and-which-one-to-pick/ : take the highest quant that still leaves ≥8k of context . Q6/Q5 are near-lossless, Q4 K M is the sweet spot, and below Q3 quality falls off fast — if you're forced down there, you usually want a smaller model instead a bigger model at Q4 beats a smaller one at Q8, but a Q2 of anything beats very little . Honest limits File sizes are computed from bits-per-weight, not scraped from Hugging Face — real files vary a little by quantizer version K-quants vs I-quants, imatrix variants . The KV-cache math assumes a GQA-typical architecture; exotic models differ. And max context here is what fits — models also have their own context limits, and quality at extreme context is its own story. Treat the numbers as a reliable guide, not a contract. The tool family Shopping rather than downloading? Can I run it? https://vettedconsumer.com/can-i-run-it/ finds hardware that fits a model. Wondering if you should buy hardware at all? The cost calculator https://vettedconsumer.com/cost-calculator/ compares buying vs renting vs the API.