A 29B mixture-of-experts text model from XingChen-AGI under Apache 2.0 that reads about 4B parameters per token. Attention is DeepSeek-style MLA with a compressed cache of about 46 KB per token, so context is cheap next to the weights: at Q4_K_M a 32K window adds only 1.4 GB. At Q4_K_M it needs about 21.2 GB at 4K, tight on a 24 GB card. IQ4_XS needs about 19.0 GB at 4K, comfortable on a 24 GB card. Q3_K_M needs about 17.0 GB at 4K, comfortable on a 20 GB card. Q2_K needs about 13.6 GB at 4K, comfortable on a 16 GB card. The full 262,144-token window at Q4_K_M needs about 33.3 GB at 256K. Each GGUF file also carries a 1.7B-parameter multi-token-prediction block that llama.cpp skips unless speculative decoding asks for it, so real use is about 1 GB under these figures at Q4_K_M. Sizes come from bartowski's GGUF files, and no per-level quality loss has been published.
Not all uncensored models are created equal