Inkling-2-Mini in 5.4 GB Memory LookAtMyQuant's quantization compiler compressed thinkingmachines/Inkling-2-Mini from 18.89 GB of text weights to 5.38 GB at 1.63 bits per weight, and to 8.87 GB at 2.69 bits per weight, using hashed block-vector trellis codes (ht2/ht3), row trellis codes (rtr1/rtr4) and a tr2 floor on the tail lm_head bucket (head.7). Evaluated against the same BF16 teacher, the 5.38 GB file scored 134/200 (67.0%) on GSM8K and 60/164 (36.6%) on HumanEval versus 147/200 (73.5%) and 66/164 (40.2%) for the original, while the 8.87 GB file reached 148/200 (74.0%) on GSM8K and 58/164 (35.4%) on HumanEval. The 5.38 GB build beat Unsloth's UD-IQ1_S (7.76 GB) on HumanEval, 36.6% against 28.0%, and the 8.87 GB build beat Unsloth UD-Q2_K_XL (9.14 GB) on both GSM8K (74.0% vs 66.0%) and HumanEval (35.4% vs 29.9%). LookAtMyQuant is a quantization compiler. It measures how much each stored piece of a model hurts the output when compressed. It then picks a code for each piece so the whole model fits a size budget with the least damage. On Gemma 4 E2B, its GGUF has 12× lower KL than Q4 K M at the same file size. The method and every result are on the site https://lookatmyquant.pages.dev . The code is under the MIT license. LookAtMyQuant 2 https://lookatmyquant2.pages.dev takes Qwen3.5-4B to 1.23 GB, 2.3 bits per weight. The file solves 330 of 540 exact-answer math problems. Prism ML's Bonsai 2 Ternary 4B solves 328 at 1.20 GB, and the original solves 359. Its KL from the original on held-out text is 0.45 nats, against 0.59 for Bonsai. The recipe is post-training quantization with trellis codes, calibrated on the model's own worked math solutions with digit tokens weighted 10×, and with bits allocated per row by Fisher information. 4,000 steps of distillation follow, in which every layer is re-quantized once from the original weights. The page has the findings, the full recipe and its cost, about $34 of GPU time. The recipe's code is here, and the next section runs it. The 1.23 GB file itself is the lm2-qwen35-4b-1.2gb release https://github.com/ShinMegamiBoson/lookatmyquant/releases/tag/lm2-qwen35-4b-1.2gb , packed for the Apple Silicon runtime. The same recipe takes Qwen3.8-27B from 53.7 GB to 7.2 GB, 2.1 bits per weight, calibrated and distilled on English prose and chat, code and math. The file https://huggingface.co/shinmegamiboson/Qwen3.8-27B-LookAtMyQuant-7.2GB solves 401 of the 540 math problems, as many as the original, and Unsloth's UD-IQ2 XXS GGUF of the same size solves 348. It also scores above that GGUF on HumanEval 73.8% against 67.1% , MMLU 78.8% against 75.0% and GSM8K 94.5% against 92.7% . Beside it sits a 4-bit DSpine drafter https://huggingface.co/shinmegamiboson/Qwen3.8-27B-LookAtMyQuant-7.2GB-drafter 1.02 GB, converted from tencent/Qwen3.8-27B-DSpine-b16 with runtime mlx/convert dspine.py and packed with pack drafter.py , which roughly doubles decode speed on math and code on Apple silicon. The same compiler and proximal fine-tuning SQYFR , extended to a 26.4B physical / 5.4B active 3×-looped MoE L6–17 looped 3× = 48 logical visits, 128 experts top-8 , compresses thinkingmachines/Inkling-2-Mini https://huggingface.co/thinkingmachines/Inkling-2-Mini from 18.89 GB of text weights to 5.38 GB 1.63 bits/weight https://huggingface.co/shinmegamiboson/Inkling-2-Mini-LookAtMyQuant-5.4GB and 8.87 GB 2.69 bits/weight https://huggingface.co/shinmegamiboson/Inkling-2-Mini-LookAtMyQuant-8.9GB using 4 KB SRAM hashed block-vector trellis codes ht2 / ht3 , row trellis codes rtr1 / rtr4 , and a tr2 floor on the tail lm head bucket head.7 . Evaluated on the same BF16 teacher, openai/gsm8k test :200 , greedy, reasoning effort="none" , all 164 executable openai/openai humaneval problems, 4 Vision VQA tests with the untouched 637 MB BF16 vision tower , and 16+13-language TextMix windows: | Model | Text GB | free KL en/chat/code | all13 KL | GSM8K 200 | HumanEval 164 | Pooled vs Teacher | |---|---|---|---|---|---|---| | Original BF16 Teacher | 18.89 | 0.0000 | 0.0000 | 147 / 200 73.5% | 66 / 164 40.2% | 100.0% | | LookAtMyQuant 5.4 GB SQYFR | 5.38 | 0.0895 | 0.2871 | 134 / 200 67.0% | 60 / 164 36.6% | 91.2% | | Unsloth UD-IQ1 S | 7.76 | 0.0808 | 0.1236 | 127 / 200 63.5% | 46 / 164 28.0% | 81.6% | | LookAtMyQuant 8.9 GB SQYFR | 8.87 | 0.0208 | 0.0571 | 148 / 200 74.0% | 58 / 164 35.4% | 96.8% | | Unsloth UD-Q2 K XL | 9.14 | 0.0486 | 0.0811 | 132 / 200 66.0% | 49 / 164 29.9% | 85.3% | | Unsloth UD-IQ3 XXS | 11.66 | 0.0189 | 0.0278 | 148 / 200 74.0% | 68 / 164 41.5% | 101.4% | At 5.38 GB , Inkling-2-Mini-LookAtMyQuant-5.4GB outscores Unsloth's 9.14 GB UD-Q2 K XL on both GSM8K and HumanEval at 41% smaller size up from 76/200 and 24/164 untuned . At 8.87 GB , Inkling-2-Mini-LookAtMyQuant-8.9GB matches the BF16 original on GSM8K and cuts KL 2.34× on English/chat/code and 29.6% across all 13 languages below UD-Q2 K XL . Both run directly through lamq and runtime mlx with custom Apple Silicon Metal kernels, optionally paired with the 178.9 MB 4-bit DSpark drafter https://huggingface.co/shinmegamiboson/Inkling-2-Mini-LookAtMyQuant-drafter : on M5 Max, 5.4 GB decodes at 73.2 tok/s serial and 125.2 tok/s median 141.8 tok/s math, 129.0 tok/s reasoning, 121.4 tok/s code, 102.3 tok/s prose; 1,037 tok/s prefill at T=512 , 5.83 GB active RAM , and 8.9 GB decodes at 63.1 tok/s serial and 101.3 tok/s median 133.9 tok/s reasoning, 118.9 tok/s math; 838 tok/s prefill at T=512 , 9.32 GB active RAM . On Apple silicon, install the runtime package directly this installs mlx , mlx-lm , numpy , and huggingface-hub without the PyTorch compiler dependencies : pip install git+https://github.com/ShinMegamiBoson/lookatmyquant.git From a local clone, pip install -e . installs the lamq command, or run python -m runtime mlx without installing. Command line. Pass either a local model folder or a Hugging Face repo ID: lamq generate shinmegamiboson/Qwen3.8-27B-LookAtMyQuant-7.2GB \ -p "What is 17 times 23? Show the steps." lamq generate shinmegamiboson/Qwen3.8-27B-LookAtMyQuant-7.2GB \ --drafter shinmegamiboson/Qwen3.8-27B-LookAtMyQuant-7.2GB-drafter \ -p "Write a Python function that merges overlapping intervals." --stats lamq chat shinmegamiboson/Qwen3.8-27B-LookAtMyQuant-7.2GB \ --drafter shinmegamiboson/Qwen3.8-27B-LookAtMyQuant-7.2GB-drafter lamq generate shinmegamiboson/Inkling-2-Mini-LookAtMyQuant-5.4GB \ --drafter shinmegamiboson/Inkling-2-Mini-LookAtMyQuant-drafter \ -p "What is 17 times 23? Show the steps." --stats lamq generate reads stdin when -p is omitted, supports --raw , --system , --thinking , --temp , --mode linear|chat-fork|copy-wide , --no-adaptive , and --json . lamq info