{"slug": "show-hn-shoehorn-quantize-any-model-down-to-run-on-your-machine", "title": "Show HN: Shoehorn – Quantize any model down to run on your machine", "summary": "Shoehorn, a new open-source tool from developer notactuallytreyanastasio, quantizes any language model to fit a user's available memory, using up to 99.99% of the budget. It solves a per-tensor mixed-precision assignment based on the user's hardware, and integrates with llama.cpp as the inference backend. The tool is available via Homebrew or Cargo, and includes a local web app that measures the machine, streams the fit, and shows the perplexity cost.", "body_md": "# shoehorn\n\nMake any language model fit the memory you actually have.\n\nPreset quantizations ignore your hardware: pick one that fits and you\neither waste hundreds of megabytes of quality headroom or find out at load time it\ndidn't fit after all. shoehorn starts from the memory you actually have, subtracts\nwhat inference itself needs, and solves a per-tensor mixed-precision assignment\nthat lands within a rounding error of the remainder — routinely using\n**99.99%** of the budget, sometimes to the byte.\n\n**weights: 519.2 MiB of 519.2 MiB budget (99.998% used, 13 KB slack)**\n\n## What fits your machine?\n\nPick your hardware and this page scans Hugging Face's most-downloaded models for ones shoehorn can fit to your budget — ranked by the quality your memory affords. Runs entirely in your browser.\n\n## Install\n\nshoehorn needs [llama.cpp](https://github.com/ggml-org/llama.cpp)\non your PATH as the inference backend (the Homebrew install pulls it in for you).\nThen `shoehorn ui`\n\nopens the local app — pick a model, press one\nbutton, chat.\n\n`brew install notactuallytreyanastasio/shoehorn/shoehorn`\n\nOr from source: `cargo install --path .`\n\nafter cloning [the repo](https://github.com/notactuallytreyanastasio/shoehorn).\nAll [releases](https://github.com/notactuallytreyanastasio/shoehorn/releases).\n\n## One button, your whole budget\n\nThe local web app measures your machine, streams the fit, renders the budget as a tape measure, puts a perplexity number on what the fit cost, and ends at a Chat button.", "url": "https://wpnews.pro/news/show-hn-shoehorn-quantize-any-model-down-to-run-on-your-machine", "canonical_source": "https://notactuallytreyanastasio.github.io/shoehorn/", "published_at": "2026-08-18 14:29:20+00:00", "updated_at": "2026-08-18 16:11:01.140038+00:00", "lang": "en", "topics": ["artificial-intelligence", "machine-learning", "large-language-models", "ai-tools", "developer-tools"], "entities": ["Shoehorn", "notactuallytreyanastasio", "llama.cpp", "Hugging Face", "Homebrew", "Cargo"], "alternates": {"html": "https://wpnews.pro/news/show-hn-shoehorn-quantize-any-model-down-to-run-on-your-machine", "markdown": "https://wpnews.pro/news/show-hn-shoehorn-quantize-any-model-down-to-run-on-your-machine.md", "text": "https://wpnews.pro/news/show-hn-shoehorn-quantize-any-model-down-to-run-on-your-machine.txt", "jsonld": "https://wpnews.pro/news/show-hn-shoehorn-quantize-any-model-down-to-run-on-your-machine.jsonld"}}