Member-only story
Hardware math, quantization tradeoffs, five inference engines benchmarked, and two real case studies where the numbers met reality on my own machine. #
Most “run an LLM locally” guides stop at ollama pull
and a happy screenshot. This one doesn't. This is the reference I wish existed when I started actually building on local models instead of just trying them once: real hardware math, the tradeoffs between five different inference engines, what quantization actually costs you, and — because benchmarks lie by omission — two real tests I ran myself where the theory either held up or didn't.
One honesty note before we start, because I think it matters: everything about hardware math and the two model case studies comes from my own machine — a 16GB Apple Silicon Mac, running through Ollama. The broader engine comparisons (vLLM, text-generation-webui, SGLang) are researched and cited, not things I’ve personally run at scale myself. I’ll flag which is which as we go, because I think a guide is more useful when you know exactly which parts are “I tested this” versus “this is what the evidence says,” rather than blurring the two.