Q8_0

OpenBMB's MiniCPM5-2B GGUF quantization, tested hands-on with llama.cpp, shows degraded performance on complex tasks: the Q8_0 model invented a nonexistent column in a SQL debugging task and failed a …

A developer built ignis, a from-scratch LLM inference engine in Rust with only two dependencies, to explore how compiler design principles apply to inference. The engine uses SSA IR, fusion passes, an…