05:00
2026-10-16
dev.to
machine-learning
Expert pruning improves decode throughput up to 1.8 with minimal quality loss
A developer's SlimWise system prunes Mixture-of-Experts experts only during the decode phase while reusing the full-model KV cache built during prefill, boosting decode throughput up to 1.81Ć at 50% eā¦