3 Ways to Boost MoE Inference on NVIDIA DGX Spark
Swapping the FlashInfer MXFP4 MoE backend for the Humming kernel library cut time per output token from 27.71 ms to 25.35 ms and raised decode throughput from 36.09 to 39.45 tokens per second, a 9.3% gain, in a batch-siz…