DFlash

Xiaomi's MiMo-V2.5-Pro-UltraSpeed, a 1.02-trillion-parameter MoE model, achieved 1,000 tokens per second inference on standard cloud GPUs using FP4 quantization, DFlash speculative decoding, and TileR…

Speculative decoding, a lossless inference optimisation that predicts future tokens to reduce latency, faces new economic constraints as modern mixture-of-experts (MoE) architectures replace dense tra…

Researchers at UC San Diego, led by Hao Zhang, have successfully implemented a diffusion-style speculative decoding method called DFlash on Google TPUs, achieving an average 3.13x increase in tokens p…