A new arXiv preprint (arXiv:2609.04453v1) finds that expert pruning in Mixture-of-Experts (MoE) models fails under over-dispersed routing, where aggressive load-balancing makes router probabilities un…
A new arXiv paper (2609.03150v1) reports that token-level routing alone does not prevent negative transfer in MoE+LoRA fine-tuning, as adding biomedical data to code and math training increased code p…
A new open-source project, Expert Sniper, pools ordinary Macs into a network that runs Mixture-of-Experts (MoE) models none of them could run alone, achieving 4.0 tok/s on a 16 GB M4 mini running Qwen…
Frink, a pure-Rust GGUF inference engine written by Antonello Frink, reached parity with llama.cpp on Apple Metal decode and passed it on prefill at version v0.12.0, with OLMoE-1B-7B prefill rising fr…
Ferrox v0.9.1, a pure-Rust GGUF inference engine, shipped today with MoE prefill on Apple Metal 2.4x faster, closing the gap to llama.cpp from 2.62x behind to 1.11x on OLMoE-1B-7B. The update also imp…
Ferrox, a pure-Rust GGUF inference engine, now runs mixture-of-experts (MoE) prefill on Apple Metal 2.4x faster, reaching 1402 tok/s on OLMoE-1B-7B and closing the gap to llama.cpp from 2.62x behind t…