Micro-Expert-Router

mentions 2 type Organization feed RSS

// recent coverage 2 mentions

19:21

2026-06-04

dev.to

large-language-models

Running Mixtral 8x7B at 21+ TPS on Pure CPU via io_uring and Predictive Caching

Amalgafy Labs has achieved 21+ tokens per second inference on the Mixtral 8x7B model using only CPU and SSD storage, bypassing the need for GPU VRAM. The team's Micro-Expert-Router (MER) system levera…

03:32

2026-05-27

dev.to

large-language-models

I built a Rust inference engine that streams MoE expert weights from NVMe SSDs, no GPU required

A developer built Micro-Expert-Router, a Rust inference engine that streams Mixture-of-Experts model weights directly from NVMe SSDs using io_uring with O_DIRECT, eliminating the need for GPU VRAM. Th…

// co-occurs with top 8 entities

Mixtral 2 DeepSeek-V3 1 Apple 1 NVMe 1 Rust 1 SwiGLU 1 AVX2 1 Mixtral 8x7B 1