A new CPU-only inference runtime called cpubrrr achieves up to 5× faster token generation than llama.cpp's CPU path on frontier-class mixture-of-experts models, running on an Apple M4 Max without GPU …
A developer built ignis, a from-scratch LLM inference engine in Rust with only two dependencies, to explore how compiler design principles apply to inference. The engine uses SSA IR, fusion passes, an…
A developer demonstrates a technique for hot-swapping QLoRA adapters on Android devices, enabling task-specific LLM behaviors without reloading the base model. By loading a single 4-bit quantized base…
Researchers have developed ForestHG-Trace, a framework enabling traceable long-horizon ecological reasoning over large-scale forest scenes by representing multimodal NEON forest data as ecological hyp…