Google researchers built the first portable vectorized Quicksort, using Highway's portable SIMD functions to run on six instruction sets across three architectures, and reported a 9-19x speedup over t…
A new arXiv paper (2609.16338v1) introduces BITCOS, a distribution-adaptive storage layout for ternary LLMs that costs 2 − z bits per weight given a zero density z, after measuring 29 ternary models a…
NanoGEMM, a bare-metal AVX2/FMA matrix multiplication engine, claims to outperform NumPy 2.2.3 on small-to-medium matrices, achieving a 2.83x speedup on 16x16 float32 matrices (1.23 µs vs 3.21 µs) but…
A developer built a C99 inference engine that runs DeepSeek-V4-Flash, a 284B-parameter mixture-of-experts model, on a laptop with just 3.2GB of RAM by streaming weights off NVMe and caching only the e…
Llambda.lisp, a pure Common Lisp inference engine for running quantized large language models from .gguf files, achieves C-level throughput using AVX2/FMA acceleration and multi-threading without wrap…
A developer built Micro-Expert-Router, a Rust inference engine that streams Mixture-of-Experts model weights directly from NVMe SSDs using io_uring with O_DIRECT, eliminating the need for GPU VRAM. Th…