An optimized training stack can reach roughly 51% model FLOPs utilization (MFU) on consumer GPUs, according to the LLMQ paper, leaving headroom for a user currently getting about 65 TFLOPS of useful c…
Researchers introduced two CUTLASS-based kernels that accelerate the SwiGLU activation function in large language models by up to 2.47x on NVIDIA H100 GPUs, shifting workloads from memory-bound to com…
Noam Shazeer, co-author of the seminal Transformer paper and a key engineer at Google, has left for OpenAI less than two years after Google spent $2.7 billion to reacquire him from Character.AI. The m…
Researchers have introduced CosmicFish-HRM, a compact language model that uses a Hierarchical Reasoning Module (HRM) to dynamically allocate computational effort based on input complexity rather than …
A developer built Micro-Expert-Router, a Rust inference engine that streams Mixture-of-Experts model weights directly from NVMe SSDs using io_uring with O_DIRECT, eliminating the need for GPU VRAM. Th…