Cohere released a serving engine for its North Mini Code model built around a decode megakernel that runs 1.25x to 1.41x faster than vLLM end-to-end on a single H100 with BF16 precision. The engine, a…
Stanford's Hazy Research group found that the intelligence per joule (IPJ) of local AI inference improved 18-fold from mid-2024 to late 2025, driven by a 3.1x gain from model architecture improvements…
French startup Kog claims its software can deliver 30x faster LLM inference on standard datacenter GPUs, demoing 3,000 tokens per second on a 2-billion-parameter model using AMD MI300X and NVIDIA H200…
Hazy Research, the team behind ThunderKittens and Megakernels, argues that CUDA domain-specific languages (DSLs) are heading toward retirement as AI agents take over the cognitive offloading role that…
A Stanford University study published November 2025 introduced "intelligence per watt" (IPW) as a metric for AI inference efficiency, finding that local language models can accurately answer 88.7% of …