Inside the megakernel serving engine for North Mini Code
Cohere released a serving engine for its North Mini Code model built around a decode megakernel that runs 1.25x to 1.41x faster than vLLM end-to-end on a single H100 with BF16 precision. The engine, a…