French startup Kog claims its software can deliver 30x faster LLM inference on standard datacenter GPUs, demoing 3,000 tokens per second on a 2-billion-parameter model using AMD MI300X and NVIDIA H200…
Kog, an eleven-person French startup, claims that the next leap in AI inference speed will come from software optimizing existing GPUs rather than new silicon, and has demonstrated 30× faster decoding…
Kog released Laneformer 2B, a 2.3B-parameter coding model designed for high-speed decoding, on Hugging Face Hub. The model was trained from scratch with latency as the primary objective, achieving 45.…
Kog AI launched a tech preview of its Kog Inference Engine (KIE), achieving 3,000 output tokens per second per request on 8× AMD MI300X GPUs and 2,100 on 8× NVIDIA H200 GPUs using a 2B model. The comp…