Moonshot AI released the open weights for its K3 model, a 2.8-trillion-parameter sparse MoE with MXFP4 quantization, ranking #1 on Frontend Code Arena and #3 on the Artificial Analysis Intelligence In…
NVIDIA's NeMo team released Molt, a PyTorch-native agentic reinforcement learning framework designed to be compact enough for researchers and AI coding assistants to grasp, with roughly 8.6K lines of …
AMD's Instinct MI355X GPUs now run Moonshot AI's 2.8-trillion-parameter Kimi K3 model at 952 tokens per second per node, undercutting Nvidia's B300 on cost per token despite Nvidia's 1.65x raw through…
A 180-line roofline model built by an independent developer shows speculative decoding on Llama-3-70B collapses at batch 224, turning a 1.92x speedup into a 0.74x slowdown by batch 512, while the same…
DeepSeek released DeepSeek-V4-Flash-0731 on Hugging Face and moved its V4-Flash API to public beta on July 31, 2026, claiming major gains in agentic and coding benchmarks from re-post-training rather …
A team of four developers won third place at the Dell x NVIDIA AI Hackathon by building Squidward, a 100% air-gapped, self-improving IT firewall that runs locally on a Dell GB10 with a Qwen3.6-27B mod…
A benchmark study by an independent researcher found that burstier request arrivals speed up LLM inference, contradicting standard intuition. The analysis of vLLM serving shows that higher burstiness …
LocalAI's 18 custom C and C++ inference engines, including vllm.cpp and depth-anything.cpp, outperform or match upstream Python-based engines while drastically reducing footprint, with vllm.cpp delive…
An engineer at vLLM analyzed concurrent KV block load coalescing in Mooncake and vLLM, finding that while the code path for single-flight optimization exists, real traces show almost no coalescable wo…
Baidu's vLLM Kunlun plugin, open-sourced on Dec 8, 2025, enables vLLM to run on Kunlun XPU hardware, supporting 20+ models including Qwen, Llama, DeepSeek, GLM, and Gemma4, with features like quantiza…
A technical advisory recommends that production systems using raw LLM responses implement a structured content lifecycle, treating model output as a draft rather than the final artifact. The approach,…
HexCore, a low-latency paged KV cache allocator for LLM inference written in C++20 and CUDA, has been released under the Apache License 2.0 by Rasuljanov Muhammadali. The CPU-side allocator and relate…
A research paper titled 'The Silent Hyperparameter: Quantifying the Impact of Inference Backends on LLM Reproducibility' reveals that the choice of inference engine can significantly affect LLM output…
A new method called Functional Reconstruction improves draft-token acceptance in speculative decoding for multi-head latent attention (MLA) draft models, according to a paper on arXiv (2607.27269v1). …
Ollama's simplicity can mask when a local experiment becomes a shared inference service, and vLLM offers better scheduling and observability for production workloads. Migration is warranted when multi…
On July 27, 2026, Moonshot AI released Kimi K3, a 2.8 trillion parameter Mixture of Experts (MoE) model, described as the first open-weight system to reach the 3 trillion parameter class. The model ac…
Moonshot AI released Kimi K3, a 2.8 trillion parameter Mixture of Experts (MoE) model, on July 27, 2026, making it the first open-weight system to reach the 3 trillion parameter class. The model requi…
Moonshot AI released the full weights and technical report for Kimi K3 on July 27th, documenting a 2.8 trillion-parameter model that activates 104 billion parameters per token. The Beijing developer h…
DigitalOcean launched Kimi K3 on day 0, making it one of the most popular models on the platform and across the market, with the second most likes on Hugging Face and sixth most traffic on OpenCode. T…
Kubeflow 1.11, released at KubeCon Japan, introduces a pip-installable SDK that lets ML practitioners submit distributed training jobs, run hyperparameter searches, and fine-tune Llama 3.2 without wri…