Privatemode researchers Johannes Hötter and Marko Rosenmüller demonstrated that the off-the-shelf LLM GLM-5.3-Flash can make typed decisions with a probability for every option in a single forward pas…
Xiaomi released MiMo-V2.6-Flash-RL, a sparse Mixture-of-Experts omnimodal model with 309 billion total parameters and 15 billion active per token, positioned as the efficiency-focused sibling to MiMo-…
Edge0 released Audio8 ASR Infinite, an open-weight streaming speech recognition model under the Apache 2.0 license, available on Hugging Face as Edge0/Audio8-ASR-Infinite and on GitHub as Edge0-AI/Aud…
A developer benchmarked Google's Gemma 4 E2B quantization-aware trained (QAT) checkpoint against the full bf16 release on a single NVIDIA L4 SageMaker endpoint, finding the 4-bit QAT model decodes at …
A developer published a step-by-step guide and open-source repository for deploying Google's Gemma 4 E2B model to an Amazon SageMaker real-time endpoint on a single NVIDIA L4 GPU using AWS's vLLM cont…
Researchers introduced ServingStudio, an integrated workbench that combines a Rust-based LLM serving simulator with an agent to automate performance profiling and optimization. The ServingStudio Simul…
AMD reported that UltraQuant, its 4-bit KV-cache method for the Qwen3.8-2.4T MXFP4 model on AMD Instinct MI355X GPUs, serves 29% more tokens per second at 24% lower per-token latency than the fastest …
AMD detailed a local quantization workflow using AMD Quark 0.12.post1 that compressed the Qwen3.6-35B-A3B Mixture-of-Experts model from BF16 to W4A16 on a single Strix Halo system, cutting model weigh…
Xiaomi's MiMo team released MiMo-V2.6-Pro-RL, an open-weight trillion-parameter mixture-of-experts agentic model with 1.02 trillion total parameters, 42 billion active per token, and a 1 million token…
A developer benchmarked a typed-decision API (Jev), a 35B mixture-of-experts LLM (Qwen3.5-35B-A3B), and a 421M open-weight model (Laya) on 12,000 real U.S. federal IT solicitations, finding Jev led pr…
BottleCap AI released ThinkingCap-Qwen3.8-27B, a fine-tune of Qwen3.8-27B that cuts thinking tokens by an average of 37.2% across 12 benchmarks while macro-average accuracy falls 0.86 percentage point…
A developer benchmarked a Jev-style decision read of Gemma 4 on a single Google Cloud TPU v6e chip with vLLM, finding that the E2B, E4B and 12B checkpoints fit at bf16 while the 26B-A4B fp8 build also…
Developer longyijdos released Kana, an open-source, single-binary terminal AI agent that runs with zero telemetry and keeps all configs, sessions, memory, artifacts and logs on the user's machine with…
VLLM is an open-source large language model serving engine built around PagedAttention, which manages the KV cache the way an operating system manages virtual memory, and continuous batching, which ke…
A developer who built InsNet, a ~21,000-line C++14 deep learning library with 4,828 lines of hand-written CUDA kernels, starting in 2018, describes how its padding-free dynamic batching design anticip…
Developer nwjang released psychosis-guard, an open-source, model-agnostic safety middleware that tracks cumulative risk across long LLM conversations and escalates graduated interventions when a conve…
A developer detailed a year-long setup for running coding agents like Claude Code and Codex CLI on real products, built around a single 32-core, 128 GB Linux box reached over Tailscale and managed wit…
Parity engineers ran a self-hosted inference trial of the open-weight DeepSeek V4 Flash model that handled more than 144,000 requests and almost 12.9 billion tokens from 25 engineers between 16 August…
Qwen Code, the Qwen team's open-source terminal coding agent, shipped an update and now lists 28k GitHub stars, with its last update recorded in September 2026. The TypeScript project, licensed Apache…
A latent-GRPO training step for the Qwen 3.8 27B model dropped from a median of 36 minutes on a single H100 to 2.6 minutes on one 8× NVIDIA B300 node, a 14× speedup, according to the training team's a…