A developer's final part of a CUDA/ROCm optimization series details implementing Mixture of Experts (MoE) inference, including a custom top-k gating kernel, expert parallelism with All-to-All communic…
A developer's guide to self-hosting LLMs on budget VPS plans in 2026 finds that quantized 3B and 7B models are now viable on servers under $6 per month, with RAM bandwidth as the key bottleneck. The a…
Mixture of Experts (MoE) architectures enable 1-trillion-parameter large language models by activating only a subset of expert networks per token, achieving 8× parameter count with only ~2× compute. M…
A new benchmark suite called JAXBench, comprising 50 JAX workloads for AI-generated kernel optimization on Google Cloud TPUs, shows that target-specific documentation context matters more than model s…
Mixture of Experts (MoE) is a machine learning architecture that divides a model into specialized subnetworks called experts, with a gating network routing each input token to only the most relevant s…
Petals, a project from the BigScience research workshop, lets users run large language models like Llama 3.1 (up to 405B parameters) at home by distributing model parts across a peer-to-peer network, …
A new explainer on QAInsights breaks down the Mixture of Experts (MoE) architecture, which powers models like Mixtral, DeepSeek, and GPT-4o, enabling them to scale to trillions of parameters while mai…
Moonshot AI's Kimi K3, a 671-billion-parameter mixture-of-experts model, is the first Chinese open-weight model to match Western frontier AI, achieving benchmark scores competitive with GPT-4o-class m…
Thinking Machines Labs, founded by former OpenAI CTO Mira Murati, released its first open-weight multimodal AI model, Inkling, a 952-billion-parameter Mixture of Experts system that processes both tex…
A developer published a visual and practical guide to large language models, explaining that they are computer programs trained on massive text datasets to predict the next word. The guide covers how …
Tencent released Hunyuan-3, a 295-billion-parameter mixture-of-experts language model that activates only 21 billion parameters per token, designed specifically for agentic tasks such as tool calling …
Open-weight LLMs like Meta's Llama 3, Mistral's Mixtral, and Google's Gemma are democratizing AI access. A developer shows how to integrate these models via API using the OpenAI-compatible spec, with …
Mixture-of-Experts (MoE) models like Qwen3-30B-A3B and DeepSeek-V3 separate total parameters (memory) from active parameters (compute), allowing a 30B-parameter model to run at the speed of a 3B model…
Zhipu AI released GLM 5.2, a 744B parameter open-weight mixture-of-experts model that rivals Claude Opus in coding and visual design quality at lower inference cost. The model's MoE architecture activ…
A developer explains the sparse Mixture of Experts (MoE) architecture used in models like Mixtral, DeepSeek-MoE, and Grok-1, detailing how the router selects which experts to activate per token and wh…
Amalgafy Labs has achieved 21+ tokens per second inference on the Mixtral 8x7B model using only CPU and SSD storage, bypassing the need for GPU VRAM. The team's Micro-Expert-Router (MER) system levera…
A security engineer at a mid-size fintech company faced a 48-hour deadline from the Italian Data Protection Authority to list all AI systems processing EU resident data, including foundation models, t…
A developer built Micro-Expert-Router, a Rust inference engine that streams Mixture-of-Experts model weights directly from NVMe SSDs using io_uring with O_DIRECT, eliminating the need for GPU VRAM. Th…
Here is a 2-3 sentence factual summary of the article: The article argues that for local agent workloads like multi-turn tool use and code editing, Google's dense 31B-parameter Gemma 4 model is a bet…