The MiMo-V2.5 model family introduces Hybrid Sliding Window Attention (Hybrid SWA) to reduce KVCache storage to roughly 1/7 of Full Attention, sparse MoE activation to cut per-token compute, and multi…
Lazarus V5 Active Steering protocol, using Grounded Entropy framework, achieves a 146% increase in reasoning scores and 57% reduction in inference latency on quantized MoE models, eliminating the need…
NVIDIA released Nemotron-Labs-TwoTower, an open-weight diffusion language model that separates token representation and denoising into two towers, achieving 2.42× higher generation throughput while re…
Researchers introduced a three-phase deep reinforcement learning system for personalized portfolio management that overcomes ticker lock-in, monolithic objectives, and static user models. Phase 1 uses…
LongCat-2.0, a 1.6 trillion parameter Mixture of Experts (MoE) architecture, introduces a hierarchical routing mechanism and hybrid parallelism to scale model capacity while maintaining deployment fea…
A developer patched llama.cpp to improve prompt processing throughput by 20% when using Multi-Token Prediction (MTP) on GPU, recovering performance lost to MTP overhead. The proof-of-concept processes…
A developer traced a hybrid Mamba-Transformer MoE inference run and found that MoE all-to-all collective stalls dominate the tail latency, with a 69x tail ratio, despite dashboards showing 96% GPU uti…
A new dataset, fineset-io/efficient-llm-papers, compiles 1,734 records of arXiv and Semantic Scholar papers on efficient LLM techniques like quantization, LoRA, MoE, and FlashAttention, each quality-s…
Mixture-of-Experts (MoE) architecture allows large language models to use only a fraction of their total parameters for each token, enabling models with 671 billion total parameters to run at speeds c…
A high-performance Expert Parallelism (EP) kernel is essential for running large Mixture-of-Experts (MoE) language models across multiple GPUs, as it handles the dynamic routing of tokens to experts l…
The Framework Desktop is a 4.5-liter mini tower built around AMD's Ryzen AI Max+ 395 "Strix Halo" chip with up to 128 GB of soldered unified memory, designed specifically for running large local LLMs …
Mixture of Experts (MoE) models are reshaping the economics of open-model deployments by reducing GPU inference costs and altering serving stack requirements. The shift toward MoE architectures in 202…