FreeToken, an open-source inference engine developed by UC Berkeley researchers including Shuo Yang and Xiaoze Fan, reports decoding DeepSeek-V4-Flash, a 284B-parameter Mixture-of-Experts model, at 22…
FreeToken, a new inference engine from FlashML, lets users run frontier models on gaming GPUs at interactive speeds, with Qwen3.6 35B running on an 8GB RTX 4060 laptop at 39 tokens per second, DeepSee…
LM Studio's local LLM performance hinges on whether users run MLX on Apple Silicon Macs or GGUF on Windows GPU laptops, with Apple's unified memory enabling larger models while discrete GPUs like the …
A benchmark comparison of seven quantized Qwen3.8-27B checkpoints on an RTX 5090 found the cyankiwi/Qwen3.8-27B-AWQ-INT4 quant to be the most accurate, with only 156 top-1 token flips (2.193%) across …
A user asks whether a 64GB Strix Halo 395 machine is useful for local LLM inference alongside an RTX 5090, noting the main advantage is loading models larger than 32GB into memory despite slower infer…
A developer's guide explains how to calculate VRAM requirements for running local large language models, emphasizing the importance of quantization. The post provides a simple equation and tables show…
Adore LLC released Balto Speedrunner, a Windows app that turns Qwen 3.8 27B into a local coding agent for a single RTX 5090, achieving up to 300 tok/s on chat prompts and 150 tok/s during coding runs.…
A developer benchmarked the Qwen3.8-27B model with NVFP4 quantization and multi-token prediction (MTP) on an RTX 5090, comparing tier configurations and sweeping spec-draft-n values at 64K context. Th…
Nvidia has raised the price of its RTX Pro 6000 96GB Blackwell workstation GPU to $16,000, a 110% increase from its $7,600 preorder price, due to an ongoing memory shortage driven by AI demand. The co…
Meta Superintelligence Labs released Muse Glimmer, a 30-billion-parameter dense open-agentic model under Apache 2.0, designed for local agents with multi-step reasoning, tool calls, image understandin…
Nvidia has raised the price of its flagship RTX Pro 6000 workstation GPU to $16,000, nearly double its $8,565 launch price from March 2025 and a 21% increase since June when it cost $13,250. The price…
Meta released Muse Glimmer 30B on August 10, 2026, an open-weight model under Apache 2.0 designed for local AI agents, fitting on a single consumer GPU. It reached #1 on Hacker News with over 1,000 po…
Meta has released Muse Glimmer-30B, a dense 30-billion-parameter model designed for on-device agentic AI, diverging from the prevailing Mixture-of-Experts trend. The model activates all parameters per…
Meta released Muse Glimmer, a 30-billion-parameter multimodal agentic model distilled from Muse Spark, under the Apache 2.0 license, designed to run on a single consumer GPU or Mac with no network cal…
Meta's Superintelligence Labs released Muse Glimmer, a 30B-parameter local AI model, which outperforms Google's Gemma4-31B and Alibaba's Qwen3.6-27B on most benchmarks, including a 75.5 score on MCP A…
Meta released Muse Glimmer, a 30-billion-parameter open-weight model for local AI agents, available now on HuggingFace under Apache 2.0. The model, from Meta Superintelligence Labs, targets developers…
MiniMax H3, an open-weights AI video generator from Hailuo (MiniMax), can now run locally on GPUs ranging from an RTX 2060 with 6GB VRAM to an RTX 5090, with community users reporting generation times…
A 15-year-old developer from Russia built Qxern-v6, a hybrid system that compresses code into 32 latent tokens using a Qwen2.5-Coder-1.5B model and a Q-Former, with a deterministic AST sidecar to rest…
MiniMax AI released MiniMax-H3, an open video generation model, with day-0 serving support in SGLang Diffusion, matching Seedance 2.0 at one-third the cost or free locally on 2x RTX 5090 or 1 RTX Pro …
A RAM shortage has doubled memory prices in six months and forced Apple to discontinue the 64GB Mac Mini and 128GB Mac Studio, making local AI hardware significantly more expensive. A 32GB memory kit …