A developer built a fully offline voice assistant on a Raspberry Pi 4 or 5 using local AI models. The device records audio, processes it with Whisper for speech-to-text, runs a local language model vi…
A reproducible benchmark of 24 local, self-hosted, and cloud LLMs for translation into English found that a local 18 GB model (gemma-4-12b-qat) performs on par with frontier cloud models on Afrikaans→…
Modal and Z Lab released DFlash, a speculative decoding model for Qwen 3.5 397B-A17B, achieving over 4.3x throughput versus baseline and 1.5x versus MTP on HumanEval at concurrency 1. The model uses a…
A developer fixed a broken fallback chain in their OpenClaw agent that was causing request timeouts during peak hours. The new chain includes seven entries: two local Ollama models, three OpenRouter f…
Researchers developed an ensemble of large language models, including Google's Gemini and Gemma, to automatically identify EQ-5D health-related quality-of-life studies in PubMed from abstracts. The we…
Google released OpenRL, an open-source API for fine-tuning large language models on Kubernetes clusters, aiming to decouple infrastructure from AI research and improve GPU utilization by running multi…
Ollama, an open-source tool for running large language models locally, offers a beginner-friendly setup for 2026 with privacy, cost savings, and data control. Users can install it on macOS, Windows, o…
A developer calculated the actual VRAM requirements for running Llama 3 8B and Gemma 2 9B locally, revealing that the KV cache can consume far more memory than the model weights, especially at longer …
OpenRouter launched Fusion, a routing layer that sends a single prompt to multiple AI models in parallel and synthesizes their outputs, achieving performance comparable to Anthropic's Claude Fable 5 a…
A Hacker News thread on June 16 revealed that local LLMs like Qwen 3.6 35B-A3B handle about 70% of daily coding tasks but fall short on complex multi-file reasoning, creating a gap akin to a junior ve…
A Hacker News thread reveals that local AI coding models, led by Qwen 3.6 35B-A3B and harness Pi, are increasingly replacing cloud-based tools like Claude and GPT, offering privacy, zero cost, and off…
Hacker News users discuss replacing cloud-based AI coding assistants like Claude and GPT with local or self-hosted models, sharing setups and performance metrics. Some report success with models like …
The KV cache, a memory store for attention keys and values, grows linearly with context length and can exceed model weights in VRAM usage, causing out-of-memory errors for local LLM users. At 32k cont…
Researchers introduced Still, a per-layer Perceiver model that compacts KV cache in a single forward pass, enabling efficient long-context language model deployment. On Qwen and Gemma models, Still ou…
Open-weight local language models in mid-2026 have nearly matched frontier performance for everyday tasks, driven by engineering advances in sparse attention and mixture-of-experts architectures. Deep…
Google released Diffusion Gemma in early 2025 as its first open-weight diffusion language model, using a masked diffusion approach that generates text by starting with noise and iteratively refining i…
DeepMind released DiffusionGemma, a new large language model that uses discrete diffusion to generate entire sequences in parallel instead of autoregressive token-by-token generation. The model achiev…
Researchers from the SPAR Research Fellowship found that Google's Gemma 4 language model shows significantly reduced emotional instability compared to its predecessor, Gemma 3, which frequently exhibi…
Local large language models have nearly caught up to frontier models for everyday tasks as of mid-2026, driven by engineering advances in sparse attention and mixture-of-experts architectures that red…
Google released DiffusionGemma, a new open-weight AI model under the Apache 2 license, available on Hugging Face. NVIDIA is hosting the model for free on its NIM cloud API, where it generated 2,409 to…