A developer calculated the actual VRAM requirements for running Llama 3 8B and Gemma 2 9B locally, revealing that the KV cache can consume far more memory than the model weights, especially at longer …
OpenRouter launched Fusion, a routing layer that sends a single prompt to multiple AI models in parallel and synthesizes their outputs, achieving performance comparable to Anthropic's Claude Fable 5 a…
A Hacker News thread on June 16 revealed that local LLMs like Qwen 3.6 35B-A3B handle about 70% of daily coding tasks but fall short on complex multi-file reasoning, creating a gap akin to a junior ve…
A Hacker News thread reveals that local AI coding models, led by Qwen 3.6 35B-A3B and harness Pi, are increasingly replacing cloud-based tools like Claude and GPT, offering privacy, zero cost, and off…
Hacker News users discuss replacing cloud-based AI coding assistants like Claude and GPT with local or self-hosted models, sharing setups and performance metrics. Some report success with models like …
The KV cache, a memory store for attention keys and values, grows linearly with context length and can exceed model weights in VRAM usage, causing out-of-memory errors for local LLM users. At 32k cont…
Researchers introduced Still, a per-layer Perceiver model that compacts KV cache in a single forward pass, enabling efficient long-context language model deployment. On Qwen and Gemma models, Still ou…
Open-weight local language models in mid-2026 have nearly matched frontier performance for everyday tasks, driven by engineering advances in sparse attention and mixture-of-experts architectures. Deep…
Google released Diffusion Gemma in early 2025 as its first open-weight diffusion language model, using a masked diffusion approach that generates text by starting with noise and iteratively refining i…
DeepMind released DiffusionGemma, a new large language model that uses discrete diffusion to generate entire sequences in parallel instead of autoregressive token-by-token generation. The model achiev…
Researchers from the SPAR Research Fellowship found that Google's Gemma 4 language model shows significantly reduced emotional instability compared to its predecessor, Gemma 3, which frequently exhibi…
Local large language models have nearly caught up to frontier models for everyday tasks as of mid-2026, driven by engineering advances in sparse attention and mixture-of-experts architectures that red…
Google released DiffusionGemma, a new open-weight AI model under the Apache 2 license, available on Hugging Face. NVIDIA is hosting the model for free on its NIM cloud API, where it generated 2,409 to…
A developer updated a homelab's local LLM stack, catching up llama.cpp by 469 builds and upgrading the Qwen generation model from 3.5 to 3.6, the embedding model from nomic v1.5 to v2-moe, and adding …
A developer compiled a list of 10 open-source GitHub repositories that collectively replace over $123 per month in paid developer tools, including alternatives to Zapier, Calendly, and DocuSign. The r…
Google released the Google AI Edge Gallery for macOS, enabling Mac users to run Gemma models locally. The company also launched a new Gemma 4 12B variant and the Google AI Edge Eloquent dictation app …
A new open-source benchmark, "apple-silicon-llm-bench," reveals that Google's LiteRT-LM runtime outperforms MLX-Swift on the iPhone 17 Pro for Gemma 4 E2B inference, achieving 55.4 tok/s with 4.5x les…
Google launched Google AI Edge Gallery for macOS, allowing Mac users to run Gemma models locally on their devices. The company also released the Gemma 4 12B model, a multimodal AI with 12 billion para…
Konversio launched Pilot, an open-source AI customer support agent licensed under MIT, designed for self-hosting and deployment in EU data centers. The tool reads help articles, answers chat queries, …
TensorSharp, a new open-source C# inference engine, now enables developers to run large language models locally using GGUF files. The engine supports multiple model architectures including Gemma 4, Qw…