Multiple developers have released IQ3 and IQ4 quantizations of the Qwen3.8-27B large language model in GGUF format, with versions from unsloth, FenomAI, and others. These quantized models reduce the m…
A 14-hour benchmark on a single NVIDIA GeForce RTX 3090 found that the Qwen3.8-27B hybrid SSM+attention model achieves a 131K context window on 24 GB VRAM, scoring 20/21 on a frontier test set, but cr…
An engineer shared a quality-first llama.cpp configuration for running the Qwen3.8-27B model on a single RTX 4090 24GB GPU, achieving 60–70 tokens per second. The setup requires the BeeLlama fork beca…
Alibaba's Qwen team released Qwen3, a new large language model available in sizes including a 27B version, with official BF16 and FP8 weights on Hugging Face and community GGUF and MLX quantizations f…
A developer created llambda.lisp, a Common Lisp implementation of llama.cpp for running LLM models locally. The project is available on GitHub and seeks contributions for additional architecture suppo…
NVIDIA released the Nemotron-3-Ultra-550B-A55B-NVFP4 model, a 550-billion-parameter large language model with 55 billion active parameters using NVFP4 quantization, under the OpenMDW-1.1 license. The …
Qwen 3.6 27B, a dense local language model from Alibaba's Qwen team, impresses developers with its general intelligence and practical coding abilities, running efficiently on consumer hardware via lla…
A developer created llamacpp.sh, a Bash script that automates local LLM inference using llama.cpp and Podman. The script probes available VRAM from sysfs and selects an appropriate model and configura…
A developer released Pi, a coding agent that runs entirely on a local machine without API keys or cloud dependencies. Pi works with a local llama.cpp server hosting Google Gemma 4 models, enabling off…
A developer has published a guide for running Google's Gemma-4 12B instruction-tuned model on Windows Subsystem for Linux 2 (WSL2) using the llama.cpp framework. The process involves installing build …
A developer injected the full text of Marcel Proust's *Swann's Way* into the prompt of a heavily quantized Qwen 9B 3.5 model to test where local language models break under noise during coding tasks. …
The article provides instructions for running Claude Code using a local large language model (LLM) instead of Anthropic's cloud-based models. It recommends downloading specific quantized Qwen3.6 model…