A developer found that llama-server's sleep mode (--sleep-idle-seconds) has a race condition: requests arriving just before the idle timer fires either hang in the queue or crash the server with SIGSE…
Return, a document analysis tool for lawyers, replaced Ollama with a bundled llama-server from llama.cpp as a sidecar process after Ollama 0.12 introduced cloud models in September 2025 that proxy req…
Jeva.cpp, a fork of llama.cpp, adds a JEV-compatible decision API to llama-server that enables Choice, Score and Noul evaluations directly from model logits while preserving standard autoregressive ge…
Developer ArtemPodloboshnikov released Mixer, an open-source 3D editor that generates GLB/GLTF models from text prompts using local LLMs via a built-in llama-server. The MIT-licensed Tauri v2 applicat…
A developer published a llama-server configuration that runs a Qwen 3.8 27B NVFP4 model with MTP speculative decoding on an RTX 5090, reporting throughput of 90-120 tokens per second. The setup uses a…
A security analyst studying for CySA+ demonstrated that a local 27B-parameter LLM running entirely on their own GPU could audit and remediate a Splunk and Sysmon home SOC stack without any data leavin…
A user running Qwen3.8-Flash-Next at IQ3 quantization on a system with 64GB of RAM, an AMD 9950X3D, a 9070XT, and a ZFS stripe of three mid-range NVMe drives reported fitting 255K tokens of context us…
A homelab user identified as gessha detailed a home setup that uses two Nvidia RTX 3090 GPUs in an i5-8600K workstation to host Qwen 3.8 27B and Qwen 3-VL 4B models via llama-server, alongside three m…
A technical comparison examines the tradeoffs between Ollama and llama.cpp for local LLM inference, framing the choice as one between a managed model service and a toolkit operated directly. The guide…
A developer published two Rust CLI demos, gemma-rust and gemma-rust-mcp, that query a self-hosted Gemma 4 E2B model through an OpenAI-compatible HTTP endpoint and through an MCP server, respectively. …
Empero-ai released Qwen3.8-27B-Ridge-3.7bpw.gguf, a quantized version of the 27-billion-parameter Qwen3.8-27B hybrid reasoning and vision model that shrinks from roughly 50GB to 11.7GB, enabling it to…
A developer created a simple MCP server for local LLMs to call Rust-coded tools, but encountered CORS errors. The llama-cpp feature ui-mcp-proxy, passed as an argument to llama-server, sets up a proxy…
Cicada 0.4.0, an agentic Python IDE, now features zero-setup first launch with automatic model download and self-healing resilience against crashes and transient failures. The release introduces autom…
Tess-4-27B, a reasoning-native agentic fine-tune on Qwen3.6-27B, achieved 81% (122/150) on the local benchlocal full suite, ranking #1. The model scored 94% on core instruction/tools/structure tasks b…
A practitioner is running Google's Gemma 4 E2B model on a single 4 GB VRAM card to handle screen watching, voice-memo and meeting transcription, and chat simultaneously, consolidating three previously…
Build b9437 of llama.cpp fixes two default-value bugs in llama-bench that caused flash attention to be skipped on capable GPUs and GPU-layer count to use a legacy sentinel. The flash attention flag no…
A developer tested five local AI models against Claude Sonnet 4 on a real coding task—building a tag manager for a blog admin panel. Only two models shipped working code: Sonnet 4 and Qwen3-Coder 30B-…
Second-brain-mcp is a self-maintaining personal knowledge database that uses MCP, DuckDB, and biological memory models to automatically link, compress, and index saved papers, notes, and figures. The …
A developer benchmarked the Claude Agent SDK on a local LLM, finding that a 35B-parameter model running on an RTX 3090 Ti can replace Anthropic's Haiku tier in production. The local model achieved par…
Ethereum co-founder Vitalik Buterin detailed his self-sovereign AI setup running open-weight models on personal hardware, achieving up to 90 tokens per second on an Nvidia 5090 laptop. Buterin called …