Build b9437 of llama.cpp fixes two default-value bugs in llama-bench that caused flash attention to be skipped on capable GPUs and GPU-layer count to use a legacy sentinel. The flash attention flag no…
A developer tested five local AI models against Claude Sonnet 4 on a real coding task—building a tag manager for a blog admin panel. Only two models shipped working code: Sonnet 4 and Qwen3-Coder 30B-…
Second-brain-mcp is a self-maintaining personal knowledge database that uses MCP, DuckDB, and biological memory models to automatically link, compress, and index saved papers, notes, and figures. The …
A developer benchmarked the Claude Agent SDK on a local LLM, finding that a 35B-parameter model running on an RTX 3090 Ti can replace Anthropic's Haiku tier in production. The local model achieved par…
Ethereum co-founder Vitalik Buterin detailed his self-sovereign AI setup running open-weight models on personal hardware, achieving up to 90 tokens per second on an Nvidia 5090 laptop. Buterin called …
While llama.cpp's router mode allows loading and unloading individual models via HTTP API calls to `/models/unload`, there is no built-in "unload all" endpoint. The recommended approach for unloading …
Forge is a reliability layer for self-hosted LLM tool-calling that uses guardrails and context management to dramatically improve performance on multi-step agentic tasks, boosting an 8B model from 53%…
This article presents benchmark results comparing the performance of a Qwen3.6 model running in standard mode versus with Multi-Token Prediction (MTP) enabled. The MTP configuration with a draft of 3 …