llama-server

While llama.cpp's router mode allows loading and unloading individual models via HTTP API calls to `/models/unload`, there is no built-in "unload all" endpoint. The recommended approach for unloading …

Forge is a reliability layer for self-hosted LLM tool-calling that uses guardrails and context management to dramatically improve performance on multi-step agentic tasks, boosting an 8B model from 53%…

This article presents benchmark results comparing the performance of a Qwen3.6 model running in standard mode versus with Multi-Token Prediction (MTP) enabled. The MTP configuration with a draft of 3 …