A user reports that their ASRock WRX80D8-2T server with an AMD Threadripper PRO 3995WX and five AMD Radeon Pro W7900 GPUs idles at about 350W, costing roughly $30 CAD monthly with 96% of power from id…
A developer released smol-llm-proxy, a minimalist API proxy for self-hosted llama.cpp setups that routes across multiple llama-server instances with per-user API keys and token usage tracking, in appr…
Developer launches role-model, a routing protocol and runtime for hybrid local/cloud AI, enabling deterministic request routing with fallback to a controller model. The system assigns domains and role…
A developer built llama-dash, a dashboard and logging proxy for self-hosted local LLM inference stacks. It proxies OpenAI/Anthropic-compatible endpoints, logs requests with token counts and cost estim…