Frink, a pure-Rust inference engine for GGUF models created by developer Antonello F, now runs 100 architectures on its generic path with evidence, each verified against llama.cpp's own logits via lib…
A new learning roadmap breaks LLM inference into five layers — the call itself, transformer computation, hardware limits, optimizations, and serving engines — explaining that prefill builds the KV cac…
Ferrox v0.23.0, released today, adds support for PrismML's Ternary-Bonsai-2-27B, a 27-billion-parameter model quantized to 1.75 bits per weight that occupies 5.5 GB on disk and runs on a 16 GB laptop,…
Cognitora released v0.9.1 in September 2026, adding per-model inflight caps on the gateway, fleet tokens_per_watt gauges, and an energy benchmark harness to its open-source LLM inference orchestration…
Developer Antonello F. released gitgui, a Rust binary that renders a full git GUI—commit graph, staged file list, inline diffs, and hunk buttons—inside terminal panes using the kitty graphics protocol…
Frink, a pure-Rust GGUF inference engine written by Antonello Frink, reached parity with llama.cpp on Apple Metal decode and passed it on prefill at version v0.12.0, with OLMoE-1B-7B prefill rising fr…
Ferrox v0.9.1, a pure-Rust GGUF inference engine, shipped today with MoE prefill on Apple Metal 2.4x faster, closing the gap to llama.cpp from 2.62x behind to 1.11x on OLMoE-1B-7B. The update also imp…
Ferrox, a pure-Rust GGUF inference engine, now runs mixture-of-experts (MoE) prefill on Apple Metal 2.4x faster, reaching 1402 tok/s on OLMoE-1B-7B and closing the gap to llama.cpp from 2.62x behind t…
Developer Antonello F. released Ferrox, a pure-Rust inference engine that runs open LLMs locally on CPU, Apple Metal, or CUDA, achieving performance parity with llama.cpp on an Apple M2 Pro: 26.9 tok/…
Rondine, an open-source hardware-aware control plane, detects the host machine, recommends fitting models, builds an optimized serving plan, downloads weights, and starts an OpenAI-compatible local se…
Rondine, a tool that detects hardware and configures inference engines, enables running GLM-5.2, a 744B-parameter Mixture-of-Experts coding model requiring approximately 245GB of memory, locally on a …