LocalJev, a TypeScript server for Bun 1.2+, implements a Jev-compatible POST /v1/systemone API backed by the DiffusionGemma model diffusiongemma-26B-A4B-it-4bit through an OpenAI-compatible Chat Compl…
A prefill/decode disaggregation setup running DeepSeek-V4-Flash — a 284B total / 13B active model with 256 routed experts — bridged a 700,630-token cold prompt end to end in 11 minutes 26 seconds over…
A developer running five local LLMs on a 48GB M4 Pro Mac mini built a 200-line Python router to swap models because macOS's Metal cap of 37.44GB and oMLX's 90% prefill guard left only about 33GB usabl…
A developer detailed their local AI setup using oMLX and Qwen models on an M5 Max Mac with 128GB unified memory, achieving 82-123 tok/s decode with a 35B-A3B MoE worker versus 15.5-17.8 tok/s for a 27…
Retort, a framework for comparing coding stacks, reveals that a model's performance depends on the entire stack—language, weights format, serving engine, agent, context engine, sampling, and prompt—no…
A developer named Vilius Vystartas built a local coding agent on a Mac Mini M4 with 24 GB RAM. The setup uses Pi, a coding agent connected to oMLX, an OpenAI-compatible local LLM server, for file edit…
DeepSeek TUI, a terminal user interface for DeepSeek's models, has gained nearly 33,000 GitHub stars in two weeks. The tool can run against other providers including local models via Ollama or oMLX, a…