Member-only story
A provider with no socket #
Every provider Eugene has spoken to so far, going all the way back to Part 6, ends the same way: a URL, a header, a JSON body over HTTP. Anthropic’s Messages API, OpenAI’s Chat Completions, and even Ollama running on the same laptop as the agent all get the same treatment, because the Provider
trait was built around one assumption: somewhere, there is a socket. Ollama already narrows the distance to zero latency-wise, but the shape of the call is still “make an HTTP request to localhost and wait.” This closing post asks what happens when you drop that assumption entirely and drive a local model the way you’d drive any other child process: stdin in, stdout out, no port to bind, nothing to `curl`
while it’s still warming up.
The model behind this post is DwarfStar, Salvatore Sanfilippo’s local inference engine for DeepSeek V4 Flash and PRO. It is deliberately narrow: not a GGUF runner for any model you throw at it, a purpose-built engine for one model family, tuned for Metal, CUDA, and ROCm, with native tool calling, disk-backed KV caches, and even a distributed mode that splits layers across two Mac Studios over Thunderbolt. It ships two binaries. ds4-server
speaks OpenAI, Anthropic, and Responses-shaped HTTP, which means the Part 6 OpenAiProvider
already works against it with nothing more than a base_url
…