{"slug": "building-ai-agents-in-rust-part-10", "title": "Building AI Agents in Rust - part 10", "summary": "In the tenth installment of a series on building AI agents in Rust, the author explores driving a local model directly via stdin/stdout instead of HTTP, using Salvatore Sanfilippo's DwarfStar inference engine for DeepSeek V4 Flash and PRO, which supports native tool calling and distributed mode.", "body_md": "Member-only story\n\n# 🦀 Building AI Agents in Rust - part 10\n\n## A provider with no socket\n\nEvery provider Eugene has spoken to so far, going all the way back to Part 6, ends the same way: a URL, a header, a JSON body over HTTP. Anthropic’s Messages API, OpenAI’s Chat Completions, and even Ollama running on the same laptop as the agent all get the same treatment, because the `Provider`\n\ntrait was built around one assumption: somewhere, there is a socket. Ollama already narrows the distance to zero latency-wise, but the shape of the call is still “make an HTTP request to localhost and wait.” This closing post asks what happens when you drop that assumption entirely and drive a local model the way you’d drive any other child process: stdin in, stdout out, no port to bind, nothing to `curl`\n\nwhile it’s still warming up.\n\nThe model behind this post is [DwarfStar](https://github.com/antirez/ds4), Salvatore Sanfilippo’s local inference engine for DeepSeek V4 Flash and PRO. It is deliberately narrow: not a GGUF runner for any model you throw at it, a purpose-built engine for one model family, tuned for Metal, CUDA, and ROCm, with native tool calling, disk-backed KV caches, and even a distributed mode that splits layers across two Mac Studios over Thunderbolt. It ships two binaries. `ds4-server`\n\nspeaks OpenAI, Anthropic, and Responses-shaped HTTP, which means the Part 6 `OpenAiProvider`\n\nalready works against it with nothing more than a `base_url`\n\n…", "url": "https://wpnews.pro/news/building-ai-agents-in-rust-part-10", "canonical_source": "https://pub.towardsai.net/building-ai-agents-in-rust-part-10-3c1e2f47b29b?source=rss----98111c9905da---4", "published_at": "2026-07-20 05:53:27+00:00", "updated_at": "2026-07-20 14:43:23.211608+00:00", "lang": "en", "topics": ["artificial-intelligence", "ai-agents", "large-language-models", "developer-tools"], "entities": ["Anthropic", "OpenAI", "Ollama", "DwarfStar", "Salvatore Sanfilippo", "DeepSeek", "Rust"], "alternates": {"html": "https://wpnews.pro/news/building-ai-agents-in-rust-part-10", "markdown": "https://wpnews.pro/news/building-ai-agents-in-rust-part-10.md", "text": "https://wpnews.pro/news/building-ai-agents-in-rust-part-10.txt", "jsonld": "https://wpnews.pro/news/building-ai-agents-in-rust-part-10.jsonld"}}