Member-only story
When you call a hosted model, the cache is somebody else’s data structure. You send text, a provider tokenizes it, and if a prefix of your request matches something it still has warm, you get billed less. You never see the tokens. You never learn where the reuse boundary fell. The whole mechanism is a discount that shows up on an invoice.
Run the model in your own process and that discount becomes a data structure you maintain. It has an identity you have to compute, an invariant you have to preserve on every turn, and no error channel at all: when you get it wrong it does not fail, it just recomputes everything and hands you the right answer slowly. This is a report on what that turned out to cost.
The program it comes from is plank, a terminal coding agent written in Rust that links the DeepSeek V4 Flash engine directly rather than talking to a server, so the model runs inside the same process as the agent driving it. That arrangement is the whole reason any of this is my problem, and it was a deliberate choice.
Why port a C agent instead of extending a Rust one #
The obvious path was already open. Eugene, the agent from the series, had grown a provider that called straight into the DeepSeek V4 Flash engine through its C header, no subprocess and no pipe. Once you are in…