{"slug": "the-tokens-you-have-to-keep-yourself", "title": "The Tokens You Have to Keep Yourself", "summary": "A developer's report on the Rust-based terminal coding agent plank reveals that running the DeepSeek V4 Flash engine in-process turns prompt caching from a provider-side discount into a local data structure requiring manual identity computation and invariant preservation, with no error channel—mistakes silently trigger full recomputation and slower responses. The author chose to port a C agent to Rust to link the engine directly, accepting this caching overhead as a deliberate trade-off for tighter integration.", "body_md": "Member-only story\n\n# The Tokens You Have to Keep Yourself\n\nWhen you call a hosted model, the cache is somebody else’s data structure. You send text, a provider tokenizes it, and if a prefix of your request matches something it still has warm, you get billed less. You never see the tokens. You never learn where the reuse boundary fell. The whole mechanism is a discount that shows up on an invoice.\n\nRun the model in your own process and that discount becomes a data structure you maintain. It has an identity you have to compute, an invariant you have to preserve on every turn, and no error channel at all: when you get it wrong it does not fail, it just recomputes everything and hands you the right answer slowly. This is a report on what that turned out to cost.\n\nThe program it comes from is [plank](https://github.com/aovestdipaperino/plank), a terminal coding agent written in Rust that links the DeepSeek V4 Flash engine directly rather than talking to a server, so the model runs inside the same process as the agent driving it. That arrangement is the whole reason any of this is my problem, and it was a deliberate choice.\n\n## Why port a C agent instead of extending a Rust one\n\nThe obvious path was already open. [Eugene](https://levelup.gitconnected.com/building-ai-agents-in-rust-part-1-2fa195fb8b33), the agent from the series, had grown a [provider that called straight into the DeepSeek V4 Flash engine](/building-ai-agents-in-rust-part-11-67086b5bdbf1) through its C header, no subprocess and no pipe. Once you are in…", "url": "https://wpnews.pro/news/the-tokens-you-have-to-keep-yourself", "canonical_source": "https://pub.towardsai.net/the-tokens-you-have-to-keep-yourself-9220e47ad55b?source=rss----98111c9905da---4", "published_at": "2026-07-31 05:58:06+00:00", "updated_at": "2026-07-31 06:41:01.530467+00:00", "lang": "en", "topics": ["developer-tools", "artificial-intelligence"], "entities": ["plank", "DeepSeek V4 Flash", "Eugene"], "alternates": {"html": "https://wpnews.pro/news/the-tokens-you-have-to-keep-yourself", "markdown": "https://wpnews.pro/news/the-tokens-you-have-to-keep-yourself.md", "text": "https://wpnews.pro/news/the-tokens-you-have-to-keep-yourself.txt", "jsonld": "https://wpnews.pro/news/the-tokens-you-have-to-keep-yourself.jsonld"}}