Caching in Gemini 4 Argon has some quirks

Induction AI's testing of Gemini 4 Argon found Google's new model caches prompts predictably at the documented 4096-token threshold, unlike Gemini 3.1 Pro Preview, which the company found dependable only after 9000 tokens. The tests also surfaced a quirk: Gemini 4 Argon places a second cache breakpoint 2049 tokens before the end of every request and requires paying for at least 2049 tokens, so a 4100-token prompt followed by 1000 new tokens caches only 2051 tokens and bills for 3049. The prompt cache offers a 95% discount on previously seen tokens, making the breakpoint behavior significant for context-optimization costs.

At Induction AI, we’re building context optimization to make AI agents more efficient. Our service reduces inference costs while preserving task accuracy without requiring changes to the agents themselves. To accomplish this, we spend a lot of time understanding the behaviors of the caching systems in the models. Recently, Gemini 4 Argon was released https://cloud.google.com/blog/products/ai-machine-learning/gemini-4-argon-available-on-google-cloud and we were curious to see how the caching behavior had changed from previous models. In particular, we saw many quirks with caching in Gemini 3.1 Pro Preview and were hoping Gemini 4 would be more predictable. The more predictable it is, the better we can optimize our context management layer. Fortunately, Gemini 4 Argon does seem to be more predictable, though it is not without its own quirks. The basics The prompt cache works by re-using what has already been sent before. If you only ever append to a thread, you will get a discount on the tokens that the provider has seen before. Like the recent changes to OpenAI and Anthropic, this discount is significant: 95% https://blog.google/innovation-and-ai/models-and-research/gemini-models/gemini-4-argon/ off of the normal token price. So it is pretty important to get right. We tested the caching behavior by appending to a thread to see where it starts hitting. On Argon, it worked right at the documented https://ai.google.dev/gemini-api/docs/caching 4096 token threshold. That is to say if you are sending 4000 tokens on step 1, there will be no caching on the next turn. However, if you send 4100 tokens, there will be caching on the next turn. This is great to see because the 3.1 Pro Preview behavior varied wildly. In our tests, Gemini 3.1 Pro Preview was only dependable after 9000 tokens. The 2049 quirk Like the newer models from OpenAI and Anthropic, the caching behavior seems to operate on the notion of breakpoints. This means that there are explicit points on each request and that is where it can re-use data on subsequent requests. The implicit breakpoint on OpenAI, for example, is basically at the end of each request. So what actually gets cached when we send 4100 tokens and then append the next messages with say, 1000 more tokens? OpenAI does what we would expect: it caches the 4100 from the previous call and we would pay full price for the new 1000 tokens. On the next turn of 1500 tokens, the same things happens and you only pay for the new tokens. Gemini 4 seems to set its breakpoints differently. It sets it at the end of the request just like other providers. But it also sets another breakpoint 2049 tokens before the end of every request. This is new behavior from any model we have tested so far: counting backwards from the end instead of blocks from the beginning. On top of that, there looks to be a requirement that we always pay for at least 2049 tokens. What this means is that, if the new content is only 1000, then we reach back into the previous request. In this case, it only hits the cache of 2051 4100-2049 and we pay for 3049 2049+1000 . This continues even on the next request. If we send 1500 more tokens, it will go back to the -2049 breakpoint from the previous turn. One thing we do not understand is why it did not use the original 4100 breakpoint in the thirs request. This suggests that it was removed when it was not immediately put into use or only the ones from the last turns are put into use. However, if the second message is more than 2049 tokens 2500 , it will cache the entire first message. That means we pay for 2500 and 4100 is cached. If the next message does not hit the 2049 amount, it will again reach backwards into the previous request. See the code that shows this behavior here https://github.com/induction-ai/experiments/blob/main/experiments/gemini4 cache/src/exact-4100.ts . How this affects Gemini use It is great to see some more predictable behavior, but this behavior makes Gemini 4 Argon pretty inefficient if the messages are short by their nature because there is a minimum of 2049 tokens that we have to pay for. If the messages consistently exceed that threshold, then it will work as expected. This creates the counter-intuitive situation where there are cases where it is better to send more tokens in order to hit these breakpoints. At Induction, we do this kind of research to learn things like this and automatically make our customers more efficient. We will be looking at our context management layer and its benchmarks to verify that this does so.