Why output caching doesn't work Caching LLM outputs is not worth the effort because only about 6% of output tokens are visible to users, according to an analysis by software engineer Sean Goedecke. Goedecke estimates that 50% of output tokens are reasoning tokens and 44% are tool calls, so even saving two-thirds of visible output would cut total output tokens by just 4% under Amdahl's law. The real savings lie in reducing the 94% of invisible output tokens, which depends on model intelligence: a model 5x more expensive per token can be cheaper overall if it reaches the right answer with 6x fewer tokens. LLM inference economics depends on caching