Developers looking to curb the cost of AI-powered coding tools have increasingly turned to the “Caveman” prompting style, which instructs coding assistants to communicate in blunt, telegraphic language and avoid conversational padding. The theory is simple: fewer words mean fewer tokens, translating into lower inference costs for organizations deploying AI agents at scale.
A new test from IDE maker JetBrains confirms that terse prompting styles such as the viral open-source Caveman project can reduce token usage without hurting coding performance. However, the company found that the savings were far smaller than supporters claim.
JetBrains used the Harbor open-source evaluation framework and tasks from SkillsBench for its test, and found that the Caveman technique reduced usage of output tokens by about 8.5%, far below its claimed 65%.
The IDE-maker ran paired benchmarks across 86 real-world software engineering tasks in Claude Code, comparing coding sessions that used the Caveman prompting style against otherwise identical sessions without it.
While an initial evaluation of just 10 tasks indicated savings to the tune of about 30%, the reduction fell to about 8.5% as the test progressed, JetBrains engineer Denis Shiryaev, wrote in a blog post, suggesting that the Caveman technique’s impact was less pronounced across a broader and more representative workload. The open-source Caveman project suggests that if an agent drops the conversational padding around responses and communicates in terse, telegraphic fragments, the token outputs saved could translate into meaningful savings at scale.
That assumption, according to Shiryaev, does not fully account for how modern coding agents use tokens.
While shorter prompts and responses do reduce the amount of text exchanged with users, the engineer said the bulk of token consumption in agentic coding workflows comes from reading project files, reasoning through tasks, invoking tools and generating code, limiting the overall savings from trimming conversational language alone. Further, the engineer pointed out that translating token savings into lower operating costs may not always be straightforward for enterprises.
Although the Caveman technique, during testing, generally resulted in lower costs on individual coding tasks, the cumulative cost of the full benchmark was higher for the Caveman runs after a single dependency-audit task crossed Claude Code’s long-context pricing tier, Shiryaev pointed out.
That same task had produced a similar cost outlier in an earlier baseline run, indicating that the anomaly reflected the workload rather than the prompting technique itself, Shiryaev added.
However, not all of JetBrains’ findings undercut the Caveman technique.
The test found no detectable impact on task success rates, code quality or execution time, Shiryaev said, suggesting that while the prompting style may not deliver the dramatic token savings claimed by its proponents, it also did not impair the coding agent’s effectiveness.
Beyond simple cost saving, the findings from the test also add nuance to a growing body of prompt-engineering techniques aimed at reducing AI inference costs.
Besides the Caveman project, other approaches, including data analyst Drona Reddy’s Markdown-based prompting technique, have claimed meaningful token savings.
For now, enterprises and their leaders should view such prompt-engineering techniques as optimizations to be validated rather than assumptions to be adopted, with production workloads ultimately determining whether the promised savings materialize.